8218248000
* fix(hotpath): pin mimalloc allocator backend * test(hotpath): verify mimalloc allocator backend Co-Authored-By: heihutu <[email protected]> * chore(hotpath): document unsafe allocator tests Co-Authored-By: heihutu <[email protected]> * feat(kms): record real cache hit, miss and eviction metrics (#5531) * feat(kms): record real cache hit, miss and eviction metrics The metadata cache reported (entry_count, 0) because moka exposes no hit or miss counts, so the miss half of every cache report was a constant. Track lookups and removals in the cache itself: hit/miss counters on the lookup path, a moka eviction listener classifying removals by cause, and an entry gauge refreshed whenever the entry set changes. The counters are exported through the metrics facade under the rustfs_kms_ prefix with static label values only, matching the operation-policy metrics, and are also returned as a KmsCacheStats snapshot in place of the old tuple. Cache semantics are unchanged: capacity, TTL and invalidation points are the same, and remove now flushes pending maintenance so the gauge and the removal notification describe the cache the caller sees. Refs rustfs/backlog#1584 * fix(kms): report real cache counters through the admin status API KmsStatusResponse.cache_stats mapped the old (entry_count, 0) tuple onto hit_count and miss_count, so operators polling KMS status read the entry count as a hit count and a miss count that was always zero. Map the fields to the counters they claim to be, and add entry_count and eviction_count as additive, defaulted fields so the entry number that hit_count used to carry is still available. Refs rustfs/backlog#1584 * fix(kms): refresh the cache entry gauge on lookup misses The entry gauge was published only from the write paths, so an entry dropped by TTL expiry left `rustfs_kms_metadata_cache_entries` reporting a population that no longer existed until the next put, remove or clear. A cache that goes quiet — entries ageing out with no further writes — kept over-reporting indefinitely. Republish the gauge from the lookup path when the lookup misses. A miss is where expiry surfaces, and moka reaps expired entries in the maintenance it runs during that same lookup, so the count read afterwards reflects the reaping. Hits stay free of the extra work. * docs(kms): correct the entry gauge convergence claim on the miss path The comment on the miss-path gauge refresh said moka reaps expired entries in the maintenance it runs on that same lookup. It does not: `should_apply_reads` is gated on a full read log or an elapsed housekeeping interval, so the removal that decrements `entry_count` and reaches the eviction listener may land on a later lookup. The behaviour and the test are unchanged — the gauge still converges, and the test drives `run_pending_tasks` explicitly rather than riding on that interval. Only the stated guarantee was wrong, so say interval instead of same-lookup and record why forcing maintenance on the read path was not the trade taken. * chore(deps): refresh cargo dependencies Co-Authored-By: heihutu <[email protected]> --------- Co-authored-by: heihutu <[email protected]> Co-authored-by: Zhengchao An <[email protected]>
RustFS is a high-performance distributed object storage software built using Rust
Getting Started · Docs · Bug reports · Discussions
English | Simplified Chinese
RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, broad S3 API compatibility for supported features, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.
Features
- High Performance: Built with Rust, ensuring speed and efficiency.
- Distributed Architecture: Scalable and fault-tolerant design for large-scale deployments.
- S3 Compatibility: Integration with common S3-compatible applications; current coverage is tracked in the S3 compatibility matrix.
- Data Lake Support: Optimized for big data and AI workloads.
- Open Source: Licensed under Apache 2.0, encouraging community contributions and transparency.
- User-Friendly: Designed with simplicity in mind, making it easy to deploy and manage.
RustFS vs MinIO
Stress test server parameters
| Type | parameter | Remark |
|---|---|---|
| CPU | 2 Core | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |
| Memory | 4GB | |
| Network | 15Gbp | |
| Driver | 40GB x 4 | IOPS 3800 / Driver |
https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a
RustFS vs Other object storage
| RustFS | Other object storage |
|---|---|
| Powerful Console | Simple and useless Console |
| Developed based on Rust language, memory is safer | Developed in Go or C, with potential issues like memory GC/leaks |
| Does not report logs to third-party countries | Reporting logs to other third countries may violate national security laws |
| Licensed under Apache, more business-friendly | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
| S3-compatible core, with coverage tracked in the compatibility matrix | Variable S3 support and local cloud vendor coverage |
| Rust-based development, strong support for secure and innovative devices | Poor support for edge gateways and secure innovative devices |
| Stable commercial prices, free community support | High pricing, with costs up to $250,000 for 1PiB |
| No risk | Intellectual property risks and risks of prohibited uses |
Quickstart
To get started with RustFS, follow these steps:
-
One-click installation script (Option 1)
curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh -
Docker Quick Start (Option 2)
# Docker Hub (recommended)
docker run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
# Alternative using Podman
podman run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
- Access the Console: Open your web browser and navigate to
http://localhost:9001to access the RustFS console, default username and password isrustfsadmin. - Create a Bucket: Use the console to create a new bucket for your objects.
- Upload Objects: You can upload files directly through the console or use S3-compatible APIs to interact with your RustFS instance.
Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.
Getting Help
If you have any questions or need assistance, you can:
- Check the FAQ for common issues and solutions.
- Join our GitHub Discussions to ask questions and share your experiences.
- Open an issue on our GitHub Issues page for bug reports or feature requests.
Links
- Documentation - The manual you should read
- Changelog - What we broke and fixed
- GitHub Discussions - Where the community lives
Contact
- Bugs: GitHub Issues
- Business: [email protected]
- Jobs: [email protected]
- General Discussion: GitHub Discussions
- Contributing: CONTRIBUTING.md
Contributors
RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.
License
RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.