唐小鸭andGitHub
9138c24571
fix(site-replication): lift a rejoined site's restarted edit counter over stale fence marks A site removed while unreachable (unilateral removal: the receiver never dropped it from its peer map, so parse_site_replication_state's load-time mark pruning never fired) that later rejoins recreates its state object and restarts edit_generation at zero. The receiver's surviving high-water mark then silently fences out every stamped delivery from that origin — peer edits and the add finalize fan-out alike are acked without applying — until the restarted counter catches up. Allocate the generation as a hybrid logical clock instead: max(wall clock in unix nanoseconds, previous + 1), still inside the state transaction under the distributed state-object lock. Every value a lifetime hands out is capped by the wall clock at its own allocation, so a recreated lifetime's first allocation exceeds them all and clears the stale mark, while a pre-removal delivery still in flight stays below the new floor and remains correctly fenced. previous+1 keeps allocations strictly increasing across same-tick allocations and mid-lifetime clock regressions. Nothing changes on the wire or in the persisted schema: editGeneration stays the single fence param and edit_generation the single counter field, so pre-hybrid receivers get the fix as soon as the sender upgrades, old binaries preserve the field across rolling up/downgrades, and marks recorded by plain-counter receivers (small values) are cleared by any wall-clock allocation. A clock that regresses across a delete/recreate degrades to a fence that self-heals once real time passes the previous lifetime's last allocation, and introduces no rollback window beyond what the plain counter already had. An epoch-based design (editEpoch wire param + per-origin epoch marks) was built first and rejected under adversarial review: old binaries rewriting the state object drop the unknown epoch fields, which both disarms the fix mid-rolling-upgrade and — because epoch adoption lowers the generation mark — reopens the pre-restart rollback the fence exists to prevent; a backwards clock also fences an origin permanently instead of self-healing. The hybrid clock has none of these modes.
RustFS is a high-performance distributed object storage software built using Rust
Getting Started · Docs · Bug reports · Discussions
English | Simplified Chinese
RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, broad S3 API compatibility for supported features, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.
Features
- High Performance: Built with Rust, ensuring speed and efficiency.
- Distributed Architecture: Scalable and fault-tolerant design for large-scale deployments.
- S3 Compatibility: Integration with common S3-compatible applications; current coverage is tracked in the S3 compatibility matrix.
- Data Lake Support: Optimized for big data and AI workloads.
- Open Source: Licensed under Apache 2.0, encouraging community contributions and transparency.
- User-Friendly: Designed with simplicity in mind, making it easy to deploy and manage.
RustFS vs MinIO
Stress test server parameters
| Type | parameter | Remark |
|---|---|---|
| CPU | 2 Core | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |
| Memory | 4GB | |
| Network | 15Gbp | |
| Driver | 40GB x 4 | IOPS 3800 / Driver |
https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a
RustFS vs Other object storage
| RustFS | Other object storage |
|---|---|
| Powerful Console | Simple and useless Console |
| Developed based on Rust language, memory is safer | Developed in Go or C, with potential issues like memory GC/leaks |
| Does not report logs to third-party countries | Reporting logs to other third countries may violate national security laws |
| Licensed under Apache, more business-friendly | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
| S3-compatible core, with coverage tracked in the compatibility matrix | Variable S3 support and local cloud vendor coverage |
| Rust-based development, strong support for secure and innovative devices | Poor support for edge gateways and secure innovative devices |
| Stable commercial prices, free community support | High pricing, with costs up to $250,000 for 1PiB |
| No risk | Intellectual property risks and risks of prohibited uses |
Quickstart
To get started with RustFS, follow these steps:
-
One-click installation script (Option 1)
curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh -
Docker Quick Start (Option 2)
# Docker Hub (recommended)
docker run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
# Alternative using Podman
podman run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
- Access the Console: Open your web browser and navigate to
http://localhost:9001to access the RustFS console, default username and password isrustfsadmin. - Create a Bucket: Use the console to create a new bucket for your objects.
- Upload Objects: You can upload files directly through the console or use S3-compatible APIs to interact with your RustFS instance.
Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.
Getting Help
If you have any questions or need assistance, you can:
- Check the FAQ for common issues and solutions.
- Join our GitHub Discussions to ask questions and share your experiences.
- Open an issue on our GitHub Issues page for bug reports or feature requests.
Links
- Documentation - The manual you should read
- Changelog - What we broke and fixed
- GitHub Discussions - Where the community lives
Contact
- Bugs: GitHub Issues
- Business: [email protected]
- Jobs: [email protected]
- General Discussion: GitHub Discussions
- Contributing: CONTRIBUTING.md
Contributors
RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.
License
RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.