唐小鸭andGitHub
dbf51117a1
* test(replication): expect CopyObject and snowball extract to schedule replication
Red-phase TDD tests for P0-6: CopyObject never consults the bucket
replication config (no pending stamp, no schedule, and the destination
inherits the source's stale replication status metadata wholesale), and
snowball auto-extract members are never scheduled either.
- usecase white-box: observe MUST_REPLICATE_OBJECT_CALLS for
execute_copy_object (currently 0, must be 1) and
execute_put_object_extract (currently 0, must be 2 for a two-member
archive), plus stale replication-status metadata cleanup assertions
(MinIO filterReplicationStatusMetadata parity).
- e2e: CopyObject destination and snowball-extracted members must appear
on the remote replication target and reach COMPLETED on the source.
Red evidence (before fix):
copy_object_computes_replication_decision_and_strips_stale_status
assertion failed: left: 0, right: 1
put_object_extract_computes_replication_decision_per_entry
assertion failed: left: 0, right: 2
* fix(replication): schedule replication for CopyObject and snowball extracted objects
CopyObject and snowball auto-extract never consulted the bucket
replication config: no PENDING stamp, no post-commit schedule, and no
scanner-heal backstop (heal only re-drives Pending/Failed objects, and
these objects carried no status at all). Worse, the copy path cloned the
source metadata wholesale, so a destination object inherited the
source's replication bookkeeping and could present a fake
COMPLETED/REPLICA state.
Mirroring the PUT path (single immutable decision drives both the
pending metadata and the post-commit schedule, rustfs/backlog#1320):
- execute_copy_object: strip the source's replication status metadata
(internal replication/replica status + timestamps under both
compatibility prefixes, plus x-amz-replication-status) for
non-inbound requests — MinIO filterReplicationStatusMetadata parity;
the cleanup runs before the decision so an inherited REPLICA status
cannot suppress it. Then compute must_replicate_object once, stamp
PENDING when it replicates, and schedule after the copy commits and
the self-copy lock guard is released. Inbound replica writes keep
their authorized metadata and are declined inside
must_replicate_object, so replicas are never re-scheduled outbound.
- execute_put_object_extract: same stamp + schedule per extracted
member object (MinIO PutObjectExtract parity).
- execute_put_object dispatch: an authorized inbound replication PUT is
stored verbatim instead of being re-dispatched into the extract path.
Extracted members keep x-amz-meta-snowball-auto-extract in their user
metadata and the replication client replays stored metadata as
headers, so the target used to try to untar each member's own bytes,
permanently failing replication for non-archive members (surfaced by
the new snowball e2e test).
Green evidence:
- copy_object_computes_replication_decision_and_strips_stale_status,
put_object_extract_computes_replication_decision_per_entry (red: 0
decisions; green: 1 and 2), plus the existing PUT/object-lock
decision-count tests stay green.
- e2e test_copy_object_replicates_to_target and
test_snowball_extract_replicates_members_to_target pass against two
live instances.
RustFS is a high-performance distributed object storage software built using Rust
Getting Started · Docs · Bug reports · Discussions
English | Simplified Chinese
RustFS is a high-performance distributed object storage software built using Rust, one of the most popular languages worldwide. Along with MinIO, it shares a range of advantages such as simplicity, broad S3 API compatibility for supported features, open-source nature, support for data lakes, AI, and big data. Furthermore, it has a better and more user-friendly open-source license in comparison to other storage systems, being constructed under the Apache license. As Rust serves as its foundation, RustFS provides faster speed and safer distributed features for high-performance object storage.
Features
- High Performance: Built with Rust, ensuring speed and efficiency.
- Distributed Architecture: Scalable and fault-tolerant design for large-scale deployments.
- S3 Compatibility: Integration with common S3-compatible applications; current coverage is tracked in the S3 compatibility matrix.
- Data Lake Support: Optimized for big data and AI workloads.
- Open Source: Licensed under Apache 2.0, encouraging community contributions and transparency.
- User-Friendly: Designed with simplicity in mind, making it easy to deploy and manage.
RustFS vs MinIO
Stress test server parameters
| Type | parameter | Remark |
|---|---|---|
| CPU | 2 Core | Intel Xeon(Sapphire Rapids) Platinum 8475B , 2.7/3.2 GHz |
| Memory | 4GB | |
| Network | 15Gbp | |
| Driver | 40GB x 4 | IOPS 3800 / Driver |
https://github.com/user-attachments/assets/2e4979b5-260c-4f2c-ac12-c87fd558072a
RustFS vs Other object storage
| RustFS | Other object storage |
|---|---|
| Powerful Console | Simple and useless Console |
| Developed based on Rust language, memory is safer | Developed in Go or C, with potential issues like memory GC/leaks |
| Does not report logs to third-party countries | Reporting logs to other third countries may violate national security laws |
| Licensed under Apache, more business-friendly | AGPL V3 License and other License, polluted open source and License traps, infringement of intellectual property rights |
| S3-compatible core, with coverage tracked in the compatibility matrix | Variable S3 support and local cloud vendor coverage |
| Rust-based development, strong support for secure and innovative devices | Poor support for edge gateways and secure innovative devices |
| Stable commercial prices, free community support | High pricing, with costs up to $250,000 for 1PiB |
| No risk | Intellectual property risks and risks of prohibited uses |
Quickstart
To get started with RustFS, follow these steps:
-
One-click installation script (Option 1)
curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh -
Docker Quick Start (Option 2)
# Docker Hub (recommended)
docker run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
# Alternative using Podman
podman run -d -p 9000:9000 -v /data:/data rustfs/rustfs:latest
- Access the Console: Open your web browser and navigate to
http://localhost:9001to access the RustFS console, default username and password isrustfsadmin. - Create a Bucket: Use the console to create a new bucket for your objects.
- Upload Objects: You can upload files directly through the console or use S3-compatible APIs to interact with your RustFS instance.
Documentation
For detailed documentation, including configuration options, API references, and advanced usage, please visit our Documentation.
Getting Help
If you have any questions or need assistance, you can:
- Check the FAQ for common issues and solutions.
- Join our GitHub Discussions to ask questions and share your experiences.
- Open an issue on our GitHub Issues page for bug reports or feature requests.
Links
- Documentation - The manual you should read
- Changelog - What we broke and fixed
- GitHub Discussions - Where the community lives
Contact
- Bugs: GitHub Issues
- Business: [email protected]
- Jobs: [email protected]
- General Discussion: GitHub Discussions
- Contributing: CONTRIBUTING.md
Contributors
RustFS is a community-driven project, and we appreciate all contributions. Check out the Contributors page to see the amazing people who have helped make RustFS better.
License
RustFS is a trademark of RustFS, Inc. All other trademarks are the property of their respective owners.