Update the workspace s3s pin and refresh the lockfile with cargo update/upgrade.
Remove the unused lifecycle url dependency reported by cargo shear.
Tighten the s3s footprint ratchet to the current observed baseline.
Allow release builds to set RUSTFS_BUILD_VERSION at compile time while keeping the existing tag, short commit, and package-version fallback when the variable is unset or empty.
Co-authored-by: heihutu <[email protected]>
Add an authenticated scanner usage-state reset endpoint that publishes a fenced bootstrap marker for full rebuilds instead of requiring operators to delete usage metadata by hand.
Guard the reset with the scanner leader lock, storage publication epoch, and per-slot revision preconditions, and make startup resumable across stale cleanup leftovers while still rejecting newer conflicting usage state.
Co-authored-by: heihutu <[email protected]>
Co-authored-by: Zhengchao An <[email protected]>
Lower the default scanner cache save timeout so the derived usage persistence budget stays inside the effective distributed publication lease window.
Add focused regressions for the default publication budget and bootstrap-pending observational baselines, and update operator docs with the new default.
Co-authored-by: heihutu <[email protected]>
Align object-capacity refresh defaults with the production-oriented environment values and keep docs, script examples, and tests in sync.
Co-authored-by: heihutu <[email protected]>
fix(ecstore): require verification source for degraded GET
Fail closed when reconstruction has only an exact decode quorum, because no surplus source remains to validate the rebuilt data. Cover both erasure engines and the data-shards-only rollout gate.
Co-authored-by: heihutu <[email protected]>
fix(ecstore): hedge data-only GET with parity
Route the opt-in data-shards-only lockstep path through the bounded parity race and preserve deferred parity reserves across canceled hedges.
Co-authored-by: heihutu <[email protected]>
Use the seqlock version as the publication fence and keep payload reads relaxed while validating the final version. This reduces ordering overhead in disk metric recording and snapshot collection without changing the rolling-window contract.
Co-authored-by: heihutu <[email protected]>
fix(ecstore): classify system metadata volume failures
Preserve retryable quorum errors when system metadata reads or writes encounter missing volumes, and cover the create-bucket data-usage path with regressions.
Co-authored-by: heihutu <[email protected]>
Also replace deprecated Atomic::fetch_update calls with try_update so the
current Rust toolchain keeps lint and CI jobs warning-clean.
Co-authored-by: heihutu <[email protected]>
Merge newer partial observed usage into the complete authoritative admin baseline instead of replacing the full bucket set.
Keep the merged view partial and non-converged so shared consumers do not treat it as quota-authoritative.
Co-authored-by: heihutu <[email protected]>
Validate lifecycle tier references through the tier reference proof path, preserve S3 list CommonPrefix XML compatibility, and make GetObject audit completion use real S3 error status codes.
Co-authored-by: heihutu <[email protected]>
Update the s3s git dependency to 6e7b41252c7ba218a90886f58d297716ddf68acf.
This pulls the upstream SelectRequest XML alias compatibility fix while keeping the RustFS s3s compatibility boundary intact.
Co-authored-by: heihutu <[email protected]>
Add a default-on, size-aware foreground PUT admission policy so large or
unknown-size PutObject requests are backpressured before body ingest and
erasure/RPC fan-out. Preserve the explicit strict gate semantics, including
limit=0 as an opt-out, and keep small PUTs on the legacy fast path.
Closesrustfs/backlog#2038
Co-authored-by: heihutu <[email protected]>
* Revert "perf(ecstore): use AHashMap for FileInfo metadata fields (#6738)"
This reverts commit 13a2ae212e.
* fix(filemeta): restore standard HashMap metadata (#6742)
Remove the direct ahash dependency added for FileInfo metadata and revert the affected filemeta/ecstore call sites back to std::collections::HashMap.
Co-authored-by: heihutu <[email protected]>
---------
Co-authored-by: heihutu <[email protected]>
Keep /health/ready from riding the generic internode lock RPC and channel keepalive budgets when a peer host is unreachable. Add a health-specific lock online timeout, route ping failures through the existing remote lock RPC eviction path, cache the static ping payload for readiness fan-out, and cover hanging cached channels with focused tests.
Refs rustfs/backlog#2033
Refs rustfs/rustfs#6286
Co-authored-by: heihutu <[email protected]>
Remove unused direct dependency declarations found by cargo-shear and delete the unlinked ecstore mimalloc diagnostics file.
Keep feature-forwarding dependencies explicit with package-local cargo-shear ignores so hotpath feature propagation remains intact.
Co-authored-by: heihutu <[email protected]>
Switch IAM QR rendering from qrcode to qrcode-rs 2.0.0 while keeping only the std and svg feature path enabled.
Set the release profile to a single codegen unit and disable release debuginfo as requested.
Verification:
- cargo info qrcode-rs --registry crates-io
- cargo tree -p rustfs-iam -e features
- CARGO_TARGET_DIR=/private/tmp/rustfs-target-qrcode-rs-profile-tuning cargo test -p rustfs-iam --locked
- cargo fmt --all --check
- git diff --check
Co-authored-by: heihutu <[email protected]>
* perf(signer): cache signing key to avoid redundant HMAC-SHA256
Cache the AWS4 signing key per (secret, region, date, service_type)
tuple. The signing key is derived from 4 HMAC-SHA256 calls and is
constant for a given user within the same UTC day, so caching it
eliminates ~0.5-1ms of redundant crypto per request.
The cache uses a LazyLock<Mutex<HashMap>> with automatic daily
rotation (cache entries naturally expire when the date component
of the key changes).
Refs: https://github.com/rustfs/backlog/issues/2005
Co-Authored-By: heihutu <[email protected]>
* fix(signer): bound signing key cache
* fix(signer): satisfy cache lint
---------
Co-authored-by: heihutu <[email protected]>
Co-authored-by: overtrue <[email protected]>
Upgrade rustfs-mimalloc and rustfs-mimalloc-sys to 0.5.1, then call the new safe wrapper from Tokio worker thread startup so mimalloc can treat runtime threads as threadpool workers.
Keep the hint no-op on Windows, matching RustFS allocator platform boundaries.
Co-authored-by: heihutu <[email protected]>
Run replication resync target reconcile and follow-up resync recovery in a background startup task so bucket metadata transaction lock contention cannot keep a node from joining the cluster.
Co-authored-by: heihutu <[email protected]>
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <[email protected]>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <[email protected]>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): optimize opts.clone() and FileInfo clone patterns
## Changes
1. ObjectOptions helper methods:
- add as_commit_opts(): creates commit options with no_lock=true,
metadata_cache_safe=false, include_part_checksums=true
- add as_read_opts(): creates read options with
include_part_checksums=true
- add with_no_lock(): creates options with modified no_lock field
2. Replace opts.clone() in hot paths:
- commit_opts = opts.as_commit_opts() (was 4-line manual clone)
- read_opts = opts.as_read_opts() (was 2-line manual clone)
3. Optimize FileInfo clone in rename path:
- avoid double clone: clone once and modify erasure.index in place
- pass &file_info reference to rename_data_borrowed_with_fence
## A/B Results (4-node cluster, c=64)
| Size | main | optimized | Change |
|------|------|-----------|--------|
| 1KiB | 892 obj/s | 920-976 obj/s | +3%~+9% |
| 4KiB | 957 obj/s | 903 obj/s | -5.7% |
| 16KiB | 922 obj/s | 855 obj/s | -7.3% |
Note: 1KiB improvement is consistent. 4KiB/16KiB variance
likely due to test noise; needs more rounds to confirm.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): add BytesMut buffer pool to EC encoding path
Pre-allocate a Vec<BytesMut> pool in the EC encoding loop to avoid
repeated heap allocations for ingest buffers.
Changes:
- Pre-allocate buffer pool with capacity 4
- Reuse buffers from pool after encoding
- Return buffers to pool when capacity is sufficient
Expected impact: +10-20% in EC encoding path by reducing
BytesMut allocation overhead.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: hector <[email protected]>
Co-authored-by: heihutu <[email protected]>
* fix(ecstore): handle metadata-less bucket residue
Diagnose metadata-less on-disk residue before non-force DeleteBucket reaches physical deletion, and keep scanner-discovered metadata-missing objects on a non-destructive heal path.
Add explicit heal --remove cleanup for unversioned metadata-less data directories, using the existing data-dir delete primitive and fail-closed shape checks so pre-commit or unknown residue is preserved.
Co-Authored-By: heihutu <[email protected]>
* fix(connect): adapt offline array validator
Wrap the filesystem summary validator in a closure so Option::is_some_and can pass the concrete array reference accepted by serde_json::Value::as_array.
Co-Authored-By: heihutu <[email protected]>
* fix(connect): remove redundant offline test clones
Move the temporary path into the swap closure after deriving the output path, keeping clippy's redundant-clone lint clean for offline bundle tests.
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: heihutu <[email protected]>
Keep liveness probes local by avoiding readiness collection and omitting readiness-only fields from liveness payloads. Readiness and MinIO cluster probes continue to report dependency and quorum state.
Co-authored-by: heihutu <[email protected]>
Treat lock quorum as part of node readiness for both /health and /health/ready response bodies while preserving the /health liveness HTTP 200 contract.
Add focused regression coverage for lock-quorum-only degradation and make the public /health layer fixture independent from process-global readiness state.
Refs: rustfs/backlog#2011
Co-authored-by: heihutu <[email protected]>
* chore(deps): update s3s revision
Pin the workspace s3s dependency to rustfs/s3s commit 39080d610e0560c55f068f6dd76b976e267b2f67 and refresh compatible dependencies with cargo update and cargo upgrade.
Co-authored-by: heihutu <[email protected]>
* fix(s3): preserve SigV4 body validation errors
Map s3s upload stream body validation failures into existing RustFS client-error types before the PUT body readers consume them. This keeps tampered single-chunk payload hashes from surfacing as InternalError after the s3s revision update.
Co-Authored-By: heihutu <[email protected]>
* chore(deps): use s3s 0.15.0 release
Switch the workspace dependency from the temporary s3s git revision to the published 0.15.0 crate and refresh the lockfile updates that come with the release.
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: heihutu <[email protected]>
* feat(mimalloc): add arena diagnostics and configuration
Based on mimalloc maintainer feedback (microsoft/mimalloc#1372),
add diagnostics to check mimalloc arena configuration at runtime.
Changes:
- Add rustfs-mimalloc-sys to workspace dependencies
- Add log_mimalloc_diagnostics() function to check:
- arena_max_object_size
- pagemap_commit status
- mimalloc version
- Add memory_observability module with mimalloc diagnostics
This helps diagnose why allocations might be going outside arenas,
which is the suspected root cause of futex contention.
Ref: rustfs/backlog#2005
Ref: microsoft/mimalloc#1372
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): add Vec<u8> buffer pool for EC operations
Add a general-purpose buffer pool to reduce Vec<u8> allocations
in hot paths like EC encoding/decoding.
Changes:
- Add BufferPool struct in crates/ecstore/src/erasure/codec/buffer_pool.rs
- Thread-safe pool with capacity-based bucketing (power-of-two)
- Global EC_BUFFER_POOL instance with 16 buffers per bucket
- Add buffer_pool module to codec/mod.rs
Expected impact:
- Reduce heap allocations in EC encode/decode paths
- Avoid memzero overhead (proven 4.8% CPU saving in ShardBufferPool)
- Reduce mimalloc lock contention
Note: Main bottleneck remains mimalloc internal synchronization
(futex 98.64% time). Buffer pool provides modest improvement (+2-5%).
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and related files
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): add #[allow(dead_code)] to buffer pool
The BufferPool infrastructure is ready but not yet integrated
into the EC hot paths. Add #[allow(dead_code)] with clear
documentation about integration status.
Co-Authored-By: heihutu <[email protected]>
* perf(ecstore): integrate BufferPool into bitrot verify path
Replace vec![0; shard_size] with get_ec_buffer() in the bitrot
verification hot path to reduce heap allocations and avoid memzero.
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and bitrot changes
Co-Authored-By: heihutu <[email protected]>
* refactor(ecstore): clean up buffer pool code
- Remove unnecessary #[allow(dead_code)] attributes
- Update module documentation to reflect current integration status
- Simplify code structure
Co-Authored-By: heihutu <[email protected]>
* perf(runtime): cap default worker threads at 16
Testing showed 16 worker threads outperforms 32+ for 1KiB PUT
workloads due to reduced mimalloc lock contention.
A/B test results (testing 4-node cluster, c=64):
- worker_threads=32: 740 obj/s (baseline)
- worker_threads=16: 785 obj/s (+6.1%)
The default was detect_cores() which returned 32 on our testing
nodes. Cap at 16 for optimal small-object performance.
Ref: rustfs/backlog#2005
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt to buffer pool and runtime changes
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): remove unused BufferPool::new() function
The new() function was never used since EC_BUFFER_POOL
initializes directly with with_limits(16).
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): update buffer_pool tests to use with_limits
Replace BufferPool::new() with BufferPool::with_limits(16) in tests
since new() was removed in favor of with_limits().
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: hector <[email protected]>
Co-authored-by: heihutu <[email protected]>
* perf: optimize cgroup resource detection with single System instance
Consolidate two sysinfo::System instantiations into one for CPU and
memory detection. Pre-compute the metrics basis string ("cgroup"/"host")
in ContainerResources to avoid per-snapshot String allocations in the
memory observability hot path.
Co-Authored-By: heihutu <[email protected]>
* style: apply cargo fmt formatting
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: heihutu <[email protected]>
The /health endpoint (liveness) was returning a hardcoded `ready: true`
in its response body regardless of actual node readiness state. This
caused a semantic contradiction with /health/ready (readiness), which
correctly reported readiness based on storage, IAM, lock quorum, and
peer health.
This led to confusing behavior in Kubernetes deployments where:
- /health returned 200 with `ready: true` (liveness)
- /health/ready returned 503 (readiness)
- Pods remained Running but were removed from Service endpoints
Changes:
- readiness_source_for_probe(Liveness) now returns Node readiness source
- health_check_state() for Liveness reflects actual readiness in body
while keeping HTTP 200 status (process is alive)
- build_health_response_parts() for Liveness now includes dependency
details and degradedReasons when readiness report is available
This ensures the `ready` field in /health body is truthful while
maintaining backward compatibility for liveness probe behavior.
Refs: rustfs/backlog#2011
Co-authored-by: heihutu <[email protected]>
Add a read-only Prometheus report helper for backlog#2007 so the 200us vs 50us coalescer delay experiment can capture RPC, batch distribution, stage latency, and host-cost signals with one fixed evidence format.
Co-authored-by: heihutu <[email protected]>
* feat(allocator): replace mimalloc/libmimalloc-sys with rustfs-mimalloc/rustfs-mimalloc-sys
Replace the upstream xonatius/mimalloc_rust.git fork (mimalloc + libmimalloc-sys)
with the published rustfs-mimalloc (v0.5.0) and rustfs-mimalloc-sys (v0.5.0) crates
from crates.io.
The new crates are based on mimalloc V3 (v3.5.0) and provide:
- MiMalloc global allocator with safe API (collect, stats_json, process_info)
- Heap management and arena operations (heap module)
- Full FFI bindings to mimalloc V3
Changes:
- Workspace deps: mimalloc + libmimalloc-sys (git) → rustfs-mimalloc + rustfs-mimalloc-sys (crates.io)
- allocator_reclaim.rs: libmimalloc_sys::mi_collect → rustfs_mimalloc::MiMalloc::collect
- memory_observability.rs: raw FFI mi_stats_get_json → MiMalloc::stats_json()
- main.rs: heap ownership tests use Heap::contains() (V3 API)
- deny.toml: remove xonatius/mimalloc_rust.git from allow-git
Co-Authored-By: heihutu <[email protected]>
* fix(ecstore): default rename fanout to parallel early-ack path
Switch the default rename_data commit fanout from serial join_all to the
parallel JoinSet early-ack path. The serial path (#5987) was the primary
cause of the 1MiB PUT regression (-71.7%) observed in rc.3 benchmarks.
A/B verification on testing 4-node cluster (c=64, 1MiB PUT, 2min):
- Serial (join_all): 96.99 MiB/s, P50=644ms
- Early ack (JoinSet): 177.46 MiB/s, P50=407ms (+83%)
Also:
- Update rename_data_reclaims_synthetic_inline_rollback_dir_after_commit
to use rename_data_owned and await tail_drain for proper cleanup.
- Update rename_data_waits_for_tail_disk_after_write_quorum to explicitly
test the serial path (now non-default) via env override.
- Add error source chain to HTTP Body stream transport error log
(backlog#2005) so the underlying cause is visible.
Ref: rustfs/backlog#2005
Ref: rustfs/backlog#1792#issuecomment-5384346238
Ref: rustfs/backlog#1792#issuecomment-5384370938
Co-Authored-By: heihutu <[email protected]>
---------
Co-authored-by: heihutu <[email protected]>
Change DEFAULT_FSYNC_BLOCKING_THREADS from 0 to 64 to isolate
fsync/fdatasync operations into a dedicated blocking thread pool.
A/B validation on 4-node EC cluster (testing, 10.0.0.5/8/9/11:9000):
PUT 256KiB c16: p99 226ms → 135ms (−40%), p50 60ms → 19ms (−68%)
GET 256KiB c16: p99 3.97ms → 3.63ms (−9%), throughput +1.8%
GET 4KiB c64: neutral (pure read, no fsync involvement)
Without isolation, fsync operations contend with read I/O (pread/stat/open)
on the main blocking pool, causing device-bound fsync to starve read
operations under mixed PUT+GET workloads.
Co-authored-by: heihutu <[email protected]>
