Commit Graph
9 Commits
Author SHA1 Message Date
Zhengchao AnandGitHub f1f86ee9d0 chore(ecstore): drop the set_disk dead_code blanket (#6141)
* chore(ecstore): drop the set_disk dead_code blanket

Removing the blanket exposes 39 items; exactly one is deleted. The low share is a finding, not caution: unlike the disk root, where platform gating made local adjudication impossible, here the items were checked and nearly all of them are live.

Deleted: HealEntryResult, the only item with no reference anywhere.

What the checks turned up, in the order the warnings suggest deleting them:

SetDisks::rename_data looked like the head of a dead chain feeding into_legacy_tuple and RenameDataLegacyTuple. It is not: production goes through rename_data_owned, and rename_data itself has test callers at mod.rs:5809 and 5880. The chain below it is therefore live through the tests, and inferring "this is dead, so its callee is dead" would have removed three working items.

create_bitrot_readers_until_quorum, read_multiple_files and map_cleanup_join_result all have callers inside their files' test modules, so they only look dead in the lib target.

TransitionCommitBarrier and TransitionUploadedSaveProbe, with their install/wait_until_paused/release surfaces, are installed by tests behind #[cfg(all(test, feature = "test-util"))].

ctx.rs's SetDisksCtx accessors are the split seam left by the SetDisks god-object break-up (backlog#815).

heal_object_dir's two apparent references are comments, and they document an index-alignment contract that live code maintains for it, so they stay as they are.

Worth a maintainer decision: the metadata early-stop switch has a complete percentage-rollout facet — ENV_RUSTFS_GET_METADATA_EARLY_STOP_ROLLOUT_PCT, get_metadata_early_stop_rollout_pct and should_use_metadata_early_stop — with no caller, no test and no documentation, while its sibling enable flag is live. It is kept with an allow that says so rather than removed, since a rollout knob is a product call.

One placement note for anyone adding allows near heal code: check_logging_guardrails.sh requires #[instrument(level = "trace")] to sit immediately before async fn heal_object_dir, so the allow goes above the instrument attribute. Putting it between the two drops the guard's match count and fails the check.

Verification, four lanes warning-free: default, --tests, --features rio-v2 --tests, --features test-util --tests. cargo nextest run -p rustfs-ecstore 4096 passed; clippy --lib --tests -D warnings clean; make pre-commit exit 0.

Ref rustfs/backlog#1823 (step 2).

* chore(ecstore): fix duplicated and inaccurate dead_code reasons in set_disk

format_lock_error carried the same #[allow] twice. Five items in the
locking/heal roots were labelled 'asserted by this file's tests' while
having no reference at all - heal_object_dir's only two references are
comments, as this branch's own notes point out. Say what each item
actually is instead, so the next reader does not assume test coverage
that is not there.

Ref rustfs/backlog#1823.

* chore(ecstore): correct the bounded_spare_disk_index dead_code reason

The mod.rs copy is an unused test fixture, not something this module's
tests assert; the namesake that is exercised lives in the io_primitives
test module.

Ref rustfs/backlog#1823.
2026-08-16 21:38:46 +08:00
d2b1003612 perf(storage): converge Wave 2 hot-path optimizations (#6065)
* perf(get): share inline shards and lock clients

Co-Authored-By: heihutu <[email protected]>

* perf(ecstore): converge PUT encoding on contiguous blocks

Co-Authored-By: heihutu <[email protected]>

* perf(get): cache codec streaming gate config

Co-Authored-By: heihutu <[email protected]>

* fix(sse): redact projected customer headers

Co-Authored-By: heihutu <[email protected]>

* perf(ecstore): collapse GET metadata snapshots

Co-Authored-By: heihutu <[email protected]>

* perf(ecstore): reuse decode stripe scratch

Co-Authored-By: heihutu <[email protected]>

* refactor(ecstore): trim decode scratch adapters

Co-Authored-By: heihutu <[email protected]>

* test(ecstore): adapt transition checks to metadata snapshots

Co-Authored-By: heihutu <[email protected]>

* perf(get): release metadata snapshots at ownership boundary

Co-Authored-By: heihutu <[email protected]>

* refactor(ecstore): close cumulative fast-path findings

Co-Authored-By: heihutu <[email protected]>

* fix(storage): preserve lock and header invariants

Co-Authored-By: heihutu <[email protected]>

* test(ecstore): adapt cumulative paths after rebase

Co-Authored-By: heihutu <[email protected]>

* fix(rio-v2): adapt generated metadata fixture

Co-Authored-By: heihutu <[email protected]>

---------

Co-authored-by: heihutu <[email protected]>
2026-08-13 16:34:28 +08:00
59494d5089 perf(get): reuse reader paths and lock namespaces (#6015)
Co-authored-by: heihutu <[email protected]>
2026-08-12 17:37:25 +00:00
656a2f14bf fix(logging): bound hot-path span amplification (#5763)
* fix(logging): bound hot-path span amplification

* refactor(logging): reuse HTTP log target constant

Co-Authored-By: heihutu <[email protected]>

---------

Co-authored-by: Anthony Martin <[email protected]>
Co-authored-by: houseme <[email protected]>
Co-authored-by: heihutu <[email protected]>
2026-08-06 14:29:18 +00:00
035ce5d784 feat(obs): add bounded metrics dimensions (#5645)
* feat(obs): add drive topology detail metrics

Expose additive drive info, topology, state, and per-drive API metrics while preserving the existing drive metric label sets.

Backlog: rustfs/backlog#1655

Co-Authored-By: heihutu <[email protected]>

* fix(obs): preserve suspect drive runtime state

Keep suspect as a bounded drive runtime state and avoid all-zero runtime_state samples for that storage health state.

Co-Authored-By: heihutu <[email protected]>

* fix(obs): skip unknown drive inode samples

Avoid exporting zero inode gauges for missing or stale drive snapshots and ignore zero-count API latency buckets.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add scanner source work detail metrics

Expose additive scanner source and cycle work metrics with bounded server/source/state labels while leaving the existing aggregate scanner metrics unchanged.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add ilm action detail metrics

Expose additive ILM action/state task metrics with a server label while preserving the existing aggregate ILM series.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add delivery target server metrics

Expose additive audit and notification delivery target metrics with server labels and extend removed-target tombstones for the server-aware series.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add replication target flow metrics

Expose additive bucket replication target sent and failed-flow metrics while preserving existing bucket aggregates and target backlog series.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add request server metrics

Expose additive API request metrics with server labels while preserving the existing request and traffic metric label sets.

Co-Authored-By: heihutu <[email protected]>

* style(obs): apply rustfmt to metrics changes

Apply rustfmt output to the metrics dimension changes without altering behavior.

Co-Authored-By: heihutu <[email protected]>

* style(obs): reuse audit target label constant

Use the exported audit target_id label constant for legacy audit target metrics.

Co-Authored-By: heihutu <[email protected]>

* feat(obs): populate drive disk metrics

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add scanner bucket drive result metrics

Co-Authored-By: heihutu <[email protected]>

* feat(obs): add replication proxy server metrics

Co-Authored-By: heihutu <[email protected]>

* fix(obs): address metric liveness review

Use checked division for drive API latency aggregation and keep recovered drive, scanner current-cycle, replication flow, audit target, and notification target series from retaining stale values.

Co-Authored-By: heihutu <[email protected]>

* fix(obs): address metric dimension review

Co-Authored-By: heihutu <[email protected]>

* fix(obs): address additional metric review

Co-Authored-By: heihutu <[email protected]>

* fix(obs): count drive calls at start

Co-Authored-By: heihutu <[email protected]>

* fix(obs): address metrics dimension review

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): address dimension review gaps

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): address scanner review follow-ups

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): address runtime review follow-ups

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): reduce disk metric contention

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): address runtime review follow-ups

Co-Authored-By: heihutu <[email protected]>

* fix(metrics): retire stale dimension series

Co-Authored-By: heihutu <[email protected]>

---------

Co-authored-by: heihutu <[email protected]>
2026-08-03 09:03:34 +08:00
cxymdsandGitHub 67904a6c18 fix(ecstore): start with unresolved Kubernetes peers (#5460)
* fix(ecstore): start with unresolved Kubernetes peers

* fix(ecstore): infer Kubernetes endpoint identity safely

* fix(ecstore): fail closed on unsafe format migration

* fix(ecstore): reject poisoned format heal candidates

* fix(ecstore): reject unsafe legacy migration outliers

* fix(ecstore): resume interrupted format migrations

* fix(ecstore): preserve Kubernetes startup compatibility
2026-07-30 11:14:38 +08:00
Zhengchao AnandGitHub 63e57378d6 fix(ecstore): never cache fabricated bucket metadata as authoritative (#5307)
BucketMetadataSys::get_config lazily fabricated a default BucketMetadata
(object-lock off) for any bucket whose .metadata.bin was ConfigNotFound and
cached it in the map that the map-only, fail-closed metadata_sys::get()
serves. The object-lock batch-delete gate (object_lock_delete_check_required,
backlog#929 / #4297) treats that map as authoritative, so a metadata miss
became a cached "no lock" answer: a versioning peek could poison the cache
and let delete_objects skip the per-object retention/legal-hold stat. The
same fabrication raced make_bucket (lost update overwriting freshly
persisted lock-enabled metadata) and let the 15-minute refresh loop replace
good cached metadata on a transient quorum dip.

Production changes:
- get_config caches only metadata actually read from disk; misses are
  recorded in a bounded negative cache (30s TTL, 10k entries, invalidated by
  set()) so repeated lookups for metadata-less names cost no extra
  namespace-lock + erasure-set fanout (reachable pre-auth via CORS
  preflight and per-key in DeleteObjects).
- concurrent_load never lets a fabricated default REPLACE an existing map
  entry; startup insert-if-vacant behavior for legacy buckets is preserved.
- delete_objects and new_ns_lock resolve dist-erasure, versioning, and the
  object-lock gate from the set's own instance context (backlog#1052)
  instead of the ambient facade, so a second in-process instance (or, in
  tests, another test's transient DistErasure window) cannot reroute
  locking onto an empty dist locker list or answer with the wrong
  instance's bucket state.

Test-isolation changes (the bug that surfaced all of the above: the
delete_objects lock-gating test failed deterministically when sharing a
process with the lifecycle env tests):
- The MinIO-migration test builds on an isolated InstanceContext instead of
  registering soon-deleted disks in the shared bootstrap registry.
- The cached lifecycle env re-registers its disks on every use, surviving
  other serial tests' reset_local_disk_test_state.
- Hermetic SetDisks helpers gain isolated-context variants pinned to plain
  erasure; tier-free non-serial test modules use them, guard-based
  SetupTypeGuard tests stay on the bootstrap context.
- Three deterministic pin tests (nextest-safe) cover the caching contract,
  the delete gate resolution source, and the ns-lock resolution source.

Verification:
- cargo test -p rustfs-ecstore --lib -- --exact <4-test combo from the
  report> (previously failing, now green)
- cargo test -p rustfs-ecstore --lib: 3169 passed / 0 failed across
  repeated runs; cargo fmt --check and cargo clippy --lib --tests clean
- Adversarial validation (high-risk tier, all seven roles) run per
  AGENTS.md; all findings fixed or rebutted with evidence
2026-07-27 00:53:48 +08:00
Zhengchao AnandGitHub 83edafb11f fix(ecstore): reject renewing a mis-mounted drive into the wrong set (backlog#799 B19) (#4321)
fix(ecstore): reject renewing a misplaced drive into the wrong set (backlog#799 B19)

`renew_disk` located the drive's (set, disk) position from its own format via
`find_disk_index`, but never checked that the resolved `set_idx` matched this
`SetDisks`' own `set_index` before inserting the drive into `self.disks`. A
drive whose format places it in another set would be claimed by this set too,
so two sets could manage the same drive and degrade together.

Skip (with a warning) when `set_idx != self.set_index`.

Refs backlog#799 (B19), tracked in rustfs/backlog#863.
2026-07-06 22:29:05 +08:00
Zhengchao AnandGitHub a8d8e56478 refactor(ecstore): relocate NamespaceLocking + finalize God-Object split (backlog#822) (#4291) 2026-07-05 23:34:46 +08:00