Commit Graph
19 Commits
Author SHA1 Message Date
Zhengchao AnandGitHub 0a975f2fe2 docs(knowledge-base): prune stale content and add agent-facing index (#7035) 2026-09-02 08:26:59 +08:00
ab44ae7e83 fix(scanner): add supported usage state reset (#6972)
Add an authenticated scanner usage-state reset endpoint that publishes a fenced bootstrap marker for full rebuilds instead of requiring operators to delete usage metadata by hand.

Guard the reset with the scanner leader lock, storage publication epoch, and per-slot revision preconditions, and make startup resumable across stale cleanup leftovers while still rejecting newer conflicting usage state.

Co-authored-by: heihutu <[email protected]>
Co-authored-by: Zhengchao An <[email protected]>
2026-09-01 01:51:46 +00:00
281e40f1cc fix(scanner): fit usage persistence within publication lease (#6967)
Lower the default scanner cache save timeout so the derived usage persistence budget stays inside the effective distributed publication lease window.

Add focused regressions for the default publication budget and bootstrap-pending observational baselines, and update operator docs with the new default.

Co-authored-by: heihutu <[email protected]>
2026-08-31 22:58:29 +08:00
Zhengchao AnandGitHub c4ac11d22e fix(scanner): persist decommission catch-up debt (#6922) 2026-08-31 08:45:36 +08:00
cxymdsandGitHub 2fccfdeabe fix(scanner): fence timed out scan cycles (#6352)
* fix(scanner): fence timed out scan cycles

* fix(scanner): cancel scan workers with cycle scope

* fix(scanner): reject persisted timer overflow

* fix(scanner): reject terminal leadership epochs

* fix(scanner): reject trailing cycle state bytes
2026-08-22 14:38:18 +00:00
105b6fbfde fix(scanner): honor explicit cycle cadence (#6313)
Co-authored-by: Henry Guo <[email protected]>
Co-authored-by: houseme <[email protected]>
2026-08-20 23:35:03 +08:00
785ee719e7 feat(heal): aggregate replacement recovery status (#5916)
Add a replacement recovery peer RPC so Admin v4 can distinguish definitive cluster proofs from unsupported, unavailable, or conflicting peer state without extending the existing background heal v3/v1 status protocol.

Co-authored-by: heihutu <[email protected]>
2026-08-10 05:03:27 +00:00
1aeb84dd6b feat(heal): expose replacement recovery status (#5912)
Add a v4 admin status endpoint for local durable automatic replacement recovery records without changing the v3 background heal status or peer v1 payloads.

Co-authored-by: heihutu <[email protected]>
2026-08-10 01:48:18 +00:00
f17ea7f146 fix(heal): harden replacement rebuild tracking (#5892)
* fix(heal): gate auto replacement formatting

Co-Authored-By: heihutu <[email protected]>

* fix(heal): require replacement target outcomes

Co-Authored-By: heihutu <[email protected]>

* fix(heal): bind resumes to replacement targets

Co-Authored-By: heihutu <[email protected]>

* fix(heal): fence healing marker ownership

Co-Authored-By: heihutu <[email protected]>

* test(heal): cover replacement target completion

Co-Authored-By: heihutu <[email protected]>

* docs(heal): clarify replacement recovery status

Co-Authored-By: heihutu <[email protected]>

* fix(heal): canonicalize replacement target checks

Co-Authored-By: heihutu <[email protected]>

* fix(heal): satisfy marker test module lint

Co-Authored-By: heihutu <[email protected]>

* fix(heal): scope automatic replacement format

Co-Authored-By: heihutu <[email protected]>

* fix(heal): require a mounted replacement target

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): avoid cloned ref slice in test

Co-Authored-By: heihutu <[email protected]>

* fix(heal): revalidate replacement before scanning

Co-Authored-By: heihutu <[email protected]>

* fix(heal): reset stale resume checkpoints

Co-Authored-By: heihutu <[email protected]>

* fix(heal): release scanner disk map before probing

Co-Authored-By: heihutu <[email protected]>

* fix(heal): persist replacement intent before format

Co-Authored-By: heihutu <[email protected]>

* fix(heal): fail closed on mountinfo read errors

Co-Authored-By: heihutu <[email protected]>

* fix(heal): fence replacement target identity

Co-Authored-By: heihutu <[email protected]>

* fix(heal): order replacement completion cleanup

Co-Authored-By: heihutu <[email protected]>

* fix(heal): atomically seal replacement completion

Co-Authored-By: heihutu <[email protected]>

* test(heal): census replacement target shards

Co-Authored-By: heihutu <[email protected]>

* fix(heal): fence replacement recovery ownership

Co-Authored-By: heihutu <[email protected]>

* fix(heal): preserve replacement recovery anchors

Co-Authored-By: heihutu <[email protected]>

* fix(heal): satisfy replacement recovery lint gates

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): bind replacement identity to mount lease

Co-Authored-By: heihutu <[email protected]>

* test(heal): cover durable replacement recovery states

Co-Authored-By: heihutu <[email protected]>

* fix(heal): validate persisted resume task identifiers

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): avoid blocking replacement marker CAS

Co-Authored-By: heihutu <[email protected]>

* fix(heal): report failed marker rollback

Co-Authored-By: heihutu <[email protected]>

* test(heal): pin replacement resume schema compatibility

Co-Authored-By: heihutu <[email protected]>

* fix(heal): preserve durable recovery anchors

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): preserve public disk path semantics

Co-Authored-By: heihutu <[email protected]>

* test(heal): use canonical replacement task ids

Co-Authored-By: heihutu <[email protected]>

* test(heal): cover automatic replacement in 3x4 cluster

Co-Authored-By: heihutu <[email protected]>

* fix(heal): verify replacement target commits

Co-Authored-By: heihutu <[email protected]>

* fix(heal): persist replacement completion proof

Co-Authored-By: heihutu <[email protected]>

* feat(heal): expose durable replacement status

Co-Authored-By: heihutu <[email protected]>

* fix(heal): bound durable replacement discovery

Co-Authored-By: heihutu <[email protected]>

* fix(heal): remove replacement readiness bypass

Co-Authored-By: heihutu <[email protected]>

* fix(heal): retry terminal replacement cleanup

Co-Authored-By: heihutu <[email protected]>

* fix(heal): isolate replacement intents from legacy resume

Co-Authored-By: heihutu <[email protected]>

* fix(heal): migrate legacy replacement intents at startup

Co-Authored-By: heihutu <[email protected]>

* style(heal): apply strict clippy fix

Co-Authored-By: heihutu <[email protected]>

* fix(heal): prioritize active replacement recovery state

Co-Authored-By: heihutu <[email protected]>

* fix(heal): bind readiness to the admitted mount lease

Co-Authored-By: heihutu <[email protected]>

* fix(heal): atomically publish replacement intents

Co-Authored-By: heihutu <[email protected]>

* fix(heal): isolate replacement recovery directory

Co-Authored-By: heihutu <[email protected]>

* fix(heal): tolerate an empty recovery directory

Co-Authored-By: heihutu <[email protected]>

* style(heal): remove redundant disk bytes conversion

Co-Authored-By: heihutu <[email protected]>

* fix(heal): reconcile proof-first replacement recovery

Co-Authored-By: heihutu <[email protected]>

* fix(heal): fence torn intent recovery

Co-Authored-By: heihutu <[email protected]>

* test(heal): cover replacement migration conflicts

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): fence replacement lease mount identity

Co-Authored-By: heihutu <[email protected]>

* test(heal): cover missing replacement path admission

Co-Authored-By: heihutu <[email protected]>

* fix(heal): reject conflicting legacy completion proof

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): fall back to proc mount identity

Co-Authored-By: heihutu <[email protected]>

* feat(admin): expose replacement recovery status

Surface the local durable replacement recovery snapshot in the background heal status response so operators can tell whether replacement cleanup is definitive or still pending.

Co-Authored-By: heihutu <[email protected]>

* fix(heal): keep replacement status compatible

Keep the existing background heal status response wire-compatible while retaining the Linux mount lease cleanup needed for the replacement recovery branch.

Co-Authored-By: heihutu <[email protected]>

* style(ecstore): match linux mount lease formatting

Keep Linux rustfmt output stable for the replacement mount lease comparison.

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): qualify mount lease test constant

Use the disk module path for the format config constant in the Linux mount lease regression test.

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): keep procfd mount roots directory-safe

Use a procfd path with an explicit directory component so Unix directory guards can open the replacement mount lease root with O_NOFOLLOW while preserving handle-relative I/O semantics.

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): delete empty leased buckets via dirfd

Use the held mount lease fd as the parent for non-force empty bucket deletion on Linux so procfd-rooted paths do not get rejected as BucketNotEmpty. Also make the download-part OpenOptions truncate behavior explicit and keep fsync test recording stable across procfd canonicalization.

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): scan leased bucket paths for emptiness

Use the local disk I/O root for bucket emptiness probes before non-force bucket deletion and table-bucket metadata checks. This keeps validation on the same mount instance as the subsequent local disk delete path.

Co-Authored-By: heihutu <[email protected]>

* test(ecstore): align lease path test probes

Co-Authored-By: heihutu <[email protected]>

* fix(heal): block unsafe replacement recovery restarts

Co-Authored-By: heihutu <[email protected]>

* fix(heal): defer blocked replacement candidates

Co-Authored-By: heihutu <[email protected]>

* fix(heal): retry transient replacement discovery

Co-Authored-By: heihutu <[email protected]>

* fix(heal): keep transient recovery errors retryable

Co-Authored-By: heihutu <[email protected]>

* fix(heal): block corrupt legacy replacement state

Co-Authored-By: heihutu <[email protected]>

* fix(heal): classify flat replacement intent corruption

Co-Authored-By: heihutu <[email protected]>

* fix(heal): keep transient resume loads retryable

Classify malformed legacy replacement state as blocking corruption while preserving disk and transient load failures for retry. This avoids permanently blocking replacement recovery on temporary storage errors.

Co-Authored-By: heihutu <[email protected]>

* fix(heal): avoid latching transient legacy publishes

Co-Authored-By: heihutu <[email protected]>

* fix(heal): retry blocked legacy migrations

Co-Authored-By: heihutu <[email protected]>

* fix(heal): defer blocked startup recoveries

Co-Authored-By: heihutu <[email protected]>

* fix(ecstore): preserve disk sync limiter across lease roots

Co-Authored-By: heihutu <[email protected]>

---------

Co-authored-by: heihutu <[email protected]>
Co-authored-by: zhi22915 <[email protected]>
2026-08-10 08:32:47 +08:00
a63b79004c fix(scanner): make distributed usage convergence authoritative (#5151)
* fix(scanner): make distributed usage cycles authoritative

* fix(scanner): close distributed refresh races

* fix(config): align scanner reload integration

* fix(admin): scope config test helpers

* fix(scanner): harden distributed usage convergence

* fix(scanner): preserve rolling activity compatibility

* fix(admin): expose non-secret optional config values

* fix(scanner): acknowledge distributed dirty usage

* fix(ecstore): make bucket mutations cancellation safe

* fix(scanner): preserve pending dirty acknowledgements

* test(obs): account for superseded scanner metric

* fix(api): reject excess detached bucket mutations

* test: close scanner convergence coverage gaps

* fix(scanner): make path tracking cleanup one-shot

---------

Co-authored-by: Henry Guo <[email protected]>
Co-authored-by: houseme <[email protected]>
2026-07-25 18:45:16 +08:00
889a45ad4d fix(scanner): back off clean idle scans across erasure clusters (#4984)
* fix(scanner): back off clean single-disk cycles

* fix(scanner): extend idle backoff across erasure clusters

---------

Co-authored-by: Henry Guo <[email protected]>
2026-07-18 10:49:45 +08:00
7bdb25ae9d feat(scanner): define replication boundary contract (#3630)
* feat(scanner): define replication boundary contract

* fix(scanner): propagate replication boundary metrics

* fix(scanner): preserve repair metadata on merge

---------

Co-authored-by: Henry Guo <[email protected]>
Co-authored-by: houseme <[email protected]>
2026-06-23 21:35:36 +08:00
b387689f26 feat(heal): expose scanner-aware operations status (#3483)
Co-authored-by: Henry Guo <[email protected]>
2026-06-15 22:28:53 +08:00
Henry GuoandGitHub d28aedfbb3 feat(scanner): expose replication repair kind metrics (#3476) 2026-06-15 16:52:39 +08:00
dd6b4c35ad fix(scanner): harden lifecycle and tiering backlog (#3469)
* fix(scanner): expose lifecycle transition backlog

* fix(scanner): expose lifecycle expiry backlog

* fix(scanner): preserve lifecycle backlog states

---------

Co-authored-by: Henry Guo <[email protected]>
2026-06-15 15:01:21 +08:00
46fb4bdc2f feat(scanner): add source-aware maintenance controls (#3461)
* feat(scanner): add source-aware maintenance controls

* fix(scanner): preserve source control between cycles

---------

Co-authored-by: Henry Guo <[email protected]>
2026-06-15 12:12:05 +08:00
2c5615d2ea feat(scanner): expose distributed metrics (#3452)
* feat(scanner): expose distributed metrics

* docs(scanner): clarify distributed metrics collection

---------

Co-authored-by: Henry Guo <[email protected]>
Co-authored-by: houseme <[email protected]>
2026-06-15 07:05:43 +08:00
0d16a86d9a docs(scanner): add benchmark runbook (#3412)
Co-authored-by: Henry Guo <[email protected]>
2026-06-13 22:39:13 +08:00
7191a3abae docs(scanner): document runtime scanner controls (#3339)
* docs(scanner): document runtime scanner controls

* docs(scanner): split English and Chinese README

---------

Co-authored-by: Henry Guo <[email protected]>
Co-authored-by: majinghe <[email protected]>
2026-06-11 12:04:34 +08:00