* perf(ecstore): reuse erasure-decode shard buffers across stripes
ParallelReader::read allocated and zero-filled a fresh
`vec![0u8; shard_size]` per shard on every erasure stripe. Erasure::decode
builds one ParallelReader and loops stripes, so those buffers can be
reused: BitrotReader::read overwrites buf[..n] and the caller truncates to
n, so leftover bytes from the prior stripe are never observed. The decode
loop now hands each stripe's shard buffers back to the reader, which
reuses them (resized to shard_size) on the next stripe. The heal path,
which consumes its buffers into Bytes, never hands them back and is
unchanged.
This path runs on every object GET. Streaming-decode benchmark
(Erasure::decode end-to-end, median):
4+2 16MiB 1MiB-block: -9.6% verify-on, -23.8% verify-off
6+3 24MiB 1MiB-block: -9.7% verify-on, -22.5% verify-off
4+2 4MiB 64KiB-block: -5.5% verify-on, -16.3% verify-off
Adds a streaming-decode benchmark covering the ParallelReader path and a
regression test that decodes a multi-stripe object with missing data
shards (so reconstructed buffers are recycled) with bitrot verification
both on and off, asserting byte-exact output.
Co-authored-by: Copilot <[email protected]>
* perf(ecstore): tighten decode buffer reuse and bench harness
Address PR review feedback on the erasure-decode buffer-reuse change:
- Only claim a recycled shard buffer inside the `Some(reader)` branch of
`ParallelReader::read`, so the take is co-located with the read that
uses it and missing shards no longer touch the recycle slot.
- Move per-shard `BitrotReader` construction into Criterion `iter_batched`
setup so reader/UUID construction stays out of the timed decode path,
and assert decode success plus full output length each iteration.
Re-measured with the cleaner harness (median, before -> after, all
p < 0.05): verify-on -5.3%..-11.7%, verify-off -15.8%..-25.3%.
Co-authored-by: Copilot <[email protected]>
* perf(ecstore): drop dead recycle slots and tidy decode bench
Address the second PR review pass:
- In `Erasure::decode`, clear recycled slots for missing-shard (None-reader)
indices before handing buffers back. `ParallelReader::read` only reuses
slots with an active reader, and `decode_data` always re-allocates the
reconstructed shard, so retaining those buffers only held memory that could
never be reused in degraded reads.
- Move the output sink into the benchmark's `iter_batched` setup and validate
decode success/output length once per config instead of inside the timed
loop, so neither sink construction nor assertions touch the measurement.
Re-measured with the final harness (median, before -> after, all p < 0.05):
verify-on -5.2%..-9.8%, verify-off -19.4%..-25.8%.
Co-authored-by: Copilot <[email protected]>
---------
Co-authored-by: Copilot <[email protected]>
* build(deps): bump brotli from 8.0.3 to 8.0.4 in the dependencies group
Bumps the dependencies group with 1 update: [brotli](https://github.com/dropbox/rust-brotli).
Updates `brotli` from 8.0.3 to 8.0.4
- [Release notes](https://github.com/dropbox/rust-brotli/releases)
- [Commits](https://github.com/dropbox/rust-brotli/compare/8.0.3...8.0.4)
---
updated-dependencies:
- dependency-name: brotli
dependency-version: 8.0.4
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <[email protected]>
* build(deps): bump s3s from cf4c3346 to c59b62f7ba in the dependencies group
---------
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <[email protected]>
* fix(bucket-repl): persist MRF retry queue to disk and reload on startup
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* fix(bucket-repl): address three blocking MRF issues from review
1. MRF replay loses delete operations — add `MrfOpKind` discriminator to
`MrfReplicateEntry` (Object | Delete, default=Object for backward
compat). `DeletedObjectReplicationInfo::to_mrf_entry` now persists
`op=Delete`, `version_id`, `delete_marker_version_id`, and
`delete_marker`. `start_mrf_processor` branches on `op`: delete
entries skip `get_object_info` and replay via
`schedule_replication_delete` with `ReplicationType::Heal`; object
entries follow the existing heal path.
2. `flush_mrf_to_disk` cleared the in-memory batch even on encode/write
failure — changed return type to `bool` and callers now only
`pending.clear()` on `true`, so a transient storage error retries
on the next tick instead of silently dropping the batch.
3. Add focused tests: encode/decode roundtrips for object, delete-marker,
versioned-delete, and mixed-batch entries; a routing test confirming
op-kind propagates correctly and that the default is Object for
legacy files; a legacy-compat test verifying old entries round-trip
cleanly through the new format.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* style: fix clippy redundant-clone in MRF tests
Replace &[entry.clone()] with std::slice::from_ref(&entry) in two
encode_mrf_file call sites flagged by clippy's redundant_clone lint
under --all-targets --features rio-v2.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
* test(bucket-repl): strengthen legacy MRF compat test with hand-built msgpack
The previous mrf_legacy_file_without_op_field_decoded_as_object test
round-tripped through encode_mrf_file, so it exercised the new format
and never touched a truly-legacy payload.
Replace it with a hand-built msgpack payload that genuinely omits the
"op", "deleteMarker", and "deleteMarkerVersionID" keys — exactly what
the old binary would have written before MrfOpKind existed. The test
now fails if #[serde(default)] is removed from the op field, which
proves real backward compatibility rather than round-trip stability.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
---------
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: houseme <[email protected]>
