f7724d223b
feat(rio): rio_v2 is compatible with minio for storing data. ( #3115 )
...
* Set up a compatibility layer for replacing old Rio components with new ones.
* fix(rio). compress range
* feat(rio). Add the experimental feature rio_v2 to support minio data at the binary level.
* feat(rio_v2): add sse-c test
* test compression component
* simple fix
* fix minlz encode
* fix metadata
* fix kms key cache error
* Update launch.json
* ci: set nix crate download user agent
* fix: gate obs pyroscope backend
* ignore minio test
* fix encrypt check
* fix
* fix
* fix
* Update object_usecase.rs
* Update ci.yml
* fix
* ci add rio-v2 test
* fix
* ci fix
* fix
* Reconstructed into a more reasonable compatibility mode
* fix
* fix
---------
Signed-off-by: houseme <[email protected] [email protected] [email protected] [email protected] [email protected]
2026-06-08 11:59:14 +00:00
houseme and GitHub
6db4c3538d
fix(allocator): restore validated jemalloc target gating ( #3236 )
...
* fix(allocator): restore validated jemalloc target gating
Restrict the global allocator and jemalloc profiling paths to linux-gnu-x86_64 so Linux ARM64 builds fall back to mimalloc again.
Update the runtime profiling, allocator reclaim, and admin profiling handlers to use the same target gating and avoid exposing jemalloc-only code on unsupported targets.
Verification:
- cargo fmt --all
- cargo fmt --all --check
- cargo check -p rustfs
- make pre-commit
* fix(profiling): restore non-jemalloc platform behavior
Restore CPU profiling support on macOS while keeping jemalloc-backed memory profiling restricted to validated linux-gnu-x86_64 targets.
Also restore Unix log directory permission hardening so the allocator regression fix does not roll back unrelated observability behavior on other supported platforms.
Verification in progress:
- cargo fmt --all
- cargo check -p rustfs
- make pre-commit (running)
* fix(profiling): qualify periodic memory log macros
Use explicit tracing macro paths in periodic jemalloc memory profiling logging so builds do not fail when the local target configuration excludes those branches from the current import set.
Verification:
- cargo fmt --all
- cargo check -p rustfs
* fix(profiling): gate unsupported memory helper
Restrict the unsupported memory profiling helper to non-linux-gnu-x86_64 targets so Linux builds do not emit dead_code warnings for an unreachable fallback.
Verification:
- cargo fmt --all
- cargo check -p rustfs
2026-06-06 08:22:29 +00:00
ad1a489f75
feat(scanner): add scanner budget progress controls ( #3185 )
...
Co-authored-by: Henry Guo <[email protected]
2026-06-03 14:37:58 +00:00
cc07946782
feat(scanner): add cycle budget observability ( #3166 )
...
* feat(scanner): add cycle budget observability
* fix(scanner): clear cycle state after budget stop
* test(scanner): stabilize timeout-based scanner tests
* fix(scanner): keep cycle ILM counts scanner-only
* test(scanner): avoid test-only pending import
---------
Co-authored-by: Henry Guo <[email protected] [email protected] [email protected]
2026-06-03 03:32:41 +00:00
1d46047d6f
feat(scanner): expand scanner observability metrics ( #3159 )
...
* feat(scanner): expand scanner observability metrics
* chore(scanner): align bucket-drive metric wording
---------
Co-authored-by: Henry Guo <[email protected]
2026-06-02 01:29:53 +00:00
f3bd838925
feat(scanner): expose cycle progress metrics ( #3152 )
...
Co-authored-by: Henry Guo <[email protected] [email protected]
2026-06-01 07:54:39 +00:00
cbb6f9c76f
chore(deps): bump the dependencies group with 6 updates ( #3151 )
...
* chore(deps): bump the dependencies group with 6 updates
Bumps the dependencies group with 6 updates:
| Package | From | To |
| --- | --- | --- |
| [hyper](https://github.com/hyperium/hyper ) | `1.10.0` | `1.10.1` |
| [serial_test](https://github.com/palfrey/serial_test ) | `3.4.0` | `3.5.0` |
| [snafu](https://github.com/shepmaster/snafu ) | `0.9.0` | `0.9.1` |
| [uuid](https://github.com/uuid-rs/uuid ) | `1.23.1` | `1.23.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry ) | `0.3.12` | `0.3.13` |
| [pyroscope](https://github.com/grafana/pyroscope-rs ) | `2.0.5` | `2.0.6` |
Updates `hyper` from 1.10.0 to 1.10.1
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v1.10.0...v1.10.1 )
Updates `serial_test` from 3.4.0 to 3.5.0
- [Release notes](https://github.com/palfrey/serial_test/releases )
- [Commits](https://github.com/palfrey/serial_test/compare/v3.4.0...v3.5.0 )
Updates `snafu` from 0.9.0 to 0.9.1
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md )
- [Commits](https://github.com/shepmaster/snafu/compare/0.9.0...0.9.1 )
Updates `uuid` from 1.23.1 to 1.23.2
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.23.1...v1.23.2 )
Updates `dial9-tokio-telemetry` from 0.3.12 to 0.3.13
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases )
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md )
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.12...dial9-tokio-telemetry-v0.3.13 )
Updates `pyroscope` from 2.0.5 to 2.0.6
- [Release notes](https://github.com/grafana/pyroscope-rs/releases )
- [Changelog](https://github.com/grafana/pyroscope-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/grafana/pyroscope-rs/compare/lib-2.0.5...lib-2.0.6 )
---
updated-dependencies:
- dependency-name: hyper
dependency-version: 1.10.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: serial_test
dependency-version: 3.5.0
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: dependencies
- dependency-name: snafu
dependency-version: 0.9.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: uuid
dependency-version: 1.23.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
dependency-version: 0.3.13
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
- dependency-name: pyroscope
dependency-version: 2.0.6
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <[email protected] [email protected] [email protected]
2026-06-01 07:21:47 +00:00
76da2a48d0
feat(scanner): expose cycle observability controls ( #3147 )
...
Co-authored-by: Henry Guo <[email protected]
2026-05-31 21:55:46 +00:00
a99ef64db2
feat(scanner): add scanner budgets and progress metrics ( #3145 )
...
* fix(scanner): preserve maintenance scan cadence
* feat(scanner): add scanner concurrency budget
* feat(scanner): expose scanner runtime progress
* fix(scanner): address scanner review feedback
---------
Co-authored-by: Henry Guo <[email protected] [email protected]
2026-05-31 16:42:38 +00:00
houseme and GitHub
dcbffb084f
chore(deps): refresh workspace deps and linux fs_type gating ( #3030 )
...
* chore(deps): refresh workspace deps and linux fs_type gating
- refresh workspace dependency pins and lockfile updates
- remove now-unused crate dependency entries in multiple Cargo.toml files
- enable profiling export defaults in config and scripts/run.sh
- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings
* fix(utils): simplify fs_type linux gating
- keep fs_type module-level linux cfg in os::mod
- remove redundant linux cfg on get_fs_type and test module
* chore(deps): bump s3s git revision
- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5
- refresh Cargo.lock source entry for s3s
* chore(dev): allow mysql_async git source and env overrides
- allow mysql_async git source in deny.toml allow-git list
- make scripts/run.sh core env vars overrideable via existing shell env
* fix(utils): import get_fs_type in fs_type tests
- add explicit super::get_fs_type import in fs_type test module
- fix Linux E0425 unresolved function errors in unit tests
* chore(dev): tune run script observability defaults
- make profiling export env overrideable in scripts/run.sh
- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0
- update allow-git review window comments in deny.toml
* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
安正超 and GitHub
c684438625
fix(obs): add proxied PUT replication metrics ( #3020 )
...
fix(obs): add proxied put replication metrics
2026-05-20 05:30:57 +00:00
安正超 and GitHub
c727589161
fix(obs): remove stale replication metric TODOs ( #3024 )
2026-05-20 03:59:16 +00:00
houseme and GitHub
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
houseme and GitHub
bd1e57293f
fix: harden lifecycle transition compensation and regression coverage ( #2995 )
...
* fix(ecstore): honor transition worker configuration
* fix(ecstore): add transition queue backpressure metrics
* fix(ecstore): schedule transition compensation on enqueue pressure
* fix(ecstore): log transition compensation scheduling
* test(rustfs): add transition compensation fault-injection coverage
* test(rustfs): cover delete after transition compensation
* test(scanner): cover cleanup after transition compensation
* test(rustfs): extend compensation transition coverage
* test(scanner): cover backfill idempotency after compensation
* test(scanner): cover noncurrent expiry after compensation
* test(rustfs): cover versioned delete after compensation
* test(rustfs): cover delete marker lifecycle after compensation
* test(scanner): extend versioned lifecycle compensation coverage
* test(scanner): model versioned delete after compensation
* test(scanner): clarify modeled versioned delete helper
* refactor(ecstore): optimize transition enqueue hot path
* refactor(ecstore): centralize transition runtime constants
* style(ecstore): apply rustfmt for transition timeout helper
* fix(ilm): align queue-full metric semantics
* refactor(ecstore): unify immediate enqueue failure handling
* refactor(ecstore): reuse transition worker env constant
* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
bca8b08c2b
fix: handle Windows paths in pre-commit tests ( #2974 )
...
Co-authored-by: houseme <[email protected]
2026-05-15 14:04:51 +00:00
安正超 and GitHub
77c21dc5fd
test(obs): cover replication bandwidth tombstones ( #2906 )
2026-05-11 01:55:54 +00:00
61a7820651
optimize(obs):zero and expire removed replication bandwidth series ( #2901 )
...
Co-authored-by: houseme <[email protected]
2026-05-10 14:26:18 +00:00
houseme and GitHub
c90bfe2b23
fix(ecstore): harden runtime read-path quorum handling ( #2872 )
2026-05-08 09:56:39 +00:00
houseme and GitHub
c4d5c5c5ec
fix(obs): disable profiling export by default and fix Helm env name ( #2719 )
2026-04-28 11:57:15 +00:00
houseme and GitHub
59f41eb86a
feat(obs): improve metrics coverage and dashboard performance ( #2682 )
2026-04-25 18:51:29 +00:00
13b4500212
feat(obs): improve telemetry stack, replication metrics, and Grafana alignment ( #2672 )
...
Co-authored-by: Filipe Monteiro <[email protected] [email protected] [email protected] [email protected] [email protected]
2026-04-24 13:50:17 +00:00
houseme and GitHub
47247789ad
feat(obs): support OTLP headers and timeout overrides for HTTP exporters ( #2661 )
2026-04-23 17:02:26 +00:00
houseme and GitHub
116db4f5d9
refactor(metrics): unify process sampling and split network IO ( #2590 )
2026-04-18 15:30:44 +00:00
houseme and GitHub
1cbf156559
refactor(obs): migrate metrics runtime/schema and tighten migration guards ( #2584 )
2026-04-18 07:51:15 +00:00
Tunglies and GitHub
49366ee200
chore(lint): clippy rules redundant_clone ( #2554 )
2026-04-15 13:54:07 +00:00
6a2f98b468
fix(obs): respect error log level for http warnings ( #2492 )
...
Signed-off-by: 80347547 <[email protected] [email protected] [email protected]
2026-04-12 01:32:54 +08:00
8d27170ce4
fix: limit pyroscope profiling to supported Unix targets ( #2399 )
...
Signed-off-by: houseme <[email protected] [email protected] [email protected] [email protected]
2026-04-06 23:05:30 +08:00
fb2ced4d27
feat(obs): integrate dial9-tokio-telemetry for runtime tracing ( #2285 )
...
Co-authored-by: heihutu <[email protected] [email protected] [email protected]
2026-03-25 14:23:58 +08:00
houseme and GitHub
628481be7c
fix(tracing): Fix distributed tracing context linking ( #2247 )
2026-03-21 11:07:35 +08:00
3c28f0a0ba
feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics ( #2242 )
...
Co-authored-by: houseme <[email protected]
2026-03-20 18:52:33 +08:00
weisd and GitHub
b9b7d86ae4
feat: improve legacy metadata and admin compatibility ( #2202 )
2026-03-18 21:05:09 +08:00
94cdb89e29
feat(obs): add init_obs_with_config API and signature guard test ( #2175 )
...
Signed-off-by: houseme <[email protected] [email protected] [email protected]
2026-03-16 18:17:55 +08:00
houseme and GitHub
7f3459f5a8
fix(obs): fixed unresolved import super::local::ensure_dir_permissions ( #2164 )
2026-03-15 00:33:06 +08:00
f66a90c1b2
fix: fix github action error caused by oltp modification ( #2163 )
...
Co-authored-by: houseme <[email protected]
2026-03-14 22:15:35 +08:00
a1104b45f6
fix(obs): honor target-only rust_log directives ( #2159 )
...
Signed-off-by: houseme <[email protected] [email protected] [email protected]
2026-03-14 11:14:46 +08:00
6e0f034ad1
refactor(obs): enhance log rotation robustness and refine filter logic ( #2155 )
...
Signed-off-by: houseme <[email protected] [email protected]
2026-03-14 09:20:35 +08:00
593a58c161
refactor(obs): optimize logging with custom RollingAppender and improved cleanup ( #2151 )
...
Signed-off-by: houseme <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2026-03-13 13:20:27 +08:00
845ad1fa16
fix(obs): avoid panic in telemetry init and clamp sampler boundaries ( #2118 )
...
Signed-off-by: houseme <[email protected] [email protected]
2026-03-11 01:32:46 +08:00
f89cdfe5b3
update s3s 0.14.0-dev ( #2070 )
...
Co-authored-by: houseme <[email protected]
2026-03-04 01:07:24 +08:00
houseme and GitHub
5e7495a042
build(obs): restrict pyroscope dependency to unix targets ( #2064 )
2026-03-03 20:41:37 +08:00
ac4b13def1
feat(obs): Optional continuous CPU profiling with grafana pyroscope ( #2035 )
...
Signed-off-by: houseme <[email protected] [email protected] [email protected] [email protected]
2026-03-03 15:28:58 +08:00
08e1f4670b
fix: restore default CORS fallback and STS object ACL ownership ( #2053 )
...
Co-authored-by: houseme <[email protected]
2026-03-03 01:08:50 +08:00
houseme and GitHub
2ac07c95a8
refactor(obs): enhance log cleanup and rotation ( #2040 )
2026-03-02 16:28:32 +08:00
c452f24487
Optimize log cleanup and rotation, update dependencies ( #2032 )
...
Co-authored-by: heihutu <[email protected]
2026-03-01 20:09:52 +08:00
2c01b8c49d
feat(obs): add advanced log management configuration ( #2016 )
...
Co-authored-by: houseme <[email protected] [email protected] [email protected]
2026-03-01 03:23:48 +08:00
d983638391
build: update docker config and refine s3s region handling ( #1976 )
...
Co-authored-by: houseme <[email protected]
2026-02-27 01:21:12 +08:00
houseme and GitHub
d345ace326
perf(obs): optimize metrics recorder and telemetry initialization ( #1859 )
2026-02-18 07:05:43 +08:00
9da332c47d
perf(metrics): Cache metric handles instead of creating each call ( #1852 )
...
Co-authored-by: heihutu <[email protected]
2026-02-18 01:03:35 +08:00
8010284aa3
obs(export): Add env vars to selectivly disable exporting traces/metrics/logs ( #1853 )
...
Co-authored-by: houseme <[email protected]
2026-02-18 00:38:30 +08:00
9824171995
feat(observability): Add grafana dashboard, observability changes ( #1770 )
...
Co-authored-by: loverustfs <[email protected]
2026-02-11 15:55:08 +08:00
