* Add InvalidRangeSpec error
* Add EntityTooSmall to from_u32
* Add InvalidRangeSpec to from_u32
* Map InvalidRangeSpec to correct S3ErrorCode
* Return Error::InvalidRangeSpec
* Use auto implementation
* Add default storage class to ListObjectsV2
Resolves#764
* Add storage_class to response
* Make storage class optional so default won't be an empty string
---------
Co-authored-by: houseme <[email protected]>
* improve code for dns log
* fix
* Improve comments, remove unused parameters in config.rs (opt), add observability enable flag, and enhance error logging in run function execution.
- Reduce metrics push frequency from default to 3s for better performance
- Optimize resource utilization during metrics collection
- Improve real-time monitoring responsiveness
Related to admin metrics optimization on fix/admin-metrics branch
- Normalize ETags by removing quotes before comparison in complete_multipart_upload
- Fix ETag comparison in replication logic to handle quoted ETags from API responses
- Fix ETag comparison in transition object logic
- Add unit tests for trim_etag function
This fixes the ETag mismatch error when uploading large files (5GB+) via multipart upload,
which was caused by PR #592 adding quotes to ETag responses while internal storage remains unquoted.
Fixes#625
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <[email protected]>
* chore: rebase
Signed-off-by: junxiang Mu <[email protected]>
---------
Signed-off-by: junxiang Mu <[email protected]>
- Add lock timeout support and track acquisition time in lock state
- Improve lock conflict handling with detailed error messages
- Optimize lock reuse when already held by same owner
- Refactor lock state to store owner info and timeout duration
- Update all lock operations to handle new state structure
Signed-off-by: junxiang Mu <[email protected]>
* fix: fix datausageinfo
Signed-off-by: junxiang Mu <[email protected]>
* feat(data-usage): implement local disk snapshot aggregation for data usage statistics
Signed-off-by: junxiang Mu <[email protected]>
* feat(scanner): improve data usage collection with local scan aggregation
Signed-off-by: junxiang Mu <[email protected]>
* refactor: improve object existence check and code style
Signed-off-by: junxiang Mu <[email protected]>
---------
Signed-off-by: junxiang Mu <[email protected]>
* feat(kms): implement key management service with local and vault backends
Signed-off-by: junxiang Mu <[email protected]>
* feat(kms): enhance security with zeroize for sensitive data and improve key management
Signed-off-by: junxiang Mu <[email protected]>
* remove Hashi word
Signed-off-by: junxiang Mu <[email protected]>
* refactor: remove unused request structs from kms handlers
Signed-off-by: junxiang Mu <[email protected]>
---------
Signed-off-by: junxiang Mu <[email protected]>
* Refactor: reimplement lock
Signed-off-by: junxiang Mu <[email protected]>
* Fix: fix test case failed
Signed-off-by: junxiang Mu <[email protected]>
* Improve: lock pref
Signed-off-by: junxiang Mu <[email protected]>
* fix(lock): Fix resource cleanup issue when batch lock acquisition fails
Ensure that the locks already acquired are properly released when batch lock acquisition fails to avoid memory leaks
Improve the lock protection mechanism to prevent double release issues
Add complete Apache license declarations to all files
Signed-off-by: junxiang Mu <[email protected]>
---------
Signed-off-by: junxiang Mu <[email protected]>
* feat: support conditional writes
* refactor: avoid using unwrap
* fix: obtain lock before check in CompleteMultiPartUpload
* refactor: do not obtain a lock when getting object meta
* fix: avoid using unwrap and modifying incoming arguments
* test: add e2e tests for conditional writes
---------
Co-authored-by: guojidan <[email protected]>
Co-authored-by: 安正超 <[email protected]>
Co-authored-by: loverustfs <[email protected]>
* feat: add support for range requests in upload_part_copy and implement parse_copy_source_range function
* style: format debug and error logging for improved readability
* feat: implement parse_copy_source_range function and improve error handling in range requests
* Update rustfs/src/storage/ecfs.rs
Co-authored-by: Copilot <[email protected]>
* fix: correct return type in parse_copy_source_range function
* fix: remove unnecessary unwrap in parse_copy_source_range tests
* fix: simplify etag comparison in copy condition validation
---------
Co-authored-by: DamonXue <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: loverustfs <[email protected]>
* init audit logger module
* add audit webhook default config kvs
* feat: Add comprehensive tests for authentication module (#309)
* feat: add comprehensive tests for authentication module
- Add 33 unit tests covering all public functions in auth.rs
- Test IAMAuth struct creation and secret key validation
- Test check_claims_from_token with various credential types and scenarios
- Test session token extraction from headers and query parameters
- Test condition values generation for different user types
- Test query parameter parsing with edge cases
- Test Credentials helper methods (is_expired, is_temp, is_service_account)
- Ensure tests handle global state dependencies gracefully
- All tests pass successfully with 100% coverage of testable functions
* style: fix code formatting issues
* Add verification script for checking PR branch statuses and tests
Co-authored-by: anzhengchao <[email protected]>
* fix: resolve clippy uninlined format args warning
---------
Co-authored-by: Cursor Agent <[email protected]>
* feat: add basic tests for core storage module (#313)
* feat: add basic tests for core storage module
- Add 6 unit tests for FS struct and basic functionality
- Test FS creation, Debug and Clone trait implementations
- Test RUSTFS_OWNER constant definition and values
- Test S3 error code creation and handling
- Test compression format detection for common file types
- Include comprehensive documentation about integration test needs
Note: Full S3 API testing requires complex setup with storage backend,
global configuration, and network infrastructure - better suited for
integration tests rather than unit tests.
* style: fix code formatting issues
* fix: resolve clippy warnings in storage tests
---------
Co-authored-by: Cursor Agent <[email protected]>
* feat: add tests for admin handlers module (#314)
* feat: add tests for admin handlers module
- Add 5 new unit tests for admin handler functionality
- Test AccountInfo struct creation, serialization and default values
- Test creation of all admin handler structs (13 handlers)
- Test HealOpts JSON serialization and deserialization
- Test HealOpts URL encoding/decoding with proper field types
- Maintain existing test while adding comprehensive coverage
- Include documentation about integration test requirements
All tests pass successfully with proper error handling for complex dependencies.
* style: fix code formatting issues
* fix: resolve clippy warnings in admin handlers tests
---------
Co-authored-by: Cursor Agent <[email protected]>
* build(deps): bump the dependencies group with 3 updates (#326)
* perf: avoid transmitting parity shards when the object is good (#322)
* upgrade version
* Fix: fix data integrity check
Signed-off-by: junxiang Mu <[email protected]>
* Fix: Separate Clippy's fix and check commands into two commands.
Signed-off-by: junxiang Mu <[email protected]>
* fix: miss inline metadata (#345)
* Update dependabot.yml
* fix: Fixed an issue where the list_objects_v2 API did not return dire… (#352)
* fix: Fixed an issue where the list_objects_v2 API did not return directory names when they conflicted with file names in the same bucket (e.g., test/ vs. test.txt, aaa/ vs. aaa.csv) (#335)
* fix: adjusted the order of directory listings
* init
* fix
* fix
* feat: add docker usage for rustfs mcp (#365)
* feat: enhance metadata extraction with object name for MIME type detection
Signed-off-by: junxiang Mu <[email protected]>
* Feature: lock support auto release
Signed-off-by: junxiang Mu <[email protected]>
* improve lock
Signed-off-by: junxiang Mu <[email protected]>
* Fix: fix scanner detect
Signed-off-by: junxiang Mu <[email protected]>
* Fix: clippy && fmt
Signed-off-by: junxiang Mu <[email protected]>
* refactor(ecstore): Optimize memory usage for object integrity verification
Change the object integrity verification from reading all data to streaming processing to avoid memory overflow caused by large objects.
Modify the TLS key log check to use environment variables directly instead of configuration constants.
Add memory limits for object data reading in the AHM module.
Signed-off-by: junxiang Mu <[email protected]>
* Chore: reduce PR template checklist
Signed-off-by: junxiang Mu <[email protected]>
* Chore: remove comment code (#376)
Signed-off-by: junxiang Mu <[email protected]>
* chore: upgrade actions/checkout from v4 to v5 (#381)
* chore: upgrade actions/checkout from v4 to v5
- Update GitHub Actions checkout action version
- Ensure compatibility with latest workflow features
- Maintain existing checkout behavior and configuration
* upgrade version
* fix
* add and improve code for notify
* feat: extend rustfs mcp with bucket creation and deletion (#416)
* feat: extend rustfs mcp with bucket creation and deletion
* update file to fix pipeline error
* change variable name to fix pipeline error
* fix(ecstore): add async-recursion to resolve nightly trait solver reg… (#415)
* fix(ecstore): add async-recursion to resolve nightly trait solver regression
The newest nightly compiler switched to the new trait solver, which
currently rejects async recursive functions that were previously accepted.
This causes the following compilation failures:
- `LocalDisk::delete_file()`
- `LocalDisk::scan_dir()`
Add `async-recursion` as a workspace dependency and annotate both functions with `#[async_recursion]` so that the crate compiles cleanly with the latest nightly and will continue to build once the new solver lands in stable.
Signed-off-by: reigadegr <[email protected]>
* fix: resolve duplicate bound error in scan_dir function
Replaced inline trait bounds with where clause to avoid duplication caused by macro expansion.
Signed-off-by: reigadegr <[email protected]>
---------
Signed-off-by: reigadegr <[email protected]>
Co-authored-by: 安正超 <[email protected]>
* fix:make bucket exists (#428)
* feat: include user-defined metadata in S3 response (#431)
* fix: simplify Docker entrypoint following efficient user switching pattern (#421)
* fix: simplify Docker entrypoint following efficient user switching pattern
- Remove ALL file permission modifications (no chown at all)
- Use chroot --userspec or gosu to switch user context
- Extremely simple and fast implementation
- Zero filesystem modifications for permissions
Fixes#388
* Update entrypoint.sh
Co-authored-by: Copilot <[email protected]>
* Update entrypoint.sh
Co-authored-by: Copilot <[email protected]>
* Update entrypoint.sh
Co-authored-by: Copilot <[email protected]>
* wip
* wip
* wip
---------
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: Copilot <[email protected]>
* docs: update doc/docker-data-dir README.md (#432)
* add targets crates
* feat(targets): extract targets module into a standalone crate
- Move all target-related code (MQTT, Webhook, etc.) into a new `targets` crate
- Update imports and dependencies to reference the new crate
- Refactor interfaces to ensure compatibility with the new crate structure
- Adjust Cargo.toml and workspace configuration accordingly
* fix
* fix
---------
Signed-off-by: junxiang Mu <[email protected]>
Signed-off-by: reigadegr <[email protected]>
Co-authored-by: 安正超 <[email protected]>
Co-authored-by: Cursor Agent <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: zzhpro <[email protected]>
Co-authored-by: junxiang Mu <[email protected]>
Co-authored-by: weisd <[email protected]>
Co-authored-by: shiro.lee <[email protected]>
Co-authored-by: majinghe <[email protected]>
Co-authored-by: guojidan <[email protected]>
Co-authored-by: reigadegr <[email protected]>
Co-authored-by: 0xdx2 <[email protected]>
Co-authored-by: Copilot <[email protected]>
