build(rust): Manage crate dependency versions centrally through the Cargo workspace; Upgrade Rust dependencies to their latest compatible versions.

[LinZhihao-723]

Description

This PR introduces workspace-level dependency and package inheritance so that every external crate is declared exactly once, in the root Cargo.toml, instead of being pinned independently in each of the five member manifests. Before this change a version bump was an N-file edit, and several crates had already drifted apart across members: tokio was simultaneously 1.49.0 and 1.52.3, clap was 4.5.56 and 4.6.4, tracing was 0.1 and 0.1.44, and tracing-subscriber was 0.3.22 and 0.3.23. All drift is resolved, and every dependency is aligned to the newest semver-compatible release.

No Rust source file is modified by this PR — it is confined to manifests and Cargo.lock. This holds even for the two dependency majors taken below.

All five members additionally move from a repeated version = "0.13.1-dev" / edition = "2024" to workspace inheritance. The value is unchanged, so tools/deployment/package-helm/Chart.yaml, taskfile.yaml, and the Python components stay in sync without an edit.

Workspace inheritance (root Cargo.toml)

• Adds [workspace.package] holding the two fields every member repeated identically: version = "0.13.1-dev" and edition = "2024".
• Adds [workspace.dependencies] with 45 external crates, the three Spider git dependencies, and the clp-rust-utils path dependency. clp-rust-utils is now referenced by name rather than by { path = "../clp-rust-utils" } in each of the four members that depend on it.
• resolver and members are unchanged.

Member manifests (5 crates)

• [package] now uses version = { workspace = true } and edition = { workspace = true }.
• Every entry in [dependencies] and [dev-dependencies] becomes { workspace = true }. Members retain only the features they need in addition to what the workspace entry already provides, so the common case collapses to a single bare inheritance.
• The { workspace = true } inline-table form is used throughout rather than the dotted dep.workspace = true form.

Features that remain member-local, because hoisting them would enable a feature in a member that did not declare it:

Crate	Feature	Kept local to	Why not hoisted
sqlx	chrono	api-server	Nothing else in the graph enables sqlx/chrono, so hoisting would pull the chrono crate into the four members that don't use it.
utoipa	axum_extras	api-server, log-ingestor	axum_extras activates dep:axum; clp-rust-utils has no axum dependency and would grow one on a standalone build.
strum	derive	clp-rust-utils	log-ingestor depends on strum_macros directly and doesn't need strum/derive.
opentelemetry-otlp	reqwest-blocking-client	clp-rust-utils	Only the blocking exporter path needs it; keeping it local documents who does.
tokio	full / fs / net / macros / signal	all four users	The per-member feature sets are largely disjoint. The workspace entry carries only the strict intersection, ["rt-multi-thread", "time"]; hoisting full would give three members process/io-util/net/fs on standalone builds.

default-features = false moves up to the workspace

opentelemetry-otlp is the only crate declared with default-features = false in a member, and both api-server and clp-rust-utils did so. Cargo silently ignores a member's default-features = false when the workspace entry leaves defaults enabled, so this key now lives on the workspace entry instead; leaving it in the members would have quietly re-enabled default features. No member declares default-features any more.

Dependency version alignment

Aligned to the newest compatible release

Where a crate had drifted, the drifted requirements are listed together in "Before".

Crate	Before	After	Note
anyhow	1.0.100	1.0.104
async-trait	0.1.89	0.1.92	Steps over yanked 0.1.90
aws-config	1.8.12	1.10.1
aws-sdk-s3	1.121.0	1.141.0
aws-sdk-sqs	1.92.0	1.105.0
axum	0.8.8	0.8.9
brotli	8.0.2	8.0.4
chrono	0.4	0.4.45	Requirement was not patch-exact
clap	4.5.56, 4.6.4	4.6.6	Drift resolved
const_format	0.2.35	0.2.36
futures	0.3.31	0.3.33
http-body-util	0.1	0.1.4	Requirement was not patch-exact
mongodb	3.5.0	3.8.0
num_enum	0.7.5	0.7.6
pin-project-lite	0.2.16	0.2.17
regex	1.12.3	1.13.1
serde	1.0.228	1.0.229
serde_json	1.0.149	1.0.151
serial_test	3.3.1	4.0.1	Major taken; see below
thiserror	2.0.18	2.0.20
tokio	1.49.0, 1.52.3	1.53.1	Drift resolved
tokio-util	0.7.18	0.7.19
tower	0.5	0.5.3	Requirement was not patch-exact
tower-http	0.6.8	0.7.0	Major taken; see below
tracing	0.1, 0.1.44	0.1.44	Drift resolved; requirement was not patch-exact
tracing-appender	0.2.4	0.2.5
tracing-subscriber	0.3.22, 0.3.23	0.3.23	Drift resolved
utoipa	5.4.0	5.5.0
uuid	1.20.0	1.24.0

Already newest and unchanged: async-stream 0.3.6, hex 0.4.3, non-empty-string 0.2.6, opentelemetry and opentelemetry-otlp 0.32.0, opentelemetry_sdk 0.32.1, rmp-serde 1.3.1, secrecy 0.10.3, strsim 0.11.1, strum and strum_macros 0.28.0, tonic 0.14.6, url 2.5.8, utoipa-axum 0.2.0, yaml_serde 0.10.4.

Every entry in [workspace.dependencies] is now patch-exact, per the repository convention. chrono, tracing, tower, and http-body-util previously named no patch version, and those four pre-existing deviations are corrected here.

Breaking majors taken

Both require zero source changes, verified by cargo check --workspace --all-targets and cargo clippy --workspace --all-targets -- -D warnings.

tower-http 0.6.8 → 0.7.0. CLP's entire use of the crate is use tower_http::cors::{Any, CorsLayer} plus CorsLayer::new().allow_origin(Any) in api-server/src/routes.rs and log-ingestor/src/routes.rs. Every item on 0.7.0's breaking list lands in compression (RFC 9110 * / identity;q=0 handling, SizeAbove widened u16 → u64), follow-redirect (request Extensions now forwarded across hops), trace / classify (GrpcCode and GrpcFailureClass become #[non_exhaustive]), fs / services (trailing-slash paths now 404), or the removed no-op tokio / async-compression features — none of which CLP references. The cors public API is unchanged, and 0.7.0 does not bump its http major (its manifest still declares http = "1.0", tower-layer = "0.3.3", and tower-service = "0.3"), so it stays compatible with axum 0.8.9 and tonic 0.14.6, both of which resolve to http 1.4.0.

One client-observable change is worth calling out: 0.7.0 relaxes the CORS Vary default ([tower-rs/tower-http#674]), so a wildcard configuration like ours stops emitting Vary: origin, access-control-request-method, access-control-request-headers. This is spec-correct and cache-safe — a wildcard response genuinely doesn't vary per request — but it is a real change to the API server's response headers.

Note that reqwest 0.13.4, reached via opentelemetry-otlp → opentelemetry-http, pins tower-http ^0.6.8 for its follow-redirect feature, so tower-http 0.6.10 remains in the lock as a second compiled copy. The repository has no duplicate-crate lint, so this is cosmetic.

serial_test 3.3.1 → 4.0.1, a dev-dependency of log-ingestor only. The !-marked change in 4.0.0 is refactor(derive)!: bump syn to v3, which is internal to the proc-macro crate: the full v3.5.0...v4.0.1 compare touches only ci.yml, Cargo.lock, four Cargo.toml files, README.md, and two lines inside mod tests in serial_test_derive/src/lib.rs. serial_test/src/ has zero changes, so the #[serial] attribute surface CLP uses is untouched. 4.0.1 (rather than 4.0.0, which was yanked) raises the MSRV to 1.93.1; the repository pins no toolchain and CI uses the runner's stable. As a bonus, this drops scc and sdd from the lock.

Breaking major available, deliberately not taken

Crate	Pinned at	Major available	Why deferred
sqlx	0.8.6	0.9.0	Would split the dependency graph against Spider; see below

sqlx is a hard block rather than a soft deferral. spider-core pins sqlx 0.8.6 and implements sqlx::Type, Encode, and Decode for its Id<T> against 0.8's MySql, and compression-coordinator's coordination.rs decodes a SpiderJobId through #[derive(sqlx::FromRow)]. Because 0.9 is a semver-major for a 0.x crate, taking it would put two incompatible sqlx copies in the graph and turn that derive into a compile error, not merely a duplicate crate. Independently, 0.9 narrows query*() to impl SqlSafeStr, which would require AssertSqlSafe(..) at the sites that build SQL with format!, and it removes the lifetime parameter from ArgumentBuffer. This can only move after [y-scope/spider] does.

Notes

• Resolved-feature verification. Comparing cargo metadata --filter-platform x86_64-unknown-linux-gnu between a pristine checkout and this branch, the centralization itself produces zero resolved-feature changes, both workspace-wide and per member. Eight packages differ at the same version, and all eight trace to version bumps rather than to the migration: crypto-bigint, generic-array, getrandom, ipnet, num-traits, and syn gain features transitively; tower-http 0.6.10 loses cors because that moved to the new 0.7.0 copy; and md-5 0.10.6 loses default / std purely because mongodb 3.8 moved to md-5 0.11, leaving sqlx-mysql and sqlx-postgres — which both ask for default-features = false — as its only consumers. No package loses a capability it requested.
• Feature widening from hoisting. Four member manifests now declare features they previously didn't: secrecy/serde in api-server, compression-coordinator, and log-ingestor; non-empty-string/serde in compression-coordinator; and tracing-subscriber/fmt,std in clp-tdl-package. All five are no-ops in the resolved graph, because every member depends on clp-rust-utils, which already enabled each of them. This is inherent to Cargo's feature unioning; avoiding it would mean removing the feature from the workspace entry and repeating it in the majority of members.
• Lock footprint. Cargo.lock grows from 514 to 529 entries, but on x86_64-unknown-linux-gnu the compiled set moves from 431 to 434, a net of +3. Nine crates are added (arc-swap, aws-smithy-schema, chacha20, hickory-net, prefix-trie, primeorder, rand 0.10, rand_core 0.10, and tower-http 0.7) and six stop compiling (enum-as-inner, plus the stale duplicates crypto-bigint 0.4.9, der 0.6.1, pkcs8 0.9.0, signature 1.6.4, and spki 0.6.0, all collapsed onto copies already present). The remaining new lock entries — jni and friends, walkdir, system-configuration, winapi-util — arrive under mongodb 3.8's new hickory-* DNS stack and are Android-, Apple-, or Windows-gated, so none of them builds on the platform CLP ships.
• aws-config 1.10.1 changes the identity-cache load_timeout from a hardcoded 5s to a value derived from RetryConfig (roughly 22s). CLP configures no IdentityCache or TimeoutConfig, so credential resolution against a slow or unreachable STS/IMDS endpoint can now block longer before failing. This is behavior-only; an explicit load_timeout can be set in a follow-up if the longer wait is undesirable.
• regex 1.13.1 fixes a reverse-suffix offset bug present in 1.12.3, so that bump is a correctness improvement rather than only a version alignment.
• Generated OpenAPI docs are unaffected. utoipa 5.4.0 → 5.5.0 produces byte-identical output: re-running task codegen:openapi leaves docs/src/_static/generated/api-server-openapi.json and docs/src/_static/generated/log-ingestor-openapi.json unchanged.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Ensure all workflows pass.

Summary by CodeRabbit

• Tests
  • Improved partitioning tests to verify that data remains empty until explicitly flushed, then contains the expected results.
• Chores
  • Standardized package versions, Rust editions, and dependency configuration across workspace components.
  • Centralized dependency management to improve consistency and simplify project maintenance.

[coderabbitai]

Walkthrough

The workspace now centralizes package metadata and dependency declarations. Component manifests inherit these settings while retaining required features. Partition tests now assert task input state before and after flushing.

Changes

Workspace dependency centralization

Layer / File(s)	Summary
Workspace metadata and dependency contract Cargo.toml	The workspace defines shared package metadata and dependency declarations, including versions, sources, paths, and features.
Component manifest adoption components/api-server/Cargo.toml, components/clp-rust-utils/Cargo.toml, components/clp-tdl-package/Cargo.toml, components/compression-coordinator/Cargo.toml, components/log-ingestor/Cargo.toml	Component packages inherit workspace versions and dependency settings. Required local features remain specified for selected dependencies.
Partition test assertions components/compression-coordinator/src/partition.rs	Partition tests verify empty task inputs before flushing and non-empty task inputs after flushing.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• y-scope/clp#2375: It also changes shared dependency management in clp-rust-utils, including centralized compression dependency configuration.
• y-scope/clp#2404: It changes the clp-tdl-package and compression-coordinator manifests that this PR centralizes.

Suggested reviewers: davidlion

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the two main changes: centralised workspace dependency management and compatible Rust dependency upgrades.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[LinZhihao-723]

This PR should fix the failing linting by upgrading async-trait to the latest release: https://github.com/dtolnay/async-trait/releases/tag/0.1.92

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Cargo.toml`:
- Around line 16-64: Update the dependency declarations in Cargo.toml to use
exact =version requirements for all registry dependencies currently expressed
with compatible ranges, and replace the Spider Git dependencies’ branch = "main"
selectors with explicit rev values for the intended commits. Keep the existing
dependency set and feature configuration unchanged while ensuring cargo update
cannot select newer compatible releases or branch commits.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 96e0e258-1d66-4d5a-b7a5-74577b1efa56

📥 Commits

Reviewing files that changed from the base of the PR and between 4c72731 and ba08a8d.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (7)

• Cargo.toml
• components/api-server/Cargo.toml
• components/clp-rust-utils/Cargo.toml
• components/clp-tdl-package/Cargo.toml
• components/compression-coordinator/Cargo.toml
• components/compression-coordinator/src/partition.rs
• components/log-ingestor/Cargo.toml

[jackluo923]

Nit: "No Rust source file is modified by this PR" — but partition.rs is modified. Suggested:

One Rust source file is modified: partition.rs, where three assertions are rewritten by clippy::assert_is_empty's autofix, which main needs regardless of this PR. Everything else is confined to manifests and Cargo.lock.