Skip to content

Optimize dev proxy upstream performance#896

Merged
aram356 merged 32 commits into
mainfrom
perf/dev-proxy-optimization
Jul 14, 2026
Merged

Optimize dev proxy upstream performance#896
aram356 merged 32 commits into
mainfrom
perf/dev-proxy-optimization

Conversation

@aram356

@aram356 aram356 commented Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #895

Summary

  • Add repeatable dev-proxy performance metrics and benchmark workloads for sequential, matched-concurrency, saturation, DNS, parser, and remote-latency scenarios.
  • Add a bounded HTTP/1.1 upstream connection manager with safe streaming lease return, cancellation, stale-connection retry, connector ownership, and bounded shutdown.
  • Retain evidence-backed CONNECT-head parsing and DNS caching while rejecting HTTP/2 and TCP_NODELAY experiments that did not satisfy their retention gates.
  • Precompute upstream identities and preserve TLS, Host, --resolve, Basic authentication, hop-by-hop sanitation, and blind-tunnel isolation.

Final review corrections

  • Count complete responses as successful independently from whether their upstream connection is reusable, including already-empty and Connection: close responses.
  • Force stale-readiness recovery through a fresh reservation instead of selecting another idle sender.
  • Remove the rejected HTTP/2 application-mode key state; retained TLS configurations advertise only HTTP/1.1.
  • Forward response trailers before lease reconciliation and preserve every downstream trailer declaration.
  • Backfill connector cancellation/unwind, early-response, browser-cancellation, truncated-body, trailer, credential-isolation, insecure-warning, forwarding-capacity, over-read, shutdown-order, and origin-key integration coverage.

Performance results

  • Sequential TLS: 100 connections and handshakes reduced to 1, with a 59.7% median duration improvement in the recorded alternating runs.
  • DNS churn workload: 13.6% median improvement with one miss and 99 cache hits.
  • Concurrency-20 saturation: bounded to six upstream connections and handshakes.
  • Remote saturation diagnostic: six pooled connections reduce connection and handshake churn by 94%; the six-connection safety cap intentionally trades 9.0% duration versus the unbounded baseline at concurrency 20.

Full measurements and experiment decisions are recorded in docs/superpowers/implementation-notes/2026-07-10-dev-proxy-performance.md.

Verification

  • cargo fmt --all -- --check
  • CLI cargo check --tests and Clippy with -D warnings
  • ./scripts/test-cli.sh — 147 unit + 26 E2E tests
  • All six ignored proxy_perf workloads with one test thread
  • Fastly, Axum, Cloudflare, Spin, and 13 cross-adapter parity tests
  • Native/Wasm Clippy matrices for Fastly, Axum, Cloudflare, and Spin with zero warnings
  • JS build + 405 Vitest tests + JS/docs Prettier checks

aram356 added 19 commits July 10, 2026 12:13
Replace the verbose `cargo run --manifest-path … --target … -- dev proxy`
invocations with the installed `ts` binary (via `cargo install-cli`), so the
guide reads the same way as the CLI guide. Note that `ts dev proxy` is
macOS-only and phrase the install step as "install or update".

Make the quick start a complete first run (`ca install`, then run with
`--rewrite-host` and `--launch chrome`), and make `--rewrite-host` the
recommended default for dev and staging upstreams. The Host-header guidance now
turns on whether the upstream accepts `Host: <FROM>` rather than on the upstream
being Trusted Server Compute.

Correct several stale or wrong examples, verified against the CLI source:

- Firefox `certutil` now uses `-d "sql:…"` and the full CA nickname, matching the
  auto-import the tool performs.
- Add the missing `--connect-timeout` option and note that `[COMMAND]` is `ca`.
- Prefer `--basic-auth-file` over inline `--basic-auth` in the staging example.
- Clarify the CA key is stored outside the repository only by default, and rename
  "First run: CA setup" to "The development CA".
Keep `--rewrite-host` in the quick start (it is required for upstreams that reject
`Host: <FROM>`), but state its trade-off honestly: against a real Trusted Server
adapter, first-party URLs then render on the upstream host, not the production
domain. Qualify the "first-party URLs always stay on the production domain" claim
accordingly — it holds only when the upstream preserves `X-Forwarded-Host`.

Also:

- Note that Firefox CA auto-import needs `certutil` (`brew install nss`) and
  qualify the troubleshooting row that assumed it is always present.
- State that `--launch` opens only the first mapped hostname while every mapping
  is still proxied.
- Note that connection options (`--rewrite-host`, auth, `--insecure`,
  `--upstream-plaintext`) apply to every mapping, not per-rule.
- Scope the Safari "only while the proxy runs" claim to a clean exit.
@aram356 aram356 self-assigned this Jul 12, 2026
@aram356
aram356 requested review from ChristianPavilonis and prk-Jr and removed request for ChristianPavilonis July 12, 2026 19:12

@prk-Jr prk-Jr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds a bounded, reusable HTTP/1.1 upstream connection manager (plus DNS caching, precomputed upstream identities, and a repeatable benchmark suite) to ts dev proxy. The design is careful and thoroughly tested: the manager is an actor with a bounded ordinary lane + unbounded control lane so lifecycle events (cancel / driver-closed / shutdown) can never deadlock behind a saturated acquire lane, and capacity reconciliation is airtight — every path leads back to release(id) (reservation drop → ConnectFailed, driver-guard drop → DriverClosed, shutdown connector abort → finish error → reservation drop). The AcquireTicket single-terminal-transition invariant closes the resolve/cancel race. No blocking issues found; approving.

Non-blocking

♻️ refactor

  • Crypto provider rebuilt per call: NoVerifier::supported_verify_schemes calls aws_lc_rs::default_provider() (a non-trivial clone) on every invocation — cache in a OnceLock. --insecure-only path, cosmetic. (connect.rs:282)

🌱 seedling

  • Non-idempotent 502 on stale reuse: the send_request retry is gated on provably-empty idempotent requests, so a POST racing an upstream idle-close returns 502 rather than replaying. This is the correct safe behavior; flagged only so it's a documented trade-off. (upstream/mod.rs:246)

⛏ nitpick

  • Doc comments on new pub items: several new public items (ProxyMetrics::record_*, PooledResponseBody::new, etc.) lack doc comments. CLAUDE.md asks for one per public item; missing_docs isn't enforced here and it matches surrounding CLI style, so optional.

👍 praise

  • Lifecycle & concurrency design: the can_reuse matrix + exhaustive test, DNS miss-coalescing that survives owner cancellation without poisoning the in-flight key, the trailer-frame deferral in PooledResponseBody, and the priority_shutdown_overtakes_a_saturated_ordinary_lane test are all excellent.
  • Security hardening: Basic-auth refused on non-loopback binds, hop-by-hop + Connection-token stripping, Forwarded / Fastly-SSL stripped and authoritatively re-stamped, strict host validation before PAC / Host-header embedding, and redacted BasicAuth Debug.

CI Status

  • fmt: PASS
  • clippy (fastly/axum/cloudflare/spin native+wasm): PASS
  • rust tests (fastly/axum/cloudflare/spin/CLI/parity): PASS
  • js build + vitest + docs/ts prettier: PASS

Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/connect.rs Outdated
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/mod.rs
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/body.rs

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Approved with seven actionable follow-ups noted inline. The pooled HTTP/1 architecture and coverage are strong; the highest-priority changes are preserving request trailers end-to-end and tying connector-capacity release to actual task termination.

Comment thread crates/trusted-server-cli/src/commands/dev/proxy/server.rs
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/manager.rs Outdated
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/mod.rs Outdated
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/upstream/dns.rs Outdated
Comment thread crates/trusted-server-cli/tests/proxy_perf.rs
Comment thread crates/trusted-server-cli/src/commands/dev/proxy/metrics.rs
aram356 added 2 commits July 14, 2026 09:09
Only unmatched hosts reach the blind tunnel, and real publisher pages
reference many third-party domains that are dead, blackholed, or
DNS-blocked. Logging each failure at warn level drowned out failures on
mapped upstreams, so log them at debug instead. The browser still
receives its 502 and the failure remains visible at debug level.
@aram356
aram356 merged commit 12eb256 into main Jul 14, 2026
19 checks passed
@aram356
aram356 deleted the perf/dev-proxy-optimization branch July 14, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize dev proxy upstream performance

3 participants