Skip to content

Commit 0ae8d51

Browse files
lesnik512claude
andcommitted
docs(planning): spec Bulkhead middleware (0.5.0, Epic 3 slice 2)
Concurrency limiter middleware via asyncio.Semaphore. Required max_concurrent (no default — no universal correct value), bounded acquire wait (default 1.0s, None=forever, 0=fail-fast), maps acquire timeout to BulkheadFullError (distinct from httpware.TimeoutError). Bulkhead is the shareable unit — no separate BulkheadLimit type. Recommended ordering: Bulkhead outside Retry so a retrying request holds one slot across all attempts rather than re-acquiring per retry. Out of scope (v1): per-host partitioning, queue metrics, fallback callbacks. All purely additive later. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent a24630e commit 0ae8d51

1 file changed

Lines changed: 206 additions & 0 deletions

File tree

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Spec: Bulkhead middleware (0.5.0, Epic 3 slice 2)
2+
3+
**Date:** 2026-06-05
4+
**Topic slug:** `bulkhead`
5+
**Status:** drafted, awaiting user review
6+
**Target release:** 0.5.0 (0.4.0 release notes already drafted without this slice).
7+
**Epic 3 stories rolled in:** 3-5 (Bulkhead).
8+
9+
## Purpose
10+
11+
Ship `httpware`'s second resilience primitive: a `Bulkhead` middleware that caps in-flight requests through an `AsyncClient`. This is distinct from `httpx2.Limits`, which caps the *connection pool* — Bulkhead caps the *number of concurrent requests* the caller has waiting on the network at once. Use cases:
12+
13+
- Protect a downstream service from being overwhelmed by a busy client.
14+
- Bound the client's own resource usage (memory, response handlers in flight).
15+
- Fail fast under load instead of building up unbounded queues of waiting requests.
16+
17+
This is slice 2 of Epic 3; the only remaining Epic 3 work after this is `3-6` extension-slot docs (rides with the last code PR or in a docs-only follow-up).
18+
19+
## Non-goals
20+
21+
Items deliberately deferred so this slice ships clean:
22+
23+
- **No per-host / per-route partitioning.** One Bulkhead = one semaphore. If a future caller needs per-downstream caps, they install separate Bulkheads on separate clients (or wait for a future spec that adds keying). Adding partitioning later is purely additive.
24+
- **No separate `BulkheadLimit` type.** Unlike `Retry`+`RetryBudget` (where the budget needs to be separable to share across multiple `Retry` middlewares), `Bulkhead` is *already* the sharable unit — reuse the instance across `AsyncClient(middleware=[shared])` calls.
25+
- **No queue metrics surface.** No `bulkhead.in_flight` / `bulkhead.queued` properties. Observability lands in Epic 5 (`5-1` Layer 1 hooks); revisit then.
26+
- **No fallback / shed-load callbacks.** Caller catches `BulkheadFullError` and decides what to do.
27+
- **No connection-pool integration.** `Bulkhead` is a middleware; `httpx2.Limits` is a transport-level concern. They compose but stay independent.
28+
29+
## Architecture & module layout
30+
31+
```text
32+
src/httpware/middleware/resilience/
33+
├── __init__.py # existing; add Bulkhead re-export
34+
├── budget.py # existing
35+
├── retry.py # existing
36+
├── _backoff.py # existing
37+
└── bulkhead.py # NEW
38+
```
39+
40+
`Bulkhead` middleware in `bulkhead.py`. `BulkheadFullError(ClientError)` lands in `src/httpware/errors.py` alongside the existing exception tree. Both re-exported from `httpware/__init__.py`. No new optional extra — pure stdlib (`asyncio.Semaphore`, `asyncio.timeout`).
41+
42+
File-per-middleware mirrors slice-1's layout — each unit is independently testable.
43+
44+
## Public API
45+
46+
### `Bulkhead` middleware
47+
48+
```python
49+
class Bulkhead:
50+
"""Concurrency limiter middleware. See module docstring for behavior."""
51+
52+
def __init__(
53+
self,
54+
*,
55+
max_concurrent: int, # required; no default
56+
acquire_timeout: float | None = 1.0, # seconds; None = wait forever; 0 = fail fast
57+
) -> None: ...
58+
59+
async def __call__(
60+
self,
61+
request: httpx2.Request,
62+
next: Next,
63+
) -> httpx2.Response: ...
64+
```
65+
66+
Constructor validation:
67+
68+
- `max_concurrent < 1` → `ValueError("max_concurrent must be >= 1")`
69+
- `acquire_timeout is not None and acquire_timeout < 0` → `ValueError("acquire_timeout must be >= 0")`
70+
- `acquire_timeout == 0` is accepted (fail-fast: no wait).
71+
72+
Internal state: one `asyncio.Semaphore(max_concurrent)` instantiated in `__init__`. Created with the running event loop's binding rules — semaphores in asyncio are not loop-bound until first use in 3.10+, so the middleware is safe to construct outside an event loop.
73+
74+
`max_concurrent` is required because there is no universally-correct default — the right value depends on downstream capacity, request latency, and the caller's SLA. Forcing the choice avoids hiding a production-shaping default.
75+
76+
### `BulkheadFullError`
77+
78+
```python
79+
class BulkheadFullError(ClientError):
80+
"""Raised when acquire_timeout elapses before a Bulkhead slot becomes available."""
81+
82+
max_concurrent: int
83+
acquire_timeout: float | None
84+
85+
def __init__(self, *, max_concurrent: int, acquire_timeout: float | None) -> None:
86+
self.max_concurrent = max_concurrent
87+
self.acquire_timeout = acquire_timeout
88+
super().__init__(
89+
f"bulkhead full (max_concurrent={max_concurrent}, "
90+
f"acquire_timeout={acquire_timeout})"
91+
)
92+
93+
def __reduce__(self) -> tuple[Any, ...]: ... # picklable via module-level reconstructor
94+
```
95+
96+
Picklable via the same `_reconstruct_*` pattern used for `StatusError` and `RetryBudgetExhaustedError` (see `src/httpware/errors.py:61-62, 149-156`). Tested explicitly.
97+
98+
`BulkheadFullError` inherits from `ClientError` so callers catching `ClientError` already handle it. We do NOT inherit from `httpware.TimeoutError` — semantically, a bulkhead-full event is a backpressure signal, not a network timeout. Conflating them would mislead callers writing observability/alerting around timeout vs overload.
99+
100+
## Behavior
101+
102+
### `Bulkhead.__call__` algorithm
103+
104+
```python
105+
async def __call__(self, request, next):
106+
try:
107+
if self._acquire_timeout is None:
108+
await self._sem.acquire()
109+
else:
110+
async with asyncio.timeout(self._acquire_timeout):
111+
await self._sem.acquire()
112+
except asyncio.TimeoutError as exc:
113+
raise BulkheadFullError(
114+
max_concurrent=self._max_concurrent,
115+
acquire_timeout=self._acquire_timeout,
116+
) from exc
117+
118+
try:
119+
return await next(request)
120+
finally:
121+
self._sem.release()
122+
```
123+
124+
Why two-stage (`acquire` then `try/finally`) rather than `async with self._sem`:
125+
126+
- If we used `async with self._sem`, a CancelledError fired between the `__aenter__` return and the start of `next()` would still pass through `__aexit__`, which is fine — `asyncio.Semaphore.__aexit__` releases. But it is too easy to misread the structure as "the timeout wraps both acquire and execute," which is wrong.
127+
- The explicit acquire makes the contract obvious: only the acquisition is bounded by `acquire_timeout`; `next()` runs as long as it needs to.
128+
- The `try/finally` makes release deterministic across every exit path (success, exception from `next()`, cancellation during `next()`).
129+
130+
### Cancellation semantics
131+
132+
- **Cancellation before `acquire()` returns**: `CancelledError` propagates; no slot held; no release.
133+
- **Cancellation during `next()`**: `try/finally` releases the slot; `CancelledError` propagates.
134+
- **Cancellation between `acquire()` return and start of `next()`** (vanishingly small window): `try/finally` releases the slot; `CancelledError` propagates.
135+
136+
### Middleware chain position
137+
138+
Documented recommendation (not enforced):
139+
140+
```python
141+
AsyncClient(middleware=[
142+
Bulkhead(max_concurrent=10), # outermost: cap total concurrency
143+
Retry(...), # retries happen *inside* the Bulkhead slot
144+
])
145+
```
146+
147+
Rationale: with `Bulkhead` outside `Retry`, a single request occupies one slot across all its retry attempts. Concurrency stays bounded by `max_concurrent` even under retry storms. The opposite order — `Retry` outside `Bulkhead` — lets each retry attempt re-acquire a slot, which inflates effective concurrency under the exact load conditions retry is meant to absorb.
148+
149+
If a caller wants per-attempt rejection (each retry can be rejected independently), they put `Retry` outside; that is also valid and documented as a trade-off in the docstring.
150+
151+
### Sharing across clients
152+
153+
A `Bulkhead` instance is the sharable unit. Reuse:
154+
155+
```python
156+
shared = Bulkhead(max_concurrent=20)
157+
client_a = AsyncClient(base_url="https://upstream.example.com/v1", middleware=[shared])
158+
client_b = AsyncClient(base_url="https://upstream.example.com/v2", middleware=[shared])
159+
```
160+
161+
Both clients then contend for the same 20 slots. No special API surface — the semaphore is just a Python attribute on the shared object.
162+
163+
## Errors raised
164+
165+
- **`BulkheadFullError(ClientError)`** — raised when `acquire_timeout` elapses without acquiring a slot. Fields: `max_concurrent`, `acquire_timeout`. Picklable.
166+
- **`asyncio.CancelledError`** — propagates unchanged. Never caught by `Bulkhead`.
167+
- **All other exceptions from `next()`** — propagate unchanged. `Bulkhead` only intercepts the acquisition-timeout path.
168+
169+
## Testing
170+
171+
Per `planning/engineering.md §6`:
172+
173+
- **`tests/test_bulkhead.py`** — unit tests via `httpx2.MockTransport`:
174+
- succeeds when slots available (no wait, no `BulkheadFullError`)
175+
- serializes correctly when at capacity (second call waits for first to release)
176+
- `BulkheadFullError` raised after `acquire_timeout` elapses
177+
- `BulkheadFullError` carries the configured `max_concurrent` / `acquire_timeout`
178+
- `BulkheadFullError` is picklable (round-trip)
179+
- `acquire_timeout=0` fails fast (immediate raise when full)
180+
- `acquire_timeout=None` waits forever (use `asyncio.wait_for` to time out from outside the test)
181+
- slot released after success (next request acquires immediately)
182+
- slot released after exception from `next()` (semaphore not leaked)
183+
- slot released on `asyncio.CancelledError` propagation
184+
- `max_concurrent < 1` rejected at construction with `ValueError`
185+
- negative `acquire_timeout` rejected at construction with `ValueError`
186+
- shared `Bulkhead` instance across two `AsyncClient`s enforces the joint cap
187+
188+
- **`tests/test_bulkhead_props.py`** — Hypothesis property tests:
189+
- **Invariant 1**: For any interleaving of N coroutines through a single `Bulkhead(max_concurrent=K)`, the observed in-flight count never exceeds K.
190+
- **Invariant 2**: With `acquire_timeout=0`, the total successful acquisitions across the run equals K + (number of releases observed before each acquire attempt).
191+
- **Invariant 3**: `BulkheadFullError` is raised iff acquire takes longer than `acquire_timeout`.
192+
193+
- **No time injection.** Unlike `RetryBudget` (where deterministic time matters for TTL math), `Bulkhead`'s only time-sensitive operation is `asyncio.timeout` around the acquire. Tests use real `asyncio.sleep` with sub-100ms values; the suite stays under ~50ms even with the property tests.
194+
195+
- **Coverage target**: 100% line coverage.
196+
197+
## Open questions deferred to implementation
198+
199+
- **Construct-outside-loop, use-inside-loop test.** `asyncio.Semaphore` in Python 3.10+ binds lazily on first use (no construct-inside-loop requirement). The project floor is 3.11, so this is already fine — but a test that constructs `Bulkhead` at module scope and uses it inside an `async def` test pins the behavior and survives any future stdlib regression.
200+
- **Property test seeding.** Hypothesis-driven concurrency tests can flap if the event loop scheduler is non-deterministic. Use `@settings(deadline=None)` and a sufficient `max_examples` to make flakes detectable, not deterministic. Same pattern as `test_budget_props.py`.
201+
202+
## References
203+
204+
- `planning/engineering.md` §6 (testing patterns, property tests for concurrency-sensitive code), §8 (roadmap — `3-5` Bulkhead)
205+
- `planning/specs/2026-06-05-retry-and-retry-budget-design.md` (slice-1 patterns mirrored here: module-per-middleware, `__reduce__` for picklability, public re-export)
206+
- Hystrix / Resilience4j bulkhead patterns (semaphore-based; conceptual reference, no API parity intended)

0 commit comments

Comments
 (0)