Repository navigation
feat(esi): streaming processor with parallel includes, bump Go to 1.27 - #16
Merged
Merged
Conversation
- Add esi.Stream, an io.WriteCloser that processes ESI tags incrementally: static content is forwarded immediately, include tags are fetched concurrently and written in document order, and the destination is flushed whenever the output waits for an include or more input. - Parse now fetches independent includes in parallel and assembles the output in a single pre-sized allocation. - Share one incremental scanner between Parse and Stream; replace the tag/closing regexes with byte searches. - Includes reuse a pooled transport, honour the request context and no longer leak the failed response body on alt fallback. - Rewrite writer.Writer as a thin http.ResponseWriter over esi.Stream (passes status codes through, drops Content-Length, no busy loop); simplify the caddy, roadrunner and traefik middlewares accordingly. - Bump Go to 1.27 across modules and CI, upgrade caddy to v2.11.9, migrate golangci-lint config to v2, re-vendor the traefik plugin. - Tests start their own fixture server and cover chunk boundaries, parallelism, early flushing and write errors. BREAKING CHANGE: writer.NewWriter(rw, rq) no longer takes a buffer, the AsyncBuf/Done/Iteration fields are removed and Close must be called. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- actions/checkout v7, actions/setup-go v7 (Node 24 runtime)
- golangci-lint-action v9 with golangci-lint v2.15: the v2.13.2 binary
cannot read the Go 1.27.3 export data ("export data version 5 is
greater than maximum supported version 4")
- goreleaser-action v7 pinned to GoReleaser "~> v2"; --rm-dist was
removed in v2, use --clean
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Yaegi rejected the plugin with "stream.go:128:1: cannot define new
methods on non-local type struct {}", then "undefined: clear" and
"undefined: min".
- Guard the stream error with a mutex instead of atomic.Pointer[error]:
Yaegi cannot resolve generic standard library types.
- Replace the clear and min builtins with explicit code.
- Refresh the stale traefik vendor copy.
Validated with traefik:latest through middleware/traefik: the plugin
loads and processes vars, remove, include and alt fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
darkweak
force-pushed
the
feat/stream-parallel-esi
branch
3 times, most recently
from
November 9, 2026 18:16
c138265 to
86ec746
Compare
darkweak
force-pushed
the
feat/stream-parallel-esi
branch
from
November 10, 2026 09:50
86ec746 to
0af7fb5
Compare
|
Dang it, I did local ESI reimplemention 2 weeks ago and forgot to PR it xD Should I still push it somewhere for comparison in case there's anything useful there to salvage? |
Owner
Author
|
I think you can push a PR 😄 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
esi.NewStream(dst, req): anio.WriteCloserthat processes ESI tags while the document is written. Static content goes out immediately, includes are fetched concurrently and written in document order, and the destination is flushed whenever output waits on an include or more input. Tags split across writes are handled;Writenever blocks on an include.esi.Parse: same signature, now fetches includes in parallel and builds the output in one pre-sized allocation instead of re-appending the whole buffer after each tag.writer.Writer: thinhttp.ResponseWriterover the stream. Middlewares becomeNewWriter→next.ServeHTTP→Close, removing the busy-wait goroutine (and its data race) from caddy, roadrunner and traefik.altfallback.Breaking changes / behaviour differences
writer.NewWriter(rw, rq)no longer takes a*bytes.Buffer;AsyncBuf,Done,Iterationremoved;Close()must be called.Content-Lengthis dropped properly; the invalidContent-Encoding: chunkedheader is no longer set.<esi:try>) are emitted verbatim instead of crashingParse.onerror="continue"is removed instead of truncating the rest of the document.Parseno longer mutates its input.Test plan
go test -race ./...in the root module (tests now start their own fixture server on:9080, or reuse the CI caddy)Parse, parallel includes (20 × 200 ms in ~210 ms), flush before slow include, write errors,writerstatus/Content-Length handling-racegolangci-lint run(v2.13): 0 issues on the root moduletraefik:latest,middleware/traefikstack) and processes vars, remove, include and alt fallback. Yaegi does not support generic stdlib types nor theclear/minbuiltins, so the core package avoids them.