Skip to content

Keep the response headers of prerendered pages, and write them to a headers file - #6

Open
jnadaud wants to merge 1 commit into
bakslashHQ:mainfrom
jnadaud:keep-response-headers
Open

jnadaud wants to merge 1 commit into
bakslashHQ:mainfrom
jnadaud:keep-response-headers

Conversation

@jnadaud

@jnadaud jnadaud commented Oct 21, 2026

Copy link
Copy Markdown

Closes #2.

Built along the lines of your answer on the issue.

What changes

  • RenderedPage gains array $headers = [], holding every header of the response the page was rendered from, keyed by lowercase name as HeaderBag::all() gives them. Nothing is filtered there, so listeners can read X-Robots-Tag, Link or anything else. Redirect stubs carry none.
  • RenderedPage::withContent() returns a copy with other content, keeping the format and the headers. MinifyListener uses it, so minification no longer drops them, and so does the README's RewriteLinks example.
  • headers_file (default null) registers a listener that collects the headers on PageRenderedEvent and writes the file on GenerationFinishedEvent, from $event->uris. Only pages actually written get a rule. The listener runs at -512, after the minifier and user listeners.
    • Filtering on write: a fixed list of headers that describe the page is written: CSP (and report-only), HSTS, X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy, the three Cross-Origin-*, X-Robots-Tag and Link. Set-Cookie, Cache-Control, Vary, Date, ETag and the like never are. The list isn't configurable, as you suggested.
    • Rule limit: the headers every page sends with the same value go into one /* rule, and a page only gets a rule of its own for what differs. A header whose value differs between pages is never put in /*, so no ! detaching or value joining is involved.
    • Partial runs: the rules live between two comment markers. A --route/--path run reads the pages already listed there, updates its own, and lays everything out again, so /* stays right for the whole site. A page covered by /* alone is listed as a comment, so it isn't forgotten. Everything outside the markers (hand-written rules) is kept verbatim.
    • Paths are the ones the host serves: base path included, <path>/ under clean_urls, and <path>.<format> for other formats. A value containing a line break is dropped, so it cannot forge another rule. A headers_file escaping the output (..) is rejected.
  • Docs: the headers in the Caddy and nginx examples; a "Security headers" section covering the file, the 100-rule layout, the GitHub Pages limits, and the two points below; the option in the configuration reference; CHANGELOG under "Unreleased".
    • Nonce-based CSP: it is frozen at build time, so use hashes.
    • Debug kernels: FrameworkBundle adds X-Robots-Tag: noindex when kernel.debug is on, so generating with --allow-debug writes it. The functional test caught this one.

Backward compatibility

Additive: existing new RenderedPage($content, $format) calls keep working, and nothing is written unless headers_file is set.

Tests

19 new tests:

  • renderer: all headers kept, redirects carry none;
  • minify listener: headers kept;
  • headers file:
    • shared /* rule and per-path differences;
    • differing values stay per path;
    • filtering;
    • served paths, including clean URLs and base path;
    • partial run with hand-written rules kept;
    • a /*-only page surviving a partial run;
    • injected line breaks;
    • nothing written when nothing was;
  • an end-to-end run with a real kernel, plus the configuration check.

composer test -- --fail-on-deprecation, composer stan and composer cs pass on PHP 8.5 with the highest dependencies, and the suite passes on PHP 8.3 with the lowest (Symfony 7.4, PHPUnit 12).

One choice to check with you: Link and X-Robots-Tag are in the written list, since they describe the page too. Happy to drop either if you'd rather leave them to a listener.

…eaders file

RenderedPage keeps every header of the response it was rendered from, and
withContent() carries them through listeners (minification included). With
headers_file set, the headers that describe a page (CSP, HSTS, X-Robots-Tag,
Link, ...) are written in the _headers format of Cloudflare Pages and Netlify:
what all pages share in one /* rule, per-path rules only for what differs, and
a filtered run merges with the pages already listed.

Closes bakslashHQ#2

@mtarld mtarld left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make this simpler, mostly by dropping the merge on filtered runs I asked for in #2 (see inline). Plus, the /* rule turns out to be fragile on a real site: a redirect or a removed route is enough to lose it, and to go past the 100 rules on a large site.
Link and X-Robots-Tag in the written list are fine to me.

Comment thread CHANGELOG.md
Comment on lines +5 to +10
- `RenderedPage::$headers` keeps every header of the response a page was rendered from;
`RenderedPage::withContent()` returns a copy with other content, keeping the format and
the headers, and the minification listener uses it (#2).
- `headers_file` writes the headers that describe each page (`Content-Security-Policy`,
`Strict-Transport-Security`, ...) in the `_headers` format of Cloudflare Pages and Netlify:
shared headers in one `/*` rule, merged with the existing file on filtered runs (#2).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `RenderedPage::$headers` keeps every header of the response a page was rendered from;
`RenderedPage::withContent()` returns a copy with other content, keeping the format and
the headers, and the minification listener uses it (#2).
- `headers_file` writes the headers that describe each page (`Content-Security-Policy`,
`Strict-Transport-Security`, ...) in the `_headers` format of Cloudflare Pages and Netlify:
shared headers in one `/*` rule, merged with the existing file on filtered runs (#2).
- Keep the response headers of a rendered page in `RenderedPage::$headers`, and add
`RenderedPage::withContent()` (#2).
- Add a `headers_file` option, writing the page headers in the `_headers` format of
Cloudflare Pages and Netlify (#2).

Comment on lines +20 to +23
/**
* A copy carrying other content, keeping the format and the headers: what a listener
* transforming the markup should return.
*/

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed.

Comment thread README.md
Git integration's output directory at it. `_redirects` and `_headers` files placed in the
output are honoured.

### Security headers

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this section can be way shorter: the config, the file example and the nonce note are enough IMO. The GitHub Pages paragraph is off topic, and the X-Robots-Tag note is expected when using --allow-debug.

Comment on lines +50 to +57
->scalarNode('headers_file')
->info('Write the headers that describe each page (Content-Security-Policy, Strict-Transport-Security, ...) to this file of the output, in the "_headers" format of Cloudflare Pages and Netlify (e.g. "_headers"). Disabled when null.')
->defaultNull()
->validate()
->ifTrue(static fn (mixed $value): bool => \is_string($value) && ('' === trim($value, '/') || str_contains($value, '..')))
->thenInvalid('The headers file must be a path inside the output, got %s.')
->end()
->end()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
->scalarNode('headers_file')
->info('Write the headers that describe each page (Content-Security-Policy, Strict-Transport-Security, ...) to this file of the output, in the "_headers" format of Cloudflare Pages and Netlify (e.g. "_headers"). Disabled when null.')
->defaultNull()
->validate()
->ifTrue(static fn (mixed $value): bool => \is_string($value) && ('' === trim($value, '/') || str_contains($value, '..')))
->thenInvalid('The headers file must be a path inside the output, got %s.')
->end()
->end()
->scalarNode('headers_file')
->info('Write the headers that describe each page to this file of the output, in the "_headers" format of Cloudflare Pages and Netlify (e.g. "_headers"). Disabled when null.')
->defaultNull()
->end()

Flysystem's WhitespacePathNormalizer already throws PathTraversalDetected on .. and strips the leading / (same for the ltrim() below), so I think we can drop this validation and testRejectsAHeadersFileOutsideTheOutput. Plus, str_contains($value, '..') rejects foo..bar.

ltrim($file, '/'),
$cleanUrls,
])
->addTag('kernel.event_listener', ['event' => PageRenderedEvent::class, 'method' => 'onPageRendered', 'priority' => -512])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes of a user listener registered below -512 are missed. What about decorating bakslash_static_site.dumper instead? dump() gets the final page, and the decorator only has to record it when the inner dumper doesn't throw, so no need to filter on $event->uris anymore.

/**
* The URL the host serves the written file at: what its rules match against.
*/
private function servedPath(string $uri, string $format): string

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This duplicates the match of FlysystemDumper::dump(), so both will diverge at the first change. What about computing the written path in a single place, and deriving the served URL from it? It would fix /page.html as well, as Cloudflare Pages serves it at /page.

default => $uri.'.'.$format,
};

return $this->siteContext->basePath().$path;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the base path. Pages are written at the root of the output, so a host serving it (behind a proxy stripping /docs or not) receives /blog, never /docs/blog. And if the output is a subdirectory of a larger deployment, the _headers file is ignored, as it must be at the root.
Am I missing a case where the prefix matches?

?? 'html';

return new RenderedPage($content, $format);
return new RenderedPage($content, $format, $this->headers($response));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return new RenderedPage($content, $format, $this->headers($response));
return new RenderedPage($content, $format, $response->headers->all());

null values end up as '' in written(), which already skips them, so headers() can go. The @param of RenderedPage::$headers then has to be list<string|null>, as HeaderBag::all() returns.


public function testWritesTheHeadersOfThePagesToTheHeadersFile(): void
{
$kernel = $this->kernel = new HeadersFileKernel('test', true);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about booting the kernel with debug: false? The test would then check what a real build writes, without the X-Robots-Tag coming from the debug mode.

*
* @return list<string>
*/
private static function layout(array $pages): array

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK Cloudflare silently ignores the rules past the 100th, and nothing tells it here (150 pages with a different Link give 151 rules). Should we at least log a warning, or even throw?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Response headers (CSP, HSTS, …) are dropped when a page is prerendered

2 participants