[m0hamed541]

Summary

When the scanned target redirects to a different host, emit one extra fingerprint so the platform can show which port redirects and where to.

selector:        v3.fingerprint.domain_name.service.library   (and the ip.v4 / ip.v6 variants)
library_name:    redirect
library_version: HTTP status code, e.g. 301
detail:          absolute destination URL, e.g. http://www.rexel.com/

Context: a customer saw a PHP service on three domains that only answer on port 80 with a 301 to another site. WhatWeb saw the redirect but dropped it, and the technologies of the destination were recorded against the scanned domain with nothing explaining why.

How it works

• Only the first output line is used: it is the scanned target's own response. Later lines are the hops it redirects to.
• The destination comes from the RedirectLocation plugin of that line, resolved against the line's URL with urljoin, so relative Location values work.
• It is emitted only when the status is 3xx and the destination host differs from the source host (a leading www. is ignored). http to https, example.com to www.example.com and path redirects emit nothing.
• RedirectLocation stays in BLACKLISTED_PLUGINS: the generic path would turn its string (the URL) into a library name. The new code reads the plugin before that filter.
• No v3.report.vulnerability is reported for the redirect fingerprint.
• _get_msg_data gets an optional detail_override, used only by the redirect message. The detail is the bare URL because the frontend parses it to show the destination host.

Tests

Two new tests with two fixtures (redirect_output.json, same_host_redirect_output.json): a cross-host redirect emits exactly one redirect message (301, http://www.rexel.com/) next to the normal fingerprints; a same-host redirect emits none.

pytest tests/whatweb_test.py: 35 passed (28 existing + 7 new: redirect, same-host, malformed Location, unexpected plugin shape, query/fragment/credentials stripped, length cap, same-host value before a cross-host one). ruff and mypy clean.

Not verified

• The image has not been built and no real scan has been run with this agent. The tests use WhatWeb output fixtures, not live output. The [url, status, plugins] JSON shape comes from the existing fixtures. The Dockerfile clones upstream WhatWeb at build time, so a format change upstream would silently stop redirects from being emitted.
• Companion change for Wappalyzer: Ostorlab/agent_wappalyzer, branch feature/redirect-fingerprint. That one was built and scanned on a VM.
• Frontend: Ostorlab/reporting_engine_frontend, branch feature/service-redirect-chip.

Review feedback addressed

• A malformed Location (e.g. http://[bad) no longer drops the target's fingerprints: URL parsing is guarded and unexpected plugin values are skipped.
• Only the scheme, host, port and path of the destination are stored and logged. The query string, fragment and userinfo (tokens, credentials) are dropped, and the result is capped at 2048 characters (the backend detail column holds 4096).
• A same-host value no longer stops the search: the remaining values and plugin entries are still checked.

Still open

1. First hop only. A chain like http://a.com to https://a.com to https://b.com emits nothing, because the first hop stays on the same host.
2. Other consumers: agent/ostorlab/whatweb also runs in groups next to osv / nuclei (e.g. ai_network_pentest). I did not check whether they consume v3.fingerprint.*.service.library and would look up a library named redirect.
3. Two agents, one fact: WhatWeb and Wappalyzer may report different status codes (301 vs 302) for the same service; the backend keys fingerprints on (service, name, version), so both rows would be stored.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RRPn4p16aFe9YiRsnD6YgZ

[ostorlab-ai-pr-review]

Do not persist the raw destination URL verbatim into detail. The Location header of a redirect frequently points at a login page and can carry query strings or fragments containing sessions tokens/codes; the value is stored in the backend detail column (limit 4096) and logged at INFO. Keep only the scheme, host, and path (dropping query and fragment), and cap the length, so no credential-bearing material is recorded.

[ostorlab-ai-pr-review]

This is now addressed in the head revision. _get_redirect_target feeds the destination through _clean_redirect_destination, which discards the userinfo/credentials, the query string, and the fragment (parse.urlunsplit(..., "", "")), and caps the result to REDIRECT_DESTINATION_MAX_LENGTH (2048, below the backend's 4096). The new tests testWhatWebAgent_whenDestinationHasQueryFragmentAndCredentials_keepsOnlySchemeHostPortAndPath and testWhatWebAgent_whenDestinationIsVeryLong_capsTheDetailLength cover both behaviors. Resolving this thread.

[ostorlab-ai-pr-review]

No new issues found.