Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,10 +246,11 @@ field out.
command for the first group. An error whose commands all fit names the groups
only in its commands, as before: after a handshake_failure alert on a
five-digit port, it is 426 bytes for `127.0.0.1` and 418 bytes for `::1`, and
495 bytes for `127.0.0.1` with the setting above. A test builds the error for
every host name length the scanner accepts, at ports 443 and 65535, and for
the longest name after each of the 256 TLS alert values, and finds each group
named within the 500 bytes.
495 bytes for `127.0.0.1` with `GODEBUG=tlsmlkem=0`, where the scanner offers
none of the three hybrid groups. A test builds the error for every host name
length the scanner accepts, at ports 443 and 65535, and for the longest name
after each of the 256 TLS alert values, and finds each group named within the
500 bytes.
- **A port that is not a number from 1 to 65535 is refused without quoting it.**
The error used to repeat the text after the colon, which on a `key: value` line
is the value.
Expand Down Expand Up @@ -374,6 +375,28 @@ field out.
`./tlsanalyzer version`. Building from source comes second and runs the
binary where it was built, without `sudo`. `docs/install.md` names the
archive for each platform.
- **Run with `GODEBUG=fips140=on`, a scan lists `X25519` as a key exchange
group not probed**, instead of reporting a server that offers it as not
supporting it. FIPS 140-3 mode leaves `X25519` out of the ClientHello, and
the scan read only the three hybrid groups to learn which groups it could
offer, so its probe of `X25519` could only fail. Measured with Go 1.26.3
against a local server offering `X25519MLKEM768` and `X25519`, the scan found
only `X25519MLKEM768`, and its scan coverage note named `SecP256r1MLKEM768`
and `SecP384r1MLKEM1024`; the note now names `X25519` after them, and the
grade is unchanged. Against a local server offering only `X25519`, the
handshake failure gave a command for each of those two groups, and now gives
a third, for example
`openssl s_client -connect 127.0.0.1:60464 -groups X25519`. With
`GODEBUG=fips140=on,tlsmlkem=0`, the note and the error name `X25519` after
the three hybrid groups. Four groups leave less room for the commands, so the
words before them are now `Check each group on its own, for example:`, 10
bytes shorter than `Check each group with its own command, for example:`.
With the longer words and four groups, the error for a 47-byte IPv6 address
and port, such as `[ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff]:65535`, gave no
whole command after six of the 256 TLS alert values or after an unexpected
EOF; it now gives one after each alert value and after the server closes the
connection. By default and with `GODEBUG=tlsmlkem=0` alone, the groups listed
and named are unchanged.

## [0.4.1] - 2026-08-05

Expand Down
19 changes: 10 additions & 9 deletions internal/scanner/client_hello_written_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,17 @@ import (
// will not offer, and when none is left it returns "tls: no supported elliptic
// curves for ECDHE" before writing a ClientHello. clientHelloGroups then waited
// out its 5-second deadline for a ClientHello that was never coming, and
// hybridGroupsNotOffered returned nil. With GODEBUG=fips140=on,tlsmlkem=0 on Go
// 1.26.3 a scan took 5 seconds longer, a failed handshake was reported without
// the groups this build cannot offer, and a completed scan carried no note that
// the hybrid groups were not probed.
// hybridGroupsNotOffered, now groupsNotOffered, returned nil. With
// GODEBUG=fips140=on,tlsmlkem=0 on Go 1.26.3 a scan took 5 seconds longer, a
// failed handshake was reported without the groups this build cannot offer, and
// a completed scan carried no note that the hybrid groups were not probed.
//
// The mutation each test is written to catch:
// - clientHelloGroups waits for the server's read to time out after the
// client has returned:
// TestClientHelloGroupsReturnsAsSoonAsTheClientGivesUp.
// - clientHelloGroups reports a client that wrote nothing as a ClientHello it
// could not read, so hybridGroupsNotOffered names no group:
// could not read, so groupsNotOffered names no group:
// TestClientHelloGroupsReturnsAsSoonAsTheClientGivesUp and
// TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer.
//
Expand Down Expand Up @@ -64,8 +64,9 @@ func TestClientHelloGroupsReturnsAsSoonAsTheClientGivesUp(t *testing.T) {
}
}

// fipsChildEnv marks the child process that
// TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer starts.
// fipsChildEnv marks the child processes that
// TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer and
// TestAFIPSBuildListsX25519AsNotProbed start.
const fipsChildEnv = "TLS_ANALYZER_TEST_FIPS_CHILD"

// TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer runs the tests
Expand All @@ -80,7 +81,7 @@ func TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer(t *testing.T) {
}

cmd := exec.Command(os.Args[0], "-test.count=1", "-test.v",
"-test.run=^(TestHybridGroupsNotOfferedMatchesAHandshake|TestFIPSChildScanNamesTheHybridGroupsItCannotOffer)$")
"-test.run=^(TestGroupsNotOfferedMatchesAHandshake|TestFIPSChildScanNamesTheHybridGroupsItCannotOffer)$")
cmd.Env = append(os.Environ(), "GODEBUG=fips140=on,tlsmlkem=0", fipsChildEnv+"=1")
out, err := cmd.CombinedOutput()
if strings.Contains(string(out), "--- SKIP: TestFIPSChildScanNamesTheHybridGroupsItCannotOffer") {
Expand All @@ -90,7 +91,7 @@ func TestAFIPSBuildWithoutMLKEMNamesTheHybridGroupsItCannotOffer(t *testing.T) {
t.Fatalf("the tests in the FIPS child process failed (%v):\n%s", err, out)
}
if !strings.Contains(string(out), "--- PASS: TestFIPSChildScanNamesTheHybridGroupsItCannotOffer") ||
!strings.Contains(string(out), "--- PASS: TestHybridGroupsNotOfferedMatchesAHandshake") {
!strings.Contains(string(out), "--- PASS: TestGroupsNotOfferedMatchesAHandshake") {
t.Fatalf("the FIPS child process did not run both tests:\n%s", out)
}
}
Expand Down
166 changes: 166 additions & 0 deletions internal/scanner/fips_x25519_not_probed_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
package scanner

import (
"context"
"crypto/fips140"
"crypto/tls"
"net"
"os"
"os/exec"
"slices"
"strconv"
"strings"
"testing"
"time"

"github.com/csnp/qramm-tls-analyzer/internal/sanitize"
)

// Run with GODEBUG=fips140=on, crypto/tls leaves X25519 out of every
// ClientHello, because FIPS 140-3 mode does not allow it. The scan decided which
// groups it could probe by reading a ClientHello for the three hybrid groups
// only, so it probed X25519 anyway and every probe failed. A server offering
// X25519MLKEM768 and X25519 was reported with X25519MLKEM768 alone, and the
// coverage note named SecP256r1MLKEM768 and SecP384r1MLKEM1024 but not X25519.
// For a server offering only X25519, the handshake failure named those two
// groups as the ones this scanner does not offer, and not X25519.
//
// The mutation these tests are written to catch: groupsNotOffered reads only
// the three hybrid groups, as before this change. On that code
// TestFIPSChildListsX25519AsNotProbed was observed to fail, and in the same
// child process TestGroupsNotOfferedMatchesAHandshake and
// TestAGroupThisBuildCannotNegotiateIsReportedNotProbed, each over X25519.

// fipsServersEnv carries the ports of the servers that
// TestAFIPSBuildListsX25519AsNotProbed starts to its child process, separated
// by a comma.
const fipsServersEnv = "TLS_ANALYZER_TEST_FIPS_SERVERS"

// TestAFIPSBuildListsX25519AsNotProbed is the reproduction of "scanner reports
// X25519 absent under GODEBUG=fips140=on instead of listing it as not probed".
// This process, which is not in FIPS 140-3 mode, serves the report's two
// servers: one offering X25519MLKEM768 and X25519, and one offering only
// X25519. A child process started with GODEBUG=fips140=on scans them, since
// FIPS mode is fixed when a process starts. The child also runs the tests that
// compare the groups listed as not offered with real handshakes, which in FIPS
// mode cover X25519.
func TestAFIPSBuildListsX25519AsNotProbed(t *testing.T) {
if os.Getenv(fipsChildEnv) != "" {
t.Skip("this is the child process")
}
if fips140.Enabled() {
t.Skip("this process runs in FIPS 140-3 mode, so its servers cannot offer X25519")
}

notBefore, notAfter := validFrom()
cert := issueLeaf(t, []string{"localhost"}, nil, notBefore, notAfter)
var ports []string
for _, groups := range [][]tls.CurveID{{groupX25519MLKEM768, tls.X25519}, {tls.X25519}} {
_, port := startTLSServerWithConfig(t, &tls.Config{
Certificates: []tls.Certificate{cert.serving},
MinVersion: tls.VersionTLS13,
CurvePreferences: groups,
})
ports = append(ports, strconv.Itoa(port))
}

childTests := []string{
"TestGroupsNotOfferedMatchesAHandshake",
"TestAGroupThisBuildCannotNegotiateIsReportedNotProbed",
"TestFIPSChildListsX25519AsNotProbed",
}
cmd := exec.Command(os.Args[0], "-test.count=1", "-test.v",
"-test.run=^("+strings.Join(childTests, "|")+")$")
cmd.Env = append(os.Environ(), "GODEBUG=fips140=on", fipsChildEnv+"=1",
fipsServersEnv+"="+strings.Join(ports, ","))
out, err := cmd.CombinedOutput()
if strings.Contains(string(out), "--- SKIP: TestFIPSChildListsX25519AsNotProbed") {
t.Skipf("the FIPS child process skipped the scans:\n%s", out)
}
if err != nil {
t.Fatalf("the tests in the FIPS child process failed (%v):\n%s", err, out)
}
for _, name := range childTests {
if !strings.Contains(string(out), "--- PASS: "+name+" ") {
t.Fatalf("the FIPS child process did not pass %s:\n%s", name, out)
}
}
}

// TestFIPSChildListsX25519AsNotProbed runs only in the child process that
// TestAFIPSBuildListsX25519AsNotProbed starts, and scans the servers that test
// serves. The scan of the server offering X25519MLKEM768 and X25519 must find
// X25519MLKEM768 and list X25519 as not probed, and the error for the server
// offering only X25519 must give the openssl command that checks X25519.
func TestFIPSChildListsX25519AsNotProbed(t *testing.T) {
ports := strings.Split(os.Getenv(fipsServersEnv), ",")
if os.Getenv(fipsChildEnv) == "" || len(ports) != 2 {
t.Skip("runs in the child process that TestAFIPSBuildListsX25519AsNotProbed starts")
}
if !fips140.Enabled() {
t.Skip("GODEBUG=fips140=on did not turn on FIPS 140-3 mode in this build")
}

notBefore, notAfter := validFrom()
cert := issueLeaf(t, []string{"localhost"}, nil, notBefore, notAfter)
x25519, hybrid := tlsGroups[tls.X25519].name, tlsGroups[groupX25519MLKEM768].name
if negotiatesOnly(t, cert.serving, tls.X25519) {
t.Skipf("this build negotiates %s in FIPS 140-3 mode, so the scan can probe it", x25519)
}
if !negotiatesOnly(t, cert.serving, groupX25519MLKEM768) {
t.Skipf("this build cannot negotiate %s in FIPS 140-3 mode, so it shares no group "+
"with the first server", hybrid)
}

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
result, err := New(localScanConfig()).Scan(ctx, net.JoinHostPort("127.0.0.1", ports[0]))
if err != nil {
t.Fatalf("scan of the server offering %s and %s: %v", hybrid, x25519, err)
}
var found []string
for _, ke := range result.KeyExchanges {
found = append(found, ke.Name)
}
if !slices.Contains(found, hybrid) || slices.Contains(found, x25519) {
t.Errorf("the scan found %v; it should find %s, and not %s, which this build cannot offer",
found, hybrid, x25519)
}
if notProbed := notProbedGroups(result.ScanWarnings); !slices.Contains(notProbed, x25519) {
t.Errorf("the scan does not list %s as not probed, so the server's %s reads as absent; "+
"groups listed as not probed: %q", x25519, x25519, notProbed)
}

port, err := strconv.Atoi(ports[1])
if err != nil {
t.Fatalf("the port of the server offering only %s: %v", x25519, err)
}
addr := net.JoinHostPort("127.0.0.1", ports[1])
printed := sanitize.WrapError(scanExpectingError(t, "127.0.0.1", port)).Error()
t.Logf("error for the server offering only %s, as the CLI prints it, %d bytes: %s",
x25519, len(printed), printed)
names := make([]string, len(keyExchangeCandidates))
for i, id := range keyExchangeCandidates {
names[i] = tlsGroups[id].name
}
whole, cut := opensslCommands(printed, sanitize.ShellWord(addr), names)
if cut != "" {
t.Errorf("the CLI's cut ends the error inside an openssl command: %q", cut)
}
if !slices.Contains(whole, x25519) {
t.Errorf("the error gives no command that checks %s, the server's only group and one "+
"this build does not offer; it checks %v: %q", x25519, whole, printed)
}
}

// notProbedGroups returns the groups a scan's coverage note lists as key
// exchange groups not probed, or nil when it has no such note.
func notProbedGroups(warnings []string) []string {
const note = "key exchange groups not probed because this build of the scanner cannot offer them: "
for _, w := range warnings {
if groups, ok := strings.CutPrefix(w, note); ok {
return strings.Split(groups, ", ")
}
}
return nil
}
17 changes: 9 additions & 8 deletions internal/scanner/group_probe_coverage_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,32 +18,33 @@ import (
// quantum score 0, with no note that the group was never offered. A Go 1.25
// build skips both and says they were not probed.

// TestAGroupThisBuildCannotNegotiateIsReportedNotProbed serves every hybrid
// group and checks each against what crypto/tls does in this build: a group
// this build negotiates must be found, and a group it cannot negotiate must be
// listed as not probed rather than reported absent.
// TestAGroupThisBuildCannotNegotiateIsReportedNotProbed serves every group the
// scan probes and checks each against what crypto/tls does in this build: a
// group this build negotiates must be found, and a group it cannot negotiate
// must be listed as not probed rather than reported absent.
//
// On a Go 1.26 or later toolchain this exercises the P-curve hybrids, which the
// earlier code probed and reported absent. On Go 1.25 the toolchain does not
// name them, so both versions of the code skip them and the test passes either
// way.
// way. In the FIPS 140-3 child process that TestAFIPSBuildListsX25519AsNotProbed
// starts, it exercises X25519, which that mode leaves out of the ClientHello and
// which the scan probed and reported absent when it read only the hybrid groups.
func TestAGroupThisBuildCannotNegotiateIsReportedNotProbed(t *testing.T) {
notBefore, notAfter := validFrom()
cert := issueLeaf(t, []string{"localhost"}, nil, notBefore, notAfter)
hybrids := []tls.CurveID{groupX25519MLKEM768, groupSecP256r1MLKEM768, groupSecP384r1MLKEM1024}

host, port := startTLSServerWithConfig(t, &tls.Config{
Certificates: []tls.Certificate{cert.serving},
MinVersion: tls.VersionTLS13,
CurvePreferences: append(slices.Clone(hybrids), tls.X25519),
CurvePreferences: slices.Clone(keyExchangeCandidates),
})

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
supported, skipped := New(localScanConfig()).probeKeyExchangeGroups(ctx, host, port,
&reachedAddresses{})

for _, id := range hybrids {
for _, id := range keyExchangeCandidates {
name := tlsGroups[id].name
found := slices.ContainsFunc(supported, func(ke types.KeyExchange) bool { return ke.Name == name })
notProbed := slices.Contains(skipped, name)
Expand Down
10 changes: 5 additions & 5 deletions internal/scanner/handshake_failure_changelog_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ import (
// figures the 0.5.0 entries of CHANGELOG.md give for the error of a scan that
// completed no TLS handshake to the error handshakeFailureMessage builds after
// a handshake_failure alert: its size for 127.0.0.1 and ::1 on a five-digit
// port, with and without GODEBUG=fips140=on,tlsmlkem=0, and the longest
// address for which both commands of a default build fit within the CLI's cut.
// port, with and without GODEBUG=tlsmlkem=0, and the longest address for which
// both commands of a default build fit within the CLI's cut.
//
// Those entries gave figures measured before the error gained its sentence
// about cipher suites, which made each of those errors 28 bytes longer, and
Expand Down Expand Up @@ -52,7 +52,7 @@ func TestTheChangelogGivesTheHandshakeFailureSizesThisTreeBuilds(t *testing.T) {
x25519, p256, p384 := tlsGroups[groupX25519MLKEM768].name,
tlsGroups[groupSecP256r1MLKEM768].name, tlsGroups[groupSecP384r1MLKEM1024].name
two := []string{p256, p384} // the groups a default build does not offer
three := []string{x25519, p256, p384} // GODEBUG=fips140=on,tlsmlkem=0
three := []string{x25519, p256, p384} // GODEBUG=tlsmlkem=0

// size is the length of the error as the CLI prints it, which the entries
// give for errors that hold every command and name each group only there.
Expand Down Expand Up @@ -94,8 +94,8 @@ func TestTheChangelogGivesTheHandshakeFailureSizesThisTreeBuilds(t *testing.T) {
"IPv6 address and port of up to %d characters",
[]any{size(v6, two), longestWithEveryCommand(t, 23, 63, ipv6TargetOfLength, alert, two)}},
{"after a handshake_failure alert on a five-digit port, it is %d bytes for " +
"`127.0.0.1` and %d bytes for `::1`, and %d bytes for `127.0.0.1` with the " +
"setting above",
"`127.0.0.1` and %d bytes for `::1`, and %d bytes for `127.0.0.1` with " +
"`GODEBUG=tlsmlkem=0`",
[]any{size(v4, two), size(v6, two), size(v4, three)}},
}
for _, c := range claims {
Expand Down
Loading
Loading