Skip to content

Align the Gson extension with Jackson's duplicate member name rejection - #1073

Open
masiljangajji wants to merge 4 commits into
jwtk:mainfrom
masiljangajji:issue/gson-duplicate-keys
Open

masiljangajji wants to merge 4 commits into
jwtk:mainfrom
masiljangajji:issue/gson-duplicate-keys

Conversation

@masiljangajji

@masiljangajji masiljangajji commented Aug 8, 2026 •

Copy link
Copy Markdown

Summary

The Gson extension silently accepts JSON objects with duplicate member names and keeps the last occurrence. A signed token whose header or payload repeats a name is therefore parsed without complaint:

payload: {"sub":"alice","sub":"attacker","role":"user","role":"admin"}

jjwt-gson     0.12.6   ->  sub=attacker, role=admin   (accepted)
jjwt-jackson  0.12.6   ->  JsonParseException         (rejected)

This makes the two JSON extensions disagree about whether the same signed token is valid.

Why reject

RFC 7515, Section 4 and RFC 7519, Section 4 require member names to be unique and let recipients either reject the input or use only the lexically last value. Both are conformant, so this is a policy choice rather than a bug fix.

JJWT already made that choice for Jackson in #877, which concluded:

A more targeted solution for your particular use case would be to just default to enabling STRICT_DUPLICATE_DETECTION. I agree this would likely be a better default, and, since the RFC allows it, application developers can override/unset that feature if they desire.

Nothing in that reasoning is specific to Jackson. The Gson extension wraps the same kind of input for the same purpose, and today the two extensions disagree about whether a given signed token is acceptable. This change makes the Gson extension follow the decision already taken in #877.

Implementation

Gson has no equivalent switch, and GsonBuilder refuses to register an adapter for Object.class, which is the only type GsonDeserializer ever requests:

IllegalArgumentException: Cannot override built-in adapter for class java.lang.Object

Detection is therefore done at the token reader layer, which is where Jackson's STRICT_DUPLICATE_DETECTION also operates (JsonParser.Feature). doDeserialize wraps the supplied Reader in a JsonReader subclass that keeps one name set per object depth:

  • beginObject() pushes a set, endObject() pops it
  • nextName() throws JsonParseException when Set#add reports the name was already present

Gson still performs all value parsing, so number handling, escaping and every other behavior is unchanged. No extra pass and no buffering are introduced.

One subtlety: Gson asserts that nothing follows the parsed value only when it creates the JsonReader itself — Gson#fromJson(JsonReader, Type) skips that check. Supplying our own reader would therefore have started accepting trailing content, so doDeserialize performs the same assertion after parsing. Two tests cover it.

Scope

Rejection applies to the Gson instance JJWT creates for itself. An instance supplied through GsonDeserializer(Gson) keeps Gson's behavior, which is the same approach the Jackson extension already takes: it configures only the ObjectMapper it creates and leaves an application-supplied one untouched.

Tests

Eight tests added to GsonDeserializerTest — three that must reject duplicates, three that must not, and two that pin the trailing-content behavior:

input expected
{"sub":"alice","sub":"attacker"} rejected
{"user":{"id":1,"id":2}} rejected
{"list":[{"k":"a","k":"b"}]} rejected
{"x":{"k":1},"y":{"k":2}} accepted, names are unique within each object
{"l":[{"k":1},{"k":2}]} accepted
duplicate name with a caller-supplied Gson accepted, last value wins
{"sub":"alice"}trailing rejected, as before this change
{"a":1} {"b":2} rejected, as before this change

The caller-supplied case pins the scope decision above so it cannot be widened by accident.

Compatibility

No public API changes — the new flag flows only through private constructors, and javap reports an identical public member list before and after. ./mvnw -pl extensions/gson -am clean verify passes, and the impl suite (1,511 tests, which loads this extension through ServiceLoader) passes unchanged.

RFC 7515 Section 4 and RFC 7519 Section 4 require JOSE header and JWT
claims member names to be unique, and permit parsers to either reject
such input or use only the lexically last value. Gson silently does the
latter, so a token whose header or payload repeats a member name is
accepted with the last occurrence winning.

The Jackson extension already rejects this input: the ObjectMapper that
JacksonSerializer creates enables STRICT_DUPLICATE_DETECTION. This
applies the same behavior to the Gson extension's default Gson instance
by wrapping the reader in a JsonReader that tracks member names per
object depth and throws when one repeats.

Gson forbids overriding its built-in Object type adapter, so detection
is implemented at the token reader layer, which is the same layer where
Jackson's STRICT_DUPLICATE_DETECTION operates.

A caller-supplied Gson instance retains Gson's default behavior,
mirroring how the Jackson extension only configures the ObjectMapper it
creates itself.

No public API changes.
Gson only asserts that nothing follows the parsed value when it creates
the JsonReader itself; Gson#fromJson(JsonReader, Type) skips that check.
Wrapping the reader therefore made the deserializer accept trailing
content that it previously rejected.

Perform the same assertion after parsing, and cover it with tests.
@masiljangajji masiljangajji changed the title Reject duplicate JSON member names in the default Gson deserializer Align the Gson extension with Jackson's duplicate member name rejection Aug 8, 2026
@masiljangajji

Copy link
Copy Markdown
Author

I measured the overhead this adds, since the change allocates one name set per JSON object on a hot path. Harness, raw JMH output and environment records: https://github.com/masiljangajji/jjwt-duplicate-name-bench

Both arms are this branch's GsonDeserializer built from a single JAR — new GsonDeserializer<>(gson) keeps Gson's default behaviour and new GsonDeserializer<>() rejects duplicates, so the two differ by one boolean rather than by library version. Parsers are built once and reused, and the header and claims documents are decoded out of the same token the end-to-end benchmark parses.

End to end, parseSignedClaims on an HS384 JWS with a 27 B header and a 323 B claims set:

before after delta
time 7,562.7 ns 7,698.6 ns +135.9 ns (+1.8%)
allocation 48,736 B 49,872 B +1,136 B (+2.3%)

Temurin 17.0.14, 8 warmup × 8 measurement iterations × 6 forks, confidence intervals separated (7,633 against 7,657). Corretto 21.0.5 was run as well; its JSON-level deltas sum to +152 ns, which agrees.

The allocation figures are identical to the byte across both JDKs and all twelve forks. They scale with the number of JSON objects in a document rather than its length — +328 B for the header, +824 B for the claims set.

I also measured a DupDetector-style variant that holds the first two names in fields and allocates a set only from the third name on. It runs 241 ns slower on the claims document and shows no benefit on the header, so the straightforward set is what this PR keeps.

@lhazlewood

Copy link
Copy Markdown
Contributor

@masiljangajji great work, thank you so much for the contribution!

@lhazlewood lhazlewood added this to the 0.14.0 milestone Aug 24, 2026
@lhazlewood

Copy link
Copy Markdown
Contributor

@masiljangajji looks like some minor code coverage needs to be addressed with tests

A strict JsonReader, the only kind this deserializer creates, throws
MalformedJsonException for anything following the top-level value, so peek()
never returns a token other than END_DOCUMENT and the JsonSyntaxException that
branch guarded could not be reached. Dropping it leaves the reader's own error
as the single path for trailing content, which the existing tests cover.

Also adds a test for the JsonIOException path, where the stream fails as the
check reads past the parsed value.
Removing the guard along with the unreachable token comparison changed
behavior: a top-level null literal followed by trailing content, and an empty
or blank document, all returned null before and started throwing afterwards.
Gson skips the check for a null value for the same reason, so the guard stays
and only the comparison is gone.

Adds a test for the null literal, which is also the input that exercises the
guard's false branch.
@masiljangajji

Copy link
Copy Markdown
Author

@lhazlewood Coverage is addressed. Two tests were added, and one comparison was removed because no input can reach it.

DuplicateNameRejectingJsonReader is always strict, and JsonReader.doPeek() has only two outcomes once the top level value has been read: it returns END_DOCUMENT at EOF, or checkLenient() throws MalformedJsonException. It never returns a different token, so peek() != JsonToken.END_DOCUMENT was never true and the JsonSyntaxException it guarded was unreachable. Gson performs that comparison because newJsonReader can produce a lenient reader; this deserializer cannot.

Trailing content is still rejected. The reader's own MalformedJsonException is what does it, and testTrailingContentRejected and testTrailingDocumentRejected cover that path unchanged. The value != null guard is kept so an empty document and a top level null still deserialize to null as before.

The new tests cover the two branches that were reachable but untested: a top level null literal, and a stream that fails as the check reads past the parsed value.

Locally clover:check passes at 100% on a clean build, and a corpus of 2,717 inputs run through both constructors shows no behavioral difference from the previous revision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants