Skip to content

fix(tts/kokoro-ane): stem possessives instead of G2P-ing the whole token - #884

Merged
Alex-Wengg merged 3 commits into
FluidInference:mainfrom
akmittal006:fix/kokoro-ane-possessive-clitic
Sep 13, 2026
Merged

fix(tts/kokoro-ane): stem possessives instead of G2P-ing the whole token#884
Alex-Wengg merged 3 commits into
FluidInference:mainfrom
akmittal006:fix/kokoro-ane-possessive-clitic

Conversation

@akmittal006

@akmittal006 akmittal006 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Problem

Kokoro ANE sends ordinary possessives absent from Misaki's lexicon to neural G2P as whole tokens, which can produce incorrect pronunciations. Derive known stems with the appropriate /s/, /ᵻz/, or /z/ clitic while preserving explicit dictionary pronunciations.

Resolution order

  • Explicit entries for the full inflected token retain precedence.
  • For hyphenated possessives, look up the whole bare stem directly in the custom and bundled lexicons. This preserves whole-compound stress and reduced vowels in C-section's and mother-in-law's. The probe cannot recurse, split, or invoke G2P.
  • On a whole-stem miss, split the compound and resolve each component normally. This preserves explicit use's and produce's noun readings in land-use's and fresh-produce's.
  • Ordinary possessives resolve known stems without neural fallback, retaining custom overrides and initialism letter names. Unknown stems keep the original inflected token for fallback.

Apostrophe variants are folded before tokenization. The change is confined to possessive 's; it does not add general plural stemming.

Validation

  • 70 related Kokoro ANE and StyleTTS2 phonemizer tests pass, including 50 Kokoro English tests.
  • New whole-compound regressions fail on the preceding PR head and pass after the follow-up fix.
  • Coverage includes whole compound pronunciations, heteronym components, explicit inflected entries, custom/case-sensitive precedence, apostrophe variants, initialisms, and OOV fallback arguments.
  • A separate local audit against the real cached US lexicon passed for 3,726 tokenizable compound stems and 372 explicit possessives, without G2P fallback. The local-cache audit is not part of CI.
  • Strict formatting lint and git diff --check pass.

Validation covers phoneme resolution; no new audio listening or end-to-end synthesis evaluation is claimed.

@Alex-Wengg

Copy link
Copy Markdown
Member

Calling resolvePossessive before hyphen splitting introduces a lexicon-precedence regression when the full compound misses the lexicon.

For example, land-use's previously resolved land + use's, preserving the noun pronunciation. It now resolves land + use and appends the clitic, picking the verb pronunciation of "use." Likewise, fresh-produce's now uses the verb pronunciation of "produce."

I reproduced both against the parent commit using the real cached lexicon. Could we preserve explicit possessive entries for compound components before deriving their pronunciation, and add regression tests for these cases?

@akmittal006

Copy link
Copy Markdown
Contributor Author

Good catch, and thank you for reproducing it against the real cache. The lexicon does carry glued 's keys (347 in the US cache), nearly all noun/verb heteronyms like use's and produce's, and my ordering let the stem + clitic derivation outrank them for a compound's final part.

The follow-up commit splits hyphenated compounds before the possessive rule, so a part with its own glued entry wins (land-use'slˈænd jˈusᵻz, fresh-produce'sfɹˈɛʃ pɹˈOdˌusᵻz) while a part with none still stems (mother-in-law's unchanged). Both of your examples are now regression tests that fail on the previous head. I've also corrected the PR description, which wrongly said no glued possessive keys exist.

@Alex-Wengg

Copy link
Copy Markdown
Member

Thanks, the land-use's ordering is right, but splitting first breaks the opposite case, which is far more common. The real cache has 3,731 hyphenated whole-word stems (C-section, X-ray, T-shirt, well-being, mother-in-law) and exactly one glued hyphenated possessive (re-count's). So C-section's now resolves as C + section's instead of the compound stem sˈisˌɛkʃən + z. Your mother-in-law's example hits it too: the whole stem is in the lexicon with reduced vowels, and the split path gives three full-vowel words. Misaki's stem_s tries the whole stem first.

Add one step between the whole-token miss and the hyphen split: if the token ends in 's, look the stem up as a whole against the lexicons only (no split, no G2P fallback) and append the clitic on a hit. land-use's still misses there and falls through to the split, where use's wins. Regression tests for C-section's and mother-in-law's would lock it in. I've approved the CI runs on your head.

@akmittal006

Copy link
Copy Markdown
Contributor Author

Addressed in 85d1f8a. After a whole inflected-token miss, hyphenated possessives now probe the whole stem directly in the custom and bundled lexicons, preserving their existing lookup precedence. This probe cannot split, recurse, or call G2P. Only a whole-stem miss proceeds to component resolution, so land-use's and fresh-produce's retain their explicit noun-possessive entries.

Added regressions for C-section's, mother-in-law's, X-ray's, T-shirt's, and well-being's, including apostrophe variants. Also covered case-sensitive/custom precedence, the explicit re-count's entry, OOV fallback arguments, and AI's/US's/FBI's. The early probe is limited to hyphenated stems so ordinary initialisms keep their letter-name overrides.

Validation: the new whole-compound regressions failed on the previous head and pass with this change; all 70 related Kokoro ANE and StyleTTS2 phonemizer tests pass. A separate local sweep against the real cached US lexicon passed for 3,726 tokenizable compound stems and 372 explicit possessives, with no G2P fallback. Formatting and whitespace checks also pass. This validates phoneme resolution; it is not an audio listening or end-to-end synthesis claim.

akmittal006 and others added 3 commits September 12, 2026 21:56
The Misaki lexicon stores the `-s` clitic as its own `'s` entry and carries
no glued possessive keys, so `today's` / `someone's` / `the boss's` miss
every lexicon tier in `KokoroAneEnglishPhonemizer.resolveWord` and the whole
inflected token reaches the BART G2P fallback, which mangles it
(`someone's` was heard as "Samian's" in a whisper transcription of the
rendered audio).

Python Misaki does not have this problem because `Lexicon.__call__` falls
through to `stem_s`: the stem is looked up on its own and the clitic phoneme
is appended by rule. This ports that rule.

After a full lexicon miss (and after the FluidInference#710 initialism spell-out), a token
ending in `'s` — case-insensitively, and post the FluidInference#774 apostrophe folding, so
`Today's`, `TODAY'S` and `Today\u{2019}s` all qualify — resolves its stem
through the normal chain minus the G2P fallback, then appends the clitic per
`Lexicon._s`:

  * `/s/`   after a voiceless non-sibilant (`p t k f θ`) — `cat's` -> `kˈæts`
  * `/ᵻz/`  after a sibilant (`s z ʃ ʒ ʧ ʤ`)            — `boss's` -> `bˈɑsᵻz`
  * `/z/`   otherwise                                    — `today's` -> `tədˈAz`

`ᵻ` is Misaki's US form of the epenthetic vowel (`ɪ` is the `british=True`
form); this frontend loads the US lexicon and `ᵻ` is present in the chain's
`vocab.json`.

Faithful to `stem_s`, the rule only fires when the stem is a *known* word, so
an OOV stem leaves the token on the existing whole-word G2P path rather than
being re-shaped from a guess. The stem going through the normal chain means
FluidInference#775's hyphen split still applies underneath the clitic
(`mother-in-law's` -> `mˈʌðɜɹ ɪn lˈɔz`), and a glued lexicon entry that does
exist (`it's`) still wins, because stemming runs only after the miss.

`resolveWord` / `resolveHyphenatedCompound` gain an `allowFallback` flag
(defaulting to today's behavior) to express "resolve, but only if known".

Tests: voiced / voiceless / sibilant / vowel-final stems, curly apostrophe,
uppercase `'S` with a case-sensitive stem, hyphenated-compound stem, OOV-stem
fallback, lexicon-entry precedence, and a direct table check of the clitic
rule against `Lexicon._s`.

`swift build`, `swift test --filter TTS` (57 tests) and `swift-format lint`
are clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ssives

The `-'s` stem+clitic rule was placed before the hyphenated-compound split
in `resolveWord`, which broke lexicon precedence for compounds whose final
part has a glued possessive entry of its own.

The US lexicon carries 347 glued `-'s` keys, mostly noun/verb heteronyms
whose possessive does not read as the bare word plus a clitic:

  use      = jˈuz        use's      = jˈusᵻz
  produce  = pɹədˈus     produce's  = pɹˈOdˌusᵻz

With the possessive rule first, `land-use's` stemmed to `land-use`, split
to `land` + `use`, and picked up the *verb* reading plus a clitic. Running
the hyphen split first restores the pre-existing behaviour: parts are
`land` + `use's`, and `use's` hits its own entry directly. Same for
`fresh-produce's`.

Compounds with no glued entry are unaffected, because each part is
resolved through the same chain: `mother-in-law's` splits to `mother`,
`in`, `law's`; `law's` misses, recurses, and stems to `law` + /z/.

Adds regression tests for both heteronym compounds (they fail on the
previous ordering) and updates the ordered resolution doc-comment.
@akmittal006
akmittal006 force-pushed the fix/kokoro-ane-possessive-clitic branch from 85d1f8a to 35ffe11 Compare September 12, 2026 16:26
@Alex-Wengg
Alex-Wengg merged commit fbc1b86 into FluidInference:main Sep 13, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants