Repository navigation
feat(morphology): substantivized noun tags, with a typed MISC column - #87
Merged
Merged
Conversation
… of noun-like numerals The singular accusative of an adjective holds three genders, masculine, neuter, feminine, where every other case merges masculine with neuter and leaves two. `declensionSubstAdj` read it as if it held two, so a neuter substantivized adjective printed the masculine accusative, `srědnji` for `srědnje`, and a feminine one printed the neuter, `prěměnno` for `prěměnnų`. A numeral that declines as a noun, `četverka`, `četvŕtina`, `šesť`, dropped the vocative on the way out: the paradigm never carried the slot and the case dictionary had no name for it. Nouns tagged `m.anim.subst.`, `m.subst.`, `f.subst.`, `n.subst.` now decline as adjectives without the bracketed genitive the dictionary spells out today; the bracketed entries keep working untouched.
A vocative is the case of address, and a cardinal is a quantifier: `šesť` declines like the i-stem `kosť` in the oblique cases but is never addressed in any Slavic language. Only the numerals that are nouns, `num.subst.` and `num.fract.`, carry the slot.
…s in MISC UPOS says NOUN, which is what the lexicon means by `učeny` or `prěměnna`: these are headwords with their own id and their own fixed gender, not adjectives that happen to head a phrase. That leaves nothing in the standard columns to say the forms come from the adjective tables, so the tokens carry `Derivation=Adj` in MISC. An adapter can now contribute MISC of its own; `inflectWord` merges it under whatever the caller passes.
MISC is where an annotation goes when it has no home in the standard columns, which makes it the one column that grows a key nobody remembers adding. It was `Record<string, string>`, and that is how the previous commit put `Derivation=Adj` on substantivized nouns without noticing that `derive` had already claimed `Derivation` for the code of the rule that produced a token, `v_prap` or `adj_adv`, which the website reads back and displays. The key is now `Declension=Adj`, which says the thing it actually means. The type carries the attributes the CoNLL-U spec defines (SpaceAfter, NewPar, Lang, Translit, LTranslit, Gloss) beside this project's own (LID, SID, Derivation, Declension). An unknown key is now a type error, and a misspelled one comes back with the spelling it meant. Also closes a hole the optional properties opened: `formatMisc` filtered empty values but not `undefined`, so an attribute spread in from another bundle would have printed as `Key=undefined`. `formatFeats` already guarded this.
A `*.subst.` tag spelled its own genitive out from the lemma, but only for the singular: the plural got `-ogo` where `declensionPluralNoun` wanted `-yh`, refused it, and returned nothing at all. Both numbers now agree with the bracketed route the lexicon uses today, `drobne (-yh)` and `prisęžni (-yh)` among them, and both pick up `Declension=Adj`, which until now only the singular carried. The hint is also gated. Hardness is legible in a masculine or neuter ending and nowhere else, so the feminine `-a` and both plural endings are read off the last consonant of the stem — exact, since `y` cannot follow `ž š č c`. But a lemma that ends as no adjective ends is no adjective whatever the tag says: `dom` tagged `m.subst.` was handed `*doogo` and a whole paradigm built on it. Such a lemma gets no hint, the gender half of the tag stands, and the word declines as the noun it is. `NounParadigm` goes back to holding cases and nothing else. The bracketed genitive is put to one shared predicate twice instead — by `declineNoun`, which routes on it, and by `inflectNoun`, which reports it — so `nounForms` can sweep the paradigm with `Object.values` again and the snapshots lose the 25 entries that carried a note about themselves between their cases. The numeral adapter kept a second copy of the case map, untyped, which is how it came to lack a name for the vocative. It imports the noun one. Ten paradigms are snapshotted under the new tags, `veliki` among them: its hint comes out `-ego` and the adjective tables still return `velikogo`, because `establish_root` drops the soft marker after `k g h`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The column was typed key by key one commit ago, and the list ran ahead of the repository: `NewPar`, `Lang`, `Translit` and `LTranslit` had no writer or reader anywhere, and `Gloss` appeared only in the tests that had just been written for it. A type with no index signature has to name every legal key, so those can come back when something sets them; until then they described the CoNLL-U spec rather than this column. `Derivation` was the reverse case: declared as `string`, though the codes it may carry have been a closed set since `derive` was written. `DerivationCode` moves out of `derive/virtual-id.ts` and in beside the key it fills, and loses its trailing `| string` — which had been collapsing the whole union back to `string`, so the type said nothing the name did not. `virtual-id.ts` re-exports it. `docs/conllu-conventions.md` listed `SID` alone of the five, and had never listed `Derivation`. It lists all of them, and records what the golden dump does not cover: sid, lemma, form and FEATS go through it, MISC does not, so a change to an attribute above moves snapshots and nothing else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three items from a bug report against
@interslavic/morphology, plus what follows from the third.Singular accusative of substantivized adjectives
declineAdjective(...).singular.accholds three genders: masculine (an animate/inanimate pair), neuter, feminine. Every other singular case merges masculine with neuter and leaves the feminine second.declensionSubstAdjread the accusative as if it were merged too, so:srědnjiforsrědnje (-ego)andšampanskyforšampansko;prěměnnoforprěměnna (-oj)andkrivoforkriva.Now the neuter accusative repeats the nominative and the feminine takes index
2, so those forms come outsrědnje,prěměnnų,krivų. Nine lexicon entries move.Vocative of
num.subst.andnum.fract.A numeral that delegates to
declineNounbuilt itscaseswithout avockey, andCASEShad no name for it either, soinflectNumeral('četverka', 'num.subst.')never emittedčetverko. Both tags now carry the slot.Cardinals do not get one. A vocative is the case of address, and
šesť, though it declines like the i-stemkosťin the oblique cases, is a quantifier.četverkaandčetvŕtinaare ordinary nouns and take the case as such.*.subst.noun tagsThe lexicon annotates substantivized adjectives as ordinary nouns plus a bracketed genitive hint,
učeny (-ogo). The tagsm.anim.subst.,m.subst.,f.subst.,n.subst.now work on their own, in both numbers:parsePosmarks the noun substantivized and the declension is read off the lemma.Hardness is what the lemma has to yield. Masculine and neuter endings carry it (
pųtujųćiandsrědnjeagainstdežurnyanddobro); the feminine-aand both plural endings (dobrebesidesrědnje) do not, so for those it is read off the last consonant of the stem. That step is exact, not a guess:ycannot followž š č cin the orthography, so an adjective with a soft stem is spelled soft and declines soft.A lemma that ends as no adjective ends gets no hint, whatever the tag claims.
domunderm.subst.would otherwise be handed the genitive*doogoand a whole paradigm built on it; instead the gender half of the tag is honoured andsubst.is dropped.*.pl.subst.yields the same paradigm as the bracketedm.pl. (-yh), which two lexicon entries already use. A test asserts the two routes agree on nine lemmas, and ten paradigms are snapshotted,velikiamong them: its hint comes out-ego, and the adjective tables still returnvelikogo, becauseestablish_rootdrops the soft marker afterk g h.Declension=Adjin MISCThese entries stay
NOUN. The UD guideline that keeps the ADJ tag covers adjectives that exceptionally head a nominal phrase, which is a statement about tokens in running text; here the lexicon has already made the call, givingprěměnna(35526) its own id and citation form beside the adjectiveprěměnny(35525). Tagging them ADJ would put two ADJ lemmas on one paradigm and promise a three-gender declension we do not emit, and it would drop the gender the dictionary supplies.That leaves nothing in UPOS or FEATS to record that the forms come from the adjective tables. UD has no universal feature for substantivization, so the tokens carry
Declension=Adjin MISC.AdapterResultgains an optionalmisc, andinflectWordmerges it under whatever the caller passes.The marker is computed rather than carried:
NounParadigmholds cases and nothing else, so the bracketed genitive is put to one shared predicate twice, bydeclineNoun, which routes on it, and byinflectNoun, which reports it. The two plural entries in the lexicon,drobneandprisęžni, pick the marker up for the first time.A typed MISC column
The key above started out as
Derivation=Adj, which was already taken:derivewrites the code of the rule that produced a token there,v_praporadj_adv, andwebsite/src/main.tsreads it back as a label.Record<string, string>accepted both meanings on the one key.Miscnow lists the attributes the column carries, with no index signature:SpaceAfterfrom the CoNLL-U spec, andLID,SID,DerivationandDeclensionfrom this project. An unknown key is a type error, and a misspelled one gets the intended name suggested. Spec attributes nothing in the repository sets are left out until something sets them.DerivationCodemoves fromderive/virtual-id.tsinto@interslavic/conllu, beside the key it fills, and loses its trailing| string. That member had been collapsing the whole union tostring, soDerivationwas untyped in effect as well as in name.virtual-id.tsre-exports it.The same change fixes a latent bug:
formatMiscfiltered empty values but notundefined, which optional properties make reachable, so an attribute spread in from another bundle would have printed asKey=undefined.formatFeatsalready guarded this.Also
docs/conllu-conventions.mdlisted one MISC key out of five, and had never listedDerivation. It lists all of them now, and records that the golden dump covers sid, lemma, form and FEATS but not MISC, which is why a MISC change moves snapshots and nothing else.yarn test,yarn lintandyarn buildpass; snapshots and the golden lexicon dump are updated.