Parent roadmap
Contributes to #9 ([M8] Add reproducible i18n Prompt evaluation harness and baseline reports).
Goal
Provide a small schema-validation layer for public evaluation cases so malformed fixtures fail before any model or scoring code runs.
Scope
Define a versioned schema for the stable core fields already described by M8:
schema_version
- stable
id
- source/target locale
- domain
- source text
- optional context
- glossary/required tokens
- accepted references when present
- assertions
- provenance kind/license/source metadata
- difficulty/tags
Add validation tests for representative valid and invalid cases. Validation must run offline and should produce actionable field-level errors.
Non-goals
- No evaluator runtime.
- No model calls or semantic scoring.
- No baseline comparison.
- No external dataset import.
Acceptance criteria
Coordination
Coordinate with #13 and #14. This task is useful only if it converges on the same schema used by the parent roadmap #9 rather than creating an alternative format.
Parent roadmap
Contributes to #9 (
[M8] Add reproducible i18n Prompt evaluation harness and baseline reports).Goal
Provide a small schema-validation layer for public evaluation cases so malformed fixtures fail before any model or scoring code runs.
Scope
Define a versioned schema for the stable core fields already described by M8:
schema_versionidAdd validation tests for representative valid and invalid cases. Validation must run offline and should produce actionable field-level errors.
Non-goals
Acceptance criteria
Coordination
Coordinate with #13 and #14. This task is useful only if it converges on the same schema used by the parent roadmap #9 rather than creating an alternative format.