LLM 产出按 language 生成中英文，生成层缓存按语言判定（i18n C3）

docs/sessions/vta-i18n-261019/progress/C3-llm-output-progress.md

@@ -0,0 +1,85 @@
+# C3 LLM 产出语言 — 进度存档
+
+## 2026-10-19 锁定中文 prompt 快照
+
+- **当前阶段**：implementing
+- **本段结论**：先用测试锁死 `llm/prompts/__init__.py` 相对 Base `708352f6` 字节级不变，以及中文 builder 在 `language=zh-CN` 下的产出。生产代码尚未改动。
+- **关键决策与已否决方案**：中文 prompt 留在原模块、英文另文件；否决改写 `__init__.py` 正文。英文篇幅系数仍按卡面假设 1.8。
+- **下一步唯一动作**：新增 `llm/prompts/en.py` 与按 `get_language()` 取 prompt 的访问函数，再把总结/笔记/章节处理器接到该访问函数。
+
+## 2026-10-19 英文 prompt 与篇幅系数接入处理器
+
+- **当前阶段**：implementing
+- **本段结论**：新增原生英文 prompt 模块与 `prompts/localized.py` 访问函数；总结/笔记/章节处理器按 `get_language()` 选 prompt。英文篇幅系数 1.8 作用于 summary budget 与 `min_summary_threshold`。章节标题英文按至多 12 个 word 写进 prompt，本地截断按 48 字符。
+- **关键决策与已否决方案**：中文 `__init__.py` 零改动；否决在中文 prompt 后拼接 write in English。`localized.chapters_schema()` 因与 chapters_processor 循环导入已撤，schema 在调用点分支。
+- **下一步唯一动作**：说话人兜底占位名按语言生成，并补 speaker_inferencer 测试。
+
+## 2026-10-19 说话人兜底名按语言生成
+
+- **当前阶段**：implementing
+- **本段结论**：`_build_fallback_label` 在 zh-CN 仍输出 `说话人N`，en 输出 `Speaker N`。复合推测名自动带上该前缀。
+- **关键决策与已否决方案**：复合名中的「标题关联推测」等中文标记保持原样（卡面只锁占位名）；不迁移历史 `speaker_mapping.json`。
+- **下一步唯一动作**：provenance 写入 language，缓存复用按语言判定，并打 LANGUAGE-MISMATCH-REGENERATE 日志。
+
+## 2026-10-19 provenance 语言判定与缓存隐藏
+
+- **当前阶段**：implementing
+- **本段结论**：生成层 provenance 写入 `language`（缺字段=zh-CN）。`get_cache` 对语言不一致的总结/笔记/章节视为缺失，转录路径会重跑该层。`LANGUAGE-MISMATCH-REGENERATE` 在 llm_ops 打 WARNING。校对层不因 language 重跑。分享卡文案不进 provenance，随总结重生删除旧卡。
+- **关键决策与已否决方案**：否决改缓存键；否决改 transcription.py。用 get_cache 隐藏层 + llm_ops 磁盘判定双保险。
+- **下一步唯一动作**：红验 LANGUAGE-MISMATCH-REGENERATE 断言，再跑 make test 与 scope check。
+
+## 2026-10-19 验收与红验
+
+- **当前阶段**：implementing
+- **本段结论**：改坏日志字面量后 `test_language_mismatch_regenerate_log_locked` 以 AssertionError 转红并已还原。`make test` 退出码 0。`card_scope_check.py` 对 Base..HEAD 绿。本机无 LLM 凭据且出站网络被拦，真实样本补跑留给主脑。
+- **关键决策与已否决方案**：红验只改 `log_language_mismatch_regenerate` 那一行日志前缀，未整文件 checkout。
+- **下一步唯一动作**：提交本段进度并 push 到 `card/VideoTranscriptAPI-20261019-05`。
+
+## 2026-10-19 撤回 get_cache 语言隐藏
+
+- **当前阶段**：implementing
+- **本段结论**：`get_cache` 不再按语言过滤。ViewTokenResolver 在 language=en 时仍返回原中文 summary/notes。复用判定收口为 `language_mismatched_generation_layers`，transcription / llm_ops / generate_notes 共用。HEAD `09da4805` 上该 View 测试以 AssertionError 先红。
+- **关键决策与已否决方案**：否决继续用 get_cache 隐藏层。语言不一致的笔记允许重新生成，同语言仍 400。
+- **下一步唯一动作**：跑 Narrow-Verify 与 make test，scope check 后 push。
+
+## 2026-10-19 分享卡不再单独做语言判定
+
+- **当前阶段**：implementing
+- **本段结论**：从 `_GENERATION_LAYER_FILES` 删除 share_card。en 下 summary 已是 en 且仅有分享卡文件时 mismatch 为空。HEAD `7882798b` 上该测试以 AssertionError 先红（`('share_card', 'zh-CN', 'en')`）。
+- **关键决策与已否决方案**：否决给分享卡补 provenance；它随总结重生删除重建。
+- **下一步唯一动作**：generate_notes 准入恢复 `_read_fresh_notes_status` 权威读。
+
+## 2026-10-19 generate_notes 准入恢复权威读
+
+- **当前阶段**：implementing
+- **本段结论**：准入段改回读 `llm_status.json`。函数返回整份 payload，调用方取 notes_status 并把 payload 交给语言判定。非法 JSON / 非 object 仍抛错。不再在准入段调 get_cache。
+- **关键决策与已否决方案**：保留函数名 `_read_fresh_notes_status`，扩大返回值为整份 status 对象而不是另起函数。
+- **下一步唯一动作**：跑 Narrow-Verify 与 make test，scope check 后 push。
+
+## 2026-10-19 笔记篇幅阈值按输出语言缩放
+
+- **当前阶段**：implementing
+- **本段结论**：笔记生成与重试后的超长判定都按当前输出语言系数缩放；100 字符源、81 字符输出在 en 下单次调用，在 zh-CN 下仍触发原重试。
+- **关键决策与已否决方案**：保留中文阈值 0.8，英文阈值只乘既有 `length_budget_coefficient()`，不增设配置。
+- **下一步唯一动作**：统一 coordinator 与 SummaryProcessor 的最短总结阈值计算。
+
+## 2026-10-19 总结最短长度预检共用阈值
+
+- **当前阶段**：implementing
+- **本段结论**：SummaryProcessor 与 coordinator 都调用 `effective_min_summary_threshold()`；en 下 800 字输入在 500×1.8 阈值前被预检跳过，避免两处门槛不一致。
+- **关键决策与已否决方案**：以一个计算函数封装语言系数，否决 coordinator 自行重复计算。
+- **下一步唯一动作**：按总结 provenance 语言缩放审计 hard-cap 统计。
+
+## 2026-10-19 审计统计按总结 provenance 语言缩放
+
+- **当前阶段**：implementing
+- **本段结论**：summary-ratio 读取每条媒体总结 provenance 的语言并经 `generation_layer_language()` 处理；原文长 10,000、摘要长 7,000 时 en 不超预算而 zh-CN 超预算。
+- **关键决策与已否决方案**：不读取当前配置语言；缺失 provenance 语言沿用该函数的 zh-CN 语义，英文系数引用 prompts/en.py 常量。
+- **下一步唯一动作**：合并章节标题长度限制的重复定义。
+
+## 2026-10-19 章节标题限制集中到语言 prompt 定义
+
+- **当前阶段**：implementing
+- **本段结论**：中文 24/23 字符限制集中在 localized.py，英文 12-word 提示与 48/47 字符截断都由 en.py 常量驱动；章节处理器不再另存中文数值。
+- **关键决策与已否决方案**：用英文 prompt 常量插值，JSON 示例转义后渲染文本不变；中文 prompt 模块未改动。
+- **下一步唯一动作**：跑指定 LLM/i18n 套件与 make test，再做 scope 与状态核对。

src/video_transcript_api/api/routes/tasks.py

@@ -30,6 +30,7 @@
 from ..services.view_token_resolver import ViewTokenResolver
 from ...utils.busy_lock import acquire_for_task, release_for_task
 from ...utils.llm_status import ChaptersStatus, NotesStatus, SummaryStatus
+from ...cache.cache_manager import language_mismatched_generation_layers
 from ...utils.notifications import send_view_link_wechat, get_notification_router
 from ...utils.task_status import TaskStatus, http_code_for_status
 from ..services.terminal_status import finalize_terminal_status_and_notify
@@ -125,8 +126,13 @@ def _admit_upload_reprocess(cursor, view_token, user_id, cache_data):
     return ""
 
 
-def _read_fresh_notes_status(cache_dir: str | None) -> str | None:
-    """Read notes_status directly from the authoritative cache status file."""
+def _read_fresh_notes_status(cache_dir: str | None) -> dict | None:
+    """Read the authoritative llm_status.json object from the cache directory.
+
+    Returns the JSON object so callers can take notes_status and hand the same
+    payload to language_mismatched_generation_layers. Missing file returns
+    None. Invalid JSON or a non-object payload raises (fail-loud).
+    """
     if not cache_dir:
         raise ValueError("Detailed notes cache directory is unavailable")
     status_file = Path(cache_dir) / "llm_status.json"
@@ -135,7 +141,7 @@ def _read_fresh_notes_status(cache_dir: str | None) -> str | None:
     status_payload = json.loads(status_file.read_text(encoding="utf-8"))
     if not isinstance(status_payload, dict):
         raise ValueError("llm_status.json must contain a JSON object")
-    return status_payload.get("notes_status")
+    return status_payload
 
 
 # K1 路由站点观察面（CI review 第 3 轮 major）：_fail_task_after_creation
@@ -1273,8 +1279,13 @@ async def generate_notes(
             detail="详细笔记依赖已生成的章节层，请先生成章节后再重试",
         )
     if llm_status.get("notes_status") == NotesStatus.GENERATED:
-        logger.warning(f"详细笔记已存在，拒绝重复生成: {view_token}")
-        raise HTTPException(status_code=400, detail="详细笔记已存在，无需重复生成")
+        notes_language_mismatch = any(
+            layer == "notes"
+            for layer, _old, _new in language_mismatched_generation_layers(cache_data)
+        )
+        if not notes_language_mismatch:
+            logger.warning(f"详细笔记已存在，拒绝重复生成: {view_token}")
+            raise HTTPException(status_code=400, detail="详细笔记已存在，无需重复生成")
 
     try:
         with cache_manager._get_cursor() as cursor:
@@ -1335,9 +1346,19 @@ async def generate_notes(
                 )
 
                 if inflight_notes_task_id is None:
+                    fresh_status = _read_fresh_notes_status(cache_file_path)
                     notes_generated_during_admission = (
-                        _read_fresh_notes_status(cache_file_path)
+                        (fresh_status or {}).get("notes_status")
                         == NotesStatus.GENERATED
+                        and not any(
+                            layer == "notes"
+                            for layer, _old, _new in language_mismatched_generation_layers(
+                                {
+                                    "llm_status": fresh_status or {},
+                                    "file_path": cache_file_path,
+                                }
+                            )
+                        )
                     )
                     if not notes_generated_during_admission:
                         cursor.execute(

src/video_transcript_api/api/services/llm_ops.py

@@ -55,6 +55,8 @@
     NotesStatus,
     SummaryStatus,
 )
+from ...i18n import get_language
+from ...cache.cache_manager import language_mismatched_generation_layers
 
 logger = lazy_resource(get_logger)
 config = lazy_resource(get_config)
@@ -68,6 +70,25 @@
     "notes discarded to avoid misalignment"
 )
 
+def _provenance_layers(llm_status: Any) -> dict:
+    if not isinstance(llm_status, dict):
+        return {}
+    provenance = llm_status.get("artifact_provenance")
+    if (
+        isinstance(provenance, dict)
+        and provenance.get("schema_version") == "artifact-provenance-v1"
+        and isinstance(provenance.get("layers"), dict)
+    ):
+        return provenance["layers"]
+    return {}
+
+
+def log_language_mismatch_regenerate(layer: str, old_language: str, new_language: str) -> None:
+    """Emit the locked LANGUAGE-MISMATCH-REGENERATE warning."""
+    logger.warning(
+        f"LANGUAGE-MISMATCH-REGENERATE layer={layer} old={old_language} new={new_language}"
+    )
+
 
 def _requires_llm_title(
     processing_options: dict, *, use_speaker_recognition: bool
@@ -403,6 +424,17 @@ def _handle_notes_generation(
     if not cache_dir:
         raise ValueError("Detailed notes task requires a valid cache directory")
 
+    notes_status_snapshot = cache_manager.get_cache(
+        platform,
+        media_id,
+        use_speaker_recognition=use_speaker_recognition,
+    )
+    for layer_name, old_language, new_language in language_mismatched_generation_layers(
+        notes_status_snapshot,
+    ):
+        if layer_name == "notes":
+            log_language_mismatch_regenerate(layer_name, old_language, new_language)
+
     chapter_count = 0
     try:
         chapters_payload = json.loads(
@@ -476,11 +508,30 @@ def write_notes_progress(done: int, total: int) -> None:
             ):
                 raise OSError("Failed to persist detailed notes artifact")
 
+            notes_cache = cache_manager.get_cache(
+                platform,
+                media_id,
+                use_speaker_recognition=use_speaker_recognition,
+            )
+            notes_layers = dict(
+                _provenance_layers((notes_cache or {}).get("llm_status"))
+            )
+            notes_record = notes_layers.get("notes")
+            if not isinstance(notes_record, dict):
+                notes_record = {}
+            else:
+                notes_record = dict(notes_record)
+            notes_record["language"] = get_language()
+            notes_layers["notes"] = notes_record
             cache_manager.save_llm_status(
                 platform=platform,
                 media_id=media_id,
                 use_speaker_recognition=use_speaker_recognition,
                 notes_status=NotesStatus.GENERATED,
+                artifact_provenance={
+                    "schema_version": "artifact-provenance-v1",
+                    "layers": notes_layers,
+                },
             )
         notes_outcome["notes_status"] = NotesStatus.GENERATED
         status_written = finalize_terminal_status_and_notify(
@@ -697,6 +748,31 @@ def _handle_llm_task(llm_task: dict):
                 else:
                     skip_chapters_for_coordinator = not chapters_requested
 
+                lang_cache = cache_snapshot_for_flags
+                if lang_cache is None and platform and media_id:
+                    lang_cache = cache_manager.get_cache(
+                        platform,
+                        media_id,
+                        use_speaker_recognition=use_speaker_recognition,
+                    )
+                language_mismatches = language_mismatched_generation_layers(lang_cache)
+                if not calibrate_only:
+                    for layer_name, _old_language, _new_language in language_mismatches:
+                        if layer_name == "summary" and summarize_requested:
+                            skip_summary_for_coordinator = False
+                        elif layer_name == "chapters" and chapters_requested:
+                            skip_chapters_for_coordinator = False
+                for layer_name, old_language, new_language in language_mismatches:
+                    will_generate = (
+                        layer_name == "summary" and not skip_summary_for_coordinator
+                    ) or (
+                        layer_name == "chapters" and not skip_chapters_for_coordinator
+                    )
+                    if will_generate:
+                        log_language_mismatch_regenerate(
+                            layer_name, old_language, new_language
+                        )
+
                 # Cache-side chapters seed (this-round dialogs preferred inside
                 # coordinator after calibration). Keep llm/ package pure.
                 timeline_segments_seed = None
@@ -2547,6 +2623,7 @@ def record_saved_artifact(layer: str, filename: str) -> Optional[dict]:
                     "generation_kind": source.get("generation_kind", "unknown"),
                     "actual_model": "unknown",
                     "prompt_version": "unknown",
+                    "language": get_language(),
                 }
                 requested_model = source.get("requested_model")
                 if isinstance(requested_model, str) and requested_model:
@@ -2568,6 +2645,15 @@ def record_saved_artifact(layer: str, filename: str) -> Optional[dict]:
                 else:
                     final_artifact_layers["summary"] = summary_record
 
+        if chapters_saved:
+            chapters_record = final_artifact_layers.get("chapters")
+            if not isinstance(chapters_record, dict):
+                chapters_record = {}
+            else:
+                chapters_record = dict(chapters_record)
+            chapters_record["language"] = get_language()
+            final_artifact_layers["chapters"] = chapters_record
+
         final_artifact_provenance = (
             {
                 "schema_version": "artifact-provenance-v1",

src/video_transcript_api/api/services/summary_ratio_stats.py

@@ -8,6 +8,8 @@
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Tuple
 
+from ...cache.cache_manager import generation_layer_language
+from ...llm.prompts.en import LENGTH_BUDGET_COEFFICIENT
 from ...transcriber import FunASRSpeakerClient
 from ...utils.logging import setup_logger
 from ...llm.core.summary_budget import (
@@ -205,7 +207,22 @@ def compute_summary_ratio_stats(
             band_ratios[band].append(ratio)
             if ratio > 1.0:
                 band_over_100[band] += 1
-            hard_cap = compute_summary_budget(original_length, budget_config).hard_cap
+            status_path = cache_dir / "llm_status.json"
+            if status_path.exists():
+                with status_path.open("r", encoding="utf-8") as handle:
+                    status_data = json.load(handle)
+            else:
+                status_data = {}
+            provenance = status_data.get("artifact_provenance")
+            layers = provenance.get("layers") if isinstance(provenance, dict) else None
+            summary_record = layers.get("summary") if isinstance(layers, dict) else None
+            summary_language = generation_layer_language(summary_record)
+            coefficient = (
+                LENGTH_BUDGET_COEFFICIENT if summary_language == "en" else 1.0
+            )
+            hard_cap = compute_summary_budget(
+                original_length, budget_config, coefficient=coefficient
+            ).hard_cap
             if summary_length > hard_cap:
                 band_over_hardcap[band] += 1
 

src/video_transcript_api/api/services/transcription.py

@@ -1311,7 +1311,18 @@ def _fail_task_and_notify(
                 logger.info("使用 CapsWriter 缓存文本")
 
             has_llm_calibrated = "llm_calibrated" in cache_data
-            has_llm_summary = "llm_summary" in cache_data
+            from ...cache.cache_manager import language_mismatched_generation_layers
+
+            mismatched_generation_layers = {
+                name
+                for name, _old, _new in language_mismatched_generation_layers(
+                    cache_data
+                )
+            }
+            has_llm_summary = (
+                "llm_summary" in cache_data
+                and "summary" not in mismatched_generation_layers
+            )
 
             # ---- 分层缓存命中判定（相对本次请求的 processing_options）----
             # 缓存产物只增不减：required = {transcript} ∪ (calibrate→calibrated) ∪
@@ -1377,7 +1388,9 @@ def _fail_task_and_notify(
             chapters_requested = processing_options.get("chapters", True)
             cached_chapters_status = cached_llm_status.get("chapters_status")
             has_llm_chapters = "llm_chapters" in cache_data
-            if cached_chapters_status == ChaptersStatus.GENERATED and has_llm_chapters:
+            if "chapters" in mismatched_generation_layers:
+                chapters_layer_satisfied = False
+            elif cached_chapters_status == ChaptersStatus.GENERATED and has_llm_chapters:
                 chapters_layer_satisfied = True
             elif cached_chapters_status in (
                 ChaptersStatus.SKIPPED_SHORT,