feat(gate): scan chosen folders for terms, and project codes outside their folder

skills/km/SKILL.md

@@ -297,7 +297,7 @@ All writes follow CONVENTIONS.md for frontmatter, folder placement, and naming.
 
 After writing the main file, update related pages before committing:
 
-1. **Cross-reference:** Find up to 5 related files (ranked by: shared `project:` > shared tags > same folder). Add the new file's repo-root-relative path to their `related:` frontmatter (create the field if absent). Only add genuinely useful links
+1. **Cross-reference:** Find up to 5 related files (ranked by: shared `project:` > shared tags > same folder). Add the new file's repo-root-relative path to their `related:` frontmatter (create the field if absent). Only add genuinely useful links. When the brain sets `gate.project_prefix`, never add a path under a project folder to a doc outside that folder, nor rank by a `project:` code there: the gate reports a project code outside its own folder, frontmatter included. Project notes link up to neutral docs, not the other way
 2. **Update `_index.md`:** Ensure the file is listed in its folder's `_index.md` and in the matching `project:` folder's `_index.md` (if they exist — do not create new `_index.md`)
 3. **Confirm:** List all proposed ripple updates and wait for user approval before committing
 
@@ -365,12 +365,21 @@ gate:
   forbidden_terms_file: .gate-terms.txt   # gitignored: keep real customer names out of the repo
   email_allowlist: [your-domain.com]
   # forbidden_terms: [ACME]        # or inline (test/demo only)
+  # term_scan_prefixes: [bms, products]   # also term-scan these folders' docs, whatever their audience
+  # project_prefix: projects/      # its subfolder names are project codes; flag them outside their folder
+  # project_code_pattern: 'p[0-9]+' # only subfolders fully matching this count as codes
 ```
 
 Checks, keyed per concern (so an internal `accepted` doc may legitimately name a customer):
 - **secret** (ERROR) — AWS/GitHub keys, private-key headers in ANY tracked doc (incl. exempt/reserved
   files like README/_index; excl. skip_prefixes) — a key needs no frontmatter.
 - **leak** (ERROR) — forbidden terms + external emails only in an `audience: customer` doc.
+  `term_scan_prefixes` term-scans every doc under those folders too, and with `project_prefix` a
+  project code (a tracked subfolder name) outside its own folder is an error; files directly in
+  the project folder, such as its `_index.md`, may list every code. Both run over any
+  tracked doc, `_index.md` and README included, frontmatter too (a `project:` value or a
+  `related:` path naming a project folder counts), and the code check needs no terms file, so it
+  also runs in CI where that file is absent.
 - **bergab** (ERROR) — a served doc links to an unfinished (`draft`/`review`) doc; a link to
   `superseded`/`obsolete` is a WARNING. Reuses the `status`/`audience` frontmatter, no new fields.
 - **freshness** (WARNING) — a served doc past its `review_by`.

src/km/templates/team/schema.local.team.template.yaml

@@ -22,6 +22,9 @@ gate:
   enabled: true
   served_status: accepted
   customer_audience: customer
+  # term_scan_prefixes: []        # folders whose docs are term-scanned whatever their audience
+  # project_prefix: projects/     # subfolder names are project codes, flagged outside their folder
+  # project_code_pattern: 'p[0-9]+'   # only matching subfolder names count as codes
 
 # Repo-only fields (shallow-merged over the base schema).
 fields:

src/km/validate.py

@@ -13,7 +13,9 @@
 When `schema.local.yaml` sets `gate.enabled: true`, extra checks run, keyed per concern:
   * secret   (ERROR)   AWS/GitHub keys, private-key headers in ANY tracked doc (incl. exempt/reserved,
                        excl. skip_prefixes) — a key needs no frontmatter
-  * leak     (ERROR)   forbidden terms + external emails in a CUSTOMER-facing doc (audience==customer)
+  * leak     (ERROR)   forbidden terms + external emails in a CUSTOMER-facing doc (audience==customer);
+                       forbidden terms also in any doc under `term_scan_prefixes`; and, with
+                       `project_prefix`, a project folder's name in any doc outside that folder
   * bergab   (ERROR)   a served doc (status==served_status) links to an unfinished (draft/review) doc;
                        a link to superseded/obsolete is a WARNING
   * freshness(WARNING) a served doc whose `review_by` is in the past is stale
@@ -23,7 +25,10 @@
 missing file is a WARNING, not silent. Malformed OR unknown gate config fails fast with a clear message.
 Gate off/absent = plain behaviour, unchanged. Config keys (optional): enabled, served_status
 (default 'accepted'), customer_audience (default 'customer'), forbidden_terms (list),
-forbidden_terms_file (path), email_allowlist (list). Editable-metadata advisory check: catches honest
+forbidden_terms_file (path), email_allowlist (list), term_scan_prefixes (list), project_prefix (str),
+project_code_pattern (regex). The path-scoped term scan and the project codes run in the text pre-pass
+with the secrets, so they also see exempt and _index files; project codes are the tracked subfolders
+of project_prefix (fully matching project_code_pattern, if set). Editable-metadata advisory check: catches honest
 mistakes before commit, does NOT stop a determined author (that is the publish/release step's job).
 """
 from __future__ import annotations
@@ -58,6 +63,7 @@
 _GATE_KEYS = {
     "enabled", "served_status", "customer_audience",
     "forbidden_terms", "forbidden_terms_file", "email_allowlist",
+    "term_scan_prefixes", "project_prefix", "project_code_pattern",
 }
 _gate_raw = SCHEMA.get("gate")
 _gate_cfg_errors: list[str] = []
@@ -74,10 +80,16 @@
                 _gate_cfg_errors.append(f"unknown gate key {_k!r} (allowed: {', '.join(sorted(_GATE_KEYS))})")
         if "enabled" in GATE and not isinstance(GATE["enabled"], bool):
             _gate_cfg_errors.append("gate.enabled must be true/false")
-        for _k in ("served_status", "customer_audience", "forbidden_terms_file"):
+        for _k in ("served_status", "customer_audience", "forbidden_terms_file", "project_prefix",
+                   "project_code_pattern"):
             if _k in GATE and not isinstance(GATE[_k], str):
                 _gate_cfg_errors.append(f"gate.{_k} must be a string")
-        for _k in ("forbidden_terms", "email_allowlist"):
+        if isinstance(GATE.get("project_code_pattern"), str):
+            try:
+                re.compile(GATE["project_code_pattern"])
+            except re.error as exc:
+                _gate_cfg_errors.append(f"gate.project_code_pattern is not a valid regex: {exc}")
+        for _k in ("forbidden_terms", "email_allowlist", "term_scan_prefixes"):
             if _k in GATE and not (isinstance(GATE[_k], list) and all(isinstance(x, str) for x in GATE[_k])):
                 _gate_cfg_errors.append(f"gate.{_k} must be a list of strings (a missing '-' makes it a string)")
         GATE_ON = bool(GATE.get("enabled"))
@@ -120,6 +132,29 @@ def _load_terms() -> list[str]:
 
 
 FORBIDDEN_TERMS = _load_terms()
+
+
+def _folder(s: str) -> str:
+    """'bms' and 'bms/' both become 'bms/', so they match 'bms/...' and never 'bmsx/...'."""
+    s = s.strip("/")
+    return s + "/" if s else ""
+
+
+TERM_SCAN_PREFIXES = tuple(f for f in map(_folder, GATE.get("term_scan_prefixes") or []) if f)
+PROJECT_PREFIX = _folder(GATE.get("project_prefix") or "")
+
+
+def _load_project_codes() -> list[str]:
+    """Tracked subfolders of project_prefix, so a local-only folder never changes the result."""
+    if not (GATE_ON and PROJECT_PREFIX):
+        return []
+    pat = re.compile(GATE.get("project_code_pattern") or ".*")
+    subs = {p[len(PROJECT_PREFIX):].split("/", 1)[0] for p in tracked_md()
+            if p.startswith(PROJECT_PREFIX) and "/" in p[len(PROJECT_PREFIX):]}
+    return sorted(c for c in subs if pat.fullmatch(c))
+
+
+PROJECT_CODES = _load_project_codes()
 SECRET_PATTERNS = {
     "aws-key": re.compile(r"AKIA[0-9A-Z]{16}"),
     "aws-secret": re.compile(r"(?i)aws[_-]?secret[_-]?access[_-]?key\s*[:=]\s*['\"]?[A-Za-z0-9/+]{40}"),
@@ -164,7 +199,7 @@ def targets() -> list[str]:
         out = []
         for a in given:
             try:
-                rel = str(Path(a).resolve().relative_to(ROOT))
+                rel = Path(a).resolve().relative_to(ROOT).as_posix()
             except ValueError:
                 print(f"skip (outside repo): {a}", file=sys.stderr)
                 continue
@@ -182,7 +217,7 @@ def secret_targets() -> list[str]:
         out = []
         for a in given:
             try:
-                rel = str(Path(a).resolve().relative_to(ROOT))
+                rel = Path(a).resolve().relative_to(ROOT).as_posix()
             except ValueError:
                 continue
             if rel.endswith(".md") and not rel.startswith(SKIP_PREFIXES) and (ROOT / rel).is_file():
@@ -272,6 +307,15 @@ def _norm(s: str) -> str:
     return s.translate(HYPHENS)
 
 
+def _squash(s: str) -> str:
+    """Already-normalised text as the term scans compare it: emphasis stripped, lower case."""
+    return s.replace("*", "").replace("_", "").lower()
+
+
+def _fold(s: str) -> str:
+    return _squash(_norm(s))
+
+
 def _strip_code(s: str) -> str:
     s = re.sub(r"<!--.*?-->", " ", s, flags=re.S)   # HTML comments are not links
     s = re.sub(r"```.*?```", " ", s, flags=re.S)
@@ -395,13 +439,41 @@ def _folder(p: str) -> str:
             warnings.append(("index-incomplete", idx, f"does not link {_d}"))
 
 
-# --- secret pre-pass: a key/token must never be committed to ANY tracked doc ---
+TERM_NEEDLES = [(t, n) for t in FORBIDDEN_TERMS if (n := _fold(t))]
+def _code_text(normed: str) -> str:
+    """Like _squash, but '_' separates words: a code in `acme01_log.md` must still stand alone."""
+    return normed.replace("*", "").replace("_", " ").lower()
+
+
+CODE_NEEDLES = [(c, n, re.compile(rf"(?<![a-z0-9]){re.escape(n)}(?![a-z0-9])"))
+                for c in PROJECT_CODES if (n := _code_text(_norm(c)).strip())]
+
+
+def _term_errors(rel: str, scan: str) -> list[tuple[str, str, str]]:
+    return [("gate-leak:term", rel, f"forbidden term {_redact(t)}") for t, n in TERM_NEEDLES if n in scan]
+
+
+# --- text pre-pass over ANY tracked doc (frontmatter or not): secrets, and the path-keyed leak
+# scans (terms under term_scan_prefixes, project codes outside their own folder) ---
 if GATE_ON:
     for _sf in secret_targets():
         _stext = _norm((ROOT / _sf).read_text(encoding="utf-8-sig", errors="replace"))
         for _snm, _spat in SECRET_PATTERNS.items():
             for _smm in _spat.finditer(_stext):
                 errors.append((f"gate-secret:{_snm}", _sf, f"match {_redact(_smm.group(0))}"))
+        _scoped = _sf.startswith(TERM_SCAN_PREFIXES)
+        if not (_scoped or CODE_NEEDLES):
+            continue
+        if _scoped:
+            errors += _term_errors(_sf, _squash(_stext))
+        _rest = _sf[len(PROJECT_PREFIX):] if PROJECT_PREFIX and _sf.startswith(PROJECT_PREFIX) else None
+        if _rest is not None and "/" not in _rest:
+            continue                              # a hub file such as projects/_index.md lists every code
+        _own = _rest.split("/", 1)[0] if _rest else None
+        _scan = _code_text(_stext)
+        for _c, _n, _rx in CODE_NEEDLES:
+            if _c != _own and _n in _scan and _rx.search(_scan):   # substring first: the regex is slow
+                errors.append(("gate-leak:code", _sf, f"project code {_redact(_c)} outside its own folder"))
 
 _target_list = targets()
 for rel in _target_list:
@@ -485,12 +557,10 @@ def _folder(p: str) -> str:
         served = fm.get("status") == SERVED_STATUS
         customer = fm.get("audience") == CUSTOMER_AUDIENCE
         if customer:
-            scan = _norm(text).replace("*", "").replace("_", "").lower()
-            for _term in FORBIDDEN_TERMS:
-                needle = _norm(_term).replace("*", "").replace("_", "").lower()
-                if needle and needle in scan:
-                    errors.append(("gate-leak:term", rel, f"forbidden term {_redact(_term)}"))
-            for _mm in EMAIL.finditer(_norm(text)):
+            normed = _norm(text)
+            if not rel.startswith(TERM_SCAN_PREFIXES):   # else the pre-pass already scanned it
+                errors += _term_errors(rel, _squash(normed))
+            for _mm in EMAIL.finditer(normed):
                 if _mm.group(0).rsplit("@", 1)[-1].lower() not in EMAIL_ALLOWLIST:
                     errors.append(("gate-leak:email", rel, f"external email {_redact(_mm.group(0))}"))
         if served:

tests/gate_smoke.sh

@@ -162,4 +162,61 @@ printf -- '---\ntype: note\ntitle: Left\ntimestamp: 2026-08-26\nauthor: X\nstatu
 km validate --root "$T" "$T/inbox/left.md" 2>&1 | grep -q "superseded should not set 'contribution'" \
   && ok "validate warns on a superseded doc still carrying the marker" || no "validate warns on a superseded doc still carrying the marker"
 
+# Term scope: term_scan_prefixes scans a folder's docs whatever their audience; others stay as before
+gate(){ printf 'meta: {profile: smoke}\ngate:\n  enabled: true\n%b' "$1" > "$T/schema.local.yaml"; }
+errs(){ km validate --root "$T" "$@" 2>&1; }
+gate '  forbidden_terms_file: .gate-terms.txt\n  term_scan_prefixes: [bms]\n'
+mkdir -p "$T/bmsx"
+acc "$T/bms/ts1.md" internal "Kunde PROJECT-BLUEBIRD intern"
+V "$T/bms/ts1.md" && no "term_scan_prefixes: internal doc in a scanned folder blocked" || ok "term_scan_prefixes: internal doc in a scanned folder blocked"
+acc "$T/topics/ts2.md" internal "Kunde PROJECT-BLUEBIRD intern"
+V "$T/topics/ts2.md" && ok "term_scan_prefixes: an unscanned folder still passes" || no "term_scan_prefixes: an unscanned folder still passes"
+acc "$T/bmsx/ts3.md" internal "Kunde PROJECT-BLUEBIRD intern"
+V "$T/bmsx/ts3.md" && ok "term_scan_prefixes: 'bms' does not match 'bmsx/'" || no "term_scan_prefixes: 'bms' does not match 'bmsx/'"
+printf '# bms\nKunde PROJECT-BLUEBIRD\n' > "$T/bms/README.md"
+V "$T/bms/README.md" && no "term_scan_prefixes: an exempt file in the folder is scanned too" || ok "term_scan_prefixes: an exempt file in the folder is scanned too"
+acc "$T/bms/ts4.md" customer "Kunde PROJECT-BLUEBIRD extern"
+[ "$(errs "$T/bms/ts4.md" | grep -c 'forbidden term')" = 1 ] && ok "term_scan_prefixes: a customer doc there is reported once" || no "term_scan_prefixes: a customer doc there is reported once"
+rm -f "$T/bms/README.md"
+
+# Project codes: tracked subfolder names under project_prefix must not appear outside their folder
+git -C "$T" init -q
+mkdir -p "$T/projects/acme01" "$T/projects/zeta02" "$T/projects/cra" "$T/projects/ghost09"
+acc "$T/projects/acme01/pc2.md" internal "acme01 status note"
+acc "$T/projects/zeta02/pc3.md" internal "same fault as acme01"
+acc "$T/projects/cra/pc6.md" internal "cra reading notes"
+printf -- '---\ndescription: projects\n---\n# projects\n\n## Documents\n\n- [acme01](acme01/_index.md)\n' > "$T/projects/_index.md"
+git -C "$T" add projects                     # ghost09 has no tracked file: not a code
+gate '  project_prefix: projects/\n'
+acc "$T/bms/pc1.md" internal "found on the ACME01 bench"
+V "$T/bms/pc1.md" && no "project_prefix: a code in a neutral doc blocked" || ok "project_prefix: a code in a neutral doc blocked"
+V "$T/projects/acme01/pc2.md" && ok "project_prefix: a code in its own folder passes" || no "project_prefix: a code in its own folder passes"
+V "$T/projects/zeta02/pc3.md" && no "project_prefix: another project's code blocked" || ok "project_prefix: another project's code blocked"
+V "$T/projects/_index.md" && ok "project_prefix: the hub index listing every code passes" || no "project_prefix: the hub index listing every code passes"
+printf -- '---\ndescription: b\n---\n# bms\nsee acme01\n' > "$T/bms/_index.md"
+V "$T/bms/_index.md" && no "project_prefix: a code in another folder's _index.md blocked" || ok "project_prefix: a code in another folder's _index.md blocked"
+rm -f "$T/bms/_index.md"
+acc "$T/bms/pc7.md" internal "see [log](ACME01_bench_log.md)"
+V "$T/bms/pc7.md" && no "project_prefix: a code joined by '_' in a file name blocked" || ok "project_prefix: a code joined by '_' in a file name blocked"
+acc "$T/bms/pc8.md" internal "run log_acme01 again"
+V "$T/bms/pc8.md" && no "project_prefix: a code after '_' blocked" || ok "project_prefix: a code after '_' blocked"
+printf -- '---\ntype: concept\ntitle: t\ntimestamp: 2026-08-26\nauthor: X\nstatus: review\naudience: internal\ntags: [t]\nrelated: [projects/acme01/pc2.md]\n---\nneutral body\n' > "$T/bms/pc9.md"
+V "$T/bms/pc9.md" && no "project_prefix: a related: path into a project folder counts" || ok "project_prefix: a related: path into a project folder counts"
+acc "$T/bms/pc4.md" internal "part acme012 and xacme01 and ghost09 are not codes"
+V "$T/bms/pc4.md" && ok "project_prefix: whole tracked codes only" || no "project_prefix: whole tracked codes only"
+errs "$T/bms/pc1.md" | grep -q "acme01" && no "project_prefix: the code is redacted in the report" || ok "project_prefix: the code is redacted in the report"
+acc "$T/bms/pc5.md" internal "relevant under the CRA"
+V "$T/bms/pc5.md" && no "project_prefix: without a pattern every subfolder is a code" || ok "project_prefix: without a pattern every subfolder is a code"
+gate '  project_prefix: projects/\n  project_code_pattern: "[a-z]+[0-9]+"\n'
+V "$T/bms/pc5.md" && ok "project_code_pattern: a topic folder is not a code" || no "project_code_pattern: a topic folder is not a code"
+V "$T/bms/pc1.md" && no "project_code_pattern: a matching code is still blocked" || ok "project_code_pattern: a matching code is still blocked"
+gate '  project_prefix: projects/\n  project_code_pattern: "[a-z+"\n'
+errs "$T/bms/pc1.md" | grep -q "project_code_pattern is not a valid regex" && ok "project_code_pattern: a bad regex fails fast" || no "project_code_pattern: a bad regex fails fast"
+gate '  project_prefix: [projects]\n'
+errs "$T/bms/pc1.md" | grep -q "gate.project_prefix must be a string" && ok "project_prefix: a non-string fails fast" || no "project_prefix: a non-string fails fast"
+gate '  term_scan_prefixes: bms\n'
+errs "$T/bms/pc1.md" | grep -q "gate.term_scan_prefixes must be a list" && ok "term_scan_prefixes: a non-list fails fast" || no "term_scan_prefixes: a non-list fails fast"
+printf '%s' "$BASE_LOCAL" > "$T/schema.local.yaml"
+rm -rf "$T/projects" "$T/bmsx" "$T/topics/ts2.md" "$T/.git"
+
 summary smoke || exit 1