Retrieval-grounded question answering for public agencies, as a reference implementation: it answers only from a corpus the operator supplies, shows its sources on every answer, and when no source clears the relevance threshold it refuses plainly and points to a human — no guessing, ever. A cairn marks a verified trail; where there are no stones, there is no trail.
Status: released. v0.2.0 is tagged, published to PyPI
(pip install cairn-assistant) and GHCR
(docker pull ghcr.io/chelseakr/cairn), and the branch-protection ruleset
that makes the audit job a gate rather than a report is applied and
verified — see the Standards Conformance table below. Every capability in
the specification is implemented: ingest with idempotent indexing, grounded
answers with citations, refusal as a first-class outcome, an operator
explain mode that diagnoses a bad answer to the right stage, three languages
including right-to-left, an accessible chat interface, and a fail-closed CI
audit gate against a pinned external auditor — run against the committed
evidence and, separately, against the running server. 783 tests plus
63 browser behaviour checks, standard library only, offline.
This is a demonstration of correct behavior, not a production service.
The claim is that Cairn cannot produce an answer with no source behind it. That
claim was false for one configuration. Config(max_passages=0) — a plausible
reading of "no limit" — returned an answer whose kind was "grounded" and
whose to_payload()["grounded"] was true, with no sources and no text:
composition sliced the accepted passages away while the trace still said
passages had been accepted, so the refusal branch never ran. cairn.toml could
not reach it, because the bounds were checked when loading a file and not when
building the object — and this is a reference implementation, so the caller who
skips the file is the whole audience. Fixed in
abd54ab by moving the bounds onto the type;
TestNoConfigurationCanEmitAnUnsourcedAnswer in
tests/test_answering.py fails without it.
That is the kind of thing this repository is for. The worklog is a list of others.
Start here: the walkthrough — every command in a console fence on that page is executed by the test suite, so its output is what you will get. What cannot be executed is fenced as text and says so: serve,
which never returns; the audit, which needs the network the first time; and
the stale-index refusal, which needs a corpus document edited underneath it.
- Built from a functional specification (idea-level requirements only — no code, wording, or constants were supplied). All naming, architecture, expression, and constants originate in this repository; see DESIGN.md for the decisions and their reasons, and WORKLOG.md for the dated session log.
- Started 2026-08-15. The git history is the build history: incremental, honestly dated, beginning with the design document.
- Implemented with AI agents working from the specification.
- License: Apache-2.0.
Requires Python 3.11+. The runtime is standard-library only, so from a clean checkout — with no network, no API key, no external model — the demo runs directly:
$ python3 -m cairn index
Indexed 40 passages from 10 documents (10 marked synthetic) and 1 structured table in 3 languages [ar, en, es] -> .cairn/index.json
Corpus fingerprint: b757efed04dd (corpus/demo)
$ python3 -m cairn ask "How much unpaid rent does the housing relief grant cover?"
## How much the grant covers
The grant covers up to $3,500 of unpaid rent. ...
Sources:
[1] Harbor Housing Relief Grant (housing-relief-en#2)
$ python3 -m cairn ask "Can you help me renew my drivers license?"
I don't have a source for that, so I can't help with this question. None of
the official documents this assistant is allowed to answer from cover it, and
I won't guess.
For help from a person, contact ...Answers are extractive — composed verbatim from the retrieved passages — so every fact in an answer, numbers included, appears character-for-character in a cited source. Identical corpus + configuration + question always yields identical output. Refusals carry no sources and exit 0: refusing without evidence is correct behavior, not an error.
pip install -e . additionally gives you the cairn console command;
--json on ask emits a machine-readable record.
ask --explain prints an operator trace above the answer: every candidate
passage with its score and its accept/reject verdict at the threshold, then a
verdict for each stage that could have gone wrong.
$ python3 -m cairn ask --explain "What vaccinations does my dog need?"
Threshold: 0.165 (retrieval.threshold)
...
Attempt 1 (restricted to 'en'): 16 passages scored, 24 excluded, 4 candidates
question terms: does, dog, need, vacci, what
in no passage: does, dog, vacci
1 0.069 reject grocery-allowance-en#3 [en] Fresh Start Grocery Allowance
...
matched 1/5: need
...
Stage 1 - retrieval: FAILED (below-threshold)
4 candidates were scored and none cleared the 0.165 threshold. The best,
grocery-allowance-en#3, scored 0.069 and was short by 0.096 on 1 of 5
question terms (need). No passage searched contained does, dog, vacci —
that part of the question is a corpus coverage gap, not a threshold setting.
Stage 2 - answer: NOT REACHED (no-evidence)
The answer stage was handed no passages, so it refused. ...
Verdict: NOT GROUNDED - refusal, no sources.
Diagnose at: retrieval.The point is the last line. A wrong answer whose retrieval stage passed is a
different bug from one whose retrieval stage failed, and the trace says which
you have — including the case where the right passage cleared the threshold and
was then dropped from the answer by retrieval.max_passages. The term lines
say why a score is what it is: which of the question's words each passage
actually held, which the corpus has never seen (a coverage gap), and which
were suppressed as too common (a scorer decision). Add --json for the same
trace machine-readably. Explain mode never changes the answer.
$ python3 -m cairn ask "Cuanto cubre la subvencion de alivio de vivienda?"
## Cuánto cubre la subvención
La subvención cubre hasta $3,500 de alquiler no pagado. ...
Fuentes:
[1] Subvención de Alivio de Vivienda de Harbor (housing-relief-es#2)A question is answered in the language it was asked in, from sources in that
language, and --lang states the language outright when you would rather not
rely on detection. Arabic is right-to-left in the way that matters: direction
comes from the language code, and Latin runs inside an Arabic line — passage
ids, phone numbers — are wrapped in Unicode bidi isolates so a terminal or a
browser does not reorder them.
A fourth interface language, French (fr), ships too — a full messages.py
catalogue and language-table entry, selectable with --lang fr or the web
selector — with no French corpus content bundled. That is not an
oversight; it is the same "translated interface outruns translated
documents" reality the GoPass example below demonstrates for a single
document, played out for a whole language: a French question either falls
back across languages the way the GoPass example does, or refuses, in
French, exactly as an English or Spanish question with no matching source
would. See docs/I18N.md for what shipping an interface
language does and does not require.
Corpus coverage is deliberately uneven, because real agencies' translations lag. Ask in Spanish about a document that exists only in English and Cairn says so, in Spanish, and then quotes the English exactly as published:
$ python3 -m cairn ask --lang es "How much does the GoPass cost per year?"
La única fuente que tengo para esto está escrita en otro idioma (English).
Se cita a continuación tal como fue publicada.
...It does not translate the source. A translated policy amount is an unsourced
policy amount. Set [language] cross_language_fallback = false to refuse
instead.
How far that reaches, measured rather than promised. The fallback is
lexical, so it fires only when the question contains words the document
contains — and across languages the only words that survive are proper nouns
and numbers. ما هي بطاقة GoPass؟ is answered from the English document, in
Arabic, with the English quoted untranslated: the fallback crosses scripts
perfectly well. ¿Cuánto cuesta el GoPass por año? refuses, in the same
script. What decides it is whether the program is named, which puts the
limitation on the person least likely to know its official name.
There is a second edge behind the first, and this page used to state the
measurement without it. GoPass كم سعرها؟ — the same crossing, asking what
the pass costs — is also answered, and the passage it is answered from is
the document's opening sentence, which contains no price. "GoPass" is the only
term that survives the crossing, all four passages of that document contain
it, and the ranking among them is then decided by length. Crossing the script
is not the same as answering the question, and calling the first one a success
without saying so is the kind of thing this repository is supposed to catch.
DESIGN.md carries the four measurements, this
correction, and why the available bridge — letting a document declare its name
in another language — is refused.
That path is in the audited evidence now, as ck-027. It had never been:
twenty-six recorded answers, none of them cross-language, so no audit report
this repository has ever published said anything about the behaviour described
above — which is how Answer.cited_text came to drop the notice for a whole
milestone with every check green. The
write-up lists every score
the one new item moved, including multilingual scoring it zero.
The bundled corpus under corpus/demo/ is entirely
fictional: an invented agency, invented programs, invented amounts and
deadlines, in English, Spanish, and Arabic, each file marked synthetic: true. See
its README. Point [corpus] path in
cairn.toml at your own directory of front-matter markdown
documents to use real content — swapping the corpus is a config change, never
a code change. For what real content actually looks like once imported, see
corpus/pilot-usagov/ — six real, currently-published
usa.gov pages (synthetic: false, sourced and dated in each file's front
matter) and docs/pilot-usagov.md, the measured
write-up of what importing and calibrating against them actually found.
The next pilot — federal program owners, California agencies, and a county,
assembled per county and asked questions nobody wrote for Cairn — is under
way at docs/pilot-ca.md, with its decision gate
written down before its first number.
$ python3 -m cairn serve
cairn: serving the chat interface on http://127.0.0.1:8765/ (ctrl-c to stop)
Localhost only, no external resource of any kind, and a content security
policy of default-src 'none' so the browser enforces that rather than this
README claiming it. It targets WCAG 2.2 AA as behavior, not as attributes: a
skip link that lands in the question box, a transcript announced politely that
never steals focus, a separate assertive channel that carries errors and
nothing else, a labelled input with the Enter/Shift-Enter behavior written
under it, a permanent disclosure with no dismiss control, a language selector
that mirrors the whole layout for Arabic, a visible focus ring at every stop in
both presentations, and light and dark presentations whose every colour pair
passes AA. It answers
without JavaScript, too — the form posts and the server renders.
That default — loopback, no auth, no rate limit — is what SECURITY.md
documents as the boundary. An operator who needs to reach it from more than
one machine can opt into bearer-token auth and a per-client rate limit
(--auth-token/CAIRN_AUTH_TOKEN, --rate-limit) without changing anything
about the default path; see docs/deployment.md for
what that does and does not cover. An agency that wants Cairn reachable
from its own site — an iframe, or a fetch to the JSON API — can opt into
that too (--allow-embed, --cors-origin), independently of each other and
of the deployment flags above; see
docs/embedding.md. A third opt-in, --refusal-stats,
turns refusals into an aggregate, local signal for finding corpus gaps —
language and reason only, never the question — see
docs/refusal-analytics.md. A fourth,
--followup-store, adds a real "request a follow-up" action to a
refusal — the asker's own contact info, and their question only if they
separately choose to include it — a real handoff past the static contact
string a refusal has always named; see
docs/followup.md.
No person has driven this page with a screen reader. The browser checks
verify the plumbing one depends on — the roles, the politeness settings, that
an announcement fires and focus does not move, that the assertive channel
stays quiet on success — and axe-core checks the rule set. None of that is the
same as a VoiceOver or NVDA session, that session has not happened, and no
automated check here should be read as standing in for it.
docs/screen-reader-test-script.md is a
structured, task-by-task script for whoever runs that session, so it takes an
hour rather than a day of improvising — the script itself is not that session.
Checked in two layers: tests/test_ui.py for markup, semantics, and computed
contrast, offline with no dependencies; and tests/browser/ for the behaviors
only a browser can confirm, including axe-core's WCAG 2.2 AA rule set in
light, dark, and right-to-left.
$ cd tests/browser && npm ci && npm run check
63/63 behaviour checks passedEverything tunable lives in cairn.toml, which ships with every
default written out: corpus and index locations, the relevance threshold
(bounded [0, 1]; calibrated against the demo corpus — re-check it against
probe questions when you swap corpora), how many passages compose an answer,
and the human-contact line refusals point to.
$ python3 -m unittest discover -s tests # zero third-party dependencies
$ ruff check . # lint (dev extra: pip install -e ".[dev]")The test suite covers ingestion idempotency (byte-identical re-index), grounded
answering with citation validity and numeric-fact traceability, refusal
behavior (no sources, no corpus leakage, countable in JSON output), stage
diagnosis in explain mode, multilingual behavior including script-aware
tokenizing and bidi isolation, output determinism, and the CLI contract — and
it re-measures the retrieval threshold calibration on every run rather than
trusting a comment. It also runs every command in
the walkthrough and fails if the recorded output has drifted —
and every cairn command on this page too, under a looser rule that tolerates
the wrapping and the ... but not a word the command never printed. That
second one was added after this README was found showing a two-source answer
to a question that cites one, and a refusal in wording the engine stopped
using.
The browser checks under tests/browser/ need Node and Chromium and are
deliberately not part of this path: install, lint and test work with no Node,
no browser, and no network.
Cairn does not grade itself. cairn record asks the real engine a committed
set of questions and writes what came back as an evidence bundle; the merge
gate hands that bundle to Plumbline,
a separate project, pinned to an exact commit in
plumbline.pin.
$ python3 -m cairn record # evidence, produced by the engine, not by hand
Recorded 29 items (22 answers, 7 refusals) in 3 languages [ar, en, es] -> plumbline/bundle
Bundle sha256: b7a28017910ba7e662ebfa55f0e050e2059df4c0bc80875ef694809fb72cc900
$ ./plumbline-gate.sh # the same command CI runs
GATE: PASS — target cairn-demo, dataset b7a28017910b, run ...
all 14 suites passed:
...
multilingual score 0.9655 floor 0.95 PASS n=29 ci 0.828-0.994 mde 0.134
passage_attribution score 0.9444 floor 0.90 PASS n=18 ci 0.742-0.990 mde 0.214 3 unverifiable
...
$ python3 audit_guard.py # and the check the gate cannot make on itself
GUARD: PASS — cairn-demo, run ..., against baseline 62d02d167796e3a5
declared gaps (1 suite not scored at all):
conversational_integrity: no item in the recorded bundle carries multi-turn
conversation data; this release has no feature for the suite to grade
floors that are not the harness's own (6 suites, each with a recorded reason):
accuracy: 0.35, LOOSER than the default 0.75
...
suites that could not check everything they were handed:
passage_attribution: scored 18 of 21 eligible (no_distractor 3); unverifiable
items are excluded, never passed
no suite moved against the committed baseline.
The dataset id is the first twelve characters of the bundle's own SHA-256, and
a test holds this page and docs/demo.md to it. The run id is elided because
it is a hash of the evidence, the judge configuration, the enabled floors
and the baseline, so it moves whenever any of those do — this block carried
a stale one (958f5afd…) from before the baseline was last regenerated, in a
fence nothing executes.
The gate blocks a merge today, and this is the sentence that says so. The
audit job runs on every pull request and writes a verdict; since 2026-08-22
a pull request cannot be merged while that verdict is red, because the
committed ruleset at
.github/rulesets/main.json was applied —
whether a check can block a merge is a repository setting on GitHub's side,
so a file could never grant itself that power, and this is the sentence
recording that someone with admin rights did. What it costs and the
two-part verification that it actually blocks — a real pull request needing
every check green, and a separate one with a check deliberately broken,
refused a merge outright — are in
its README. A green tick here now means the
checks had to pass, not only that they ran. A check that could have blocked
a merge and did not was the failure this whole project is about; it would
have been a poor joke to leave one sitting in it.
The harness is resolved at run time and verified to be at the pinned commit. It is in no import and no dependency list, so Cairn's install, lint and test path works with it completely unreachable — and CI proves that on every run, in the same job that proves the gate fails in that condition. A skipped check and a passed check are the same green tick on a pull request, so an unresolvable auditor has to be red: a gate that could not run is not a gate that passed.
Running it for real found four things worth fixing, including two languages disagreeing about the same policy number and a Spanish refusal a standard detector read as an answer. They are written up in DESIGN.md, along with the two known limits that are named rather than tuned away.
A floor is a minimum, not a ratchet. accuracy could fall from 0.4123 to
0.36 above a floor of 0.35 and the gate would be green the whole way down, so
the pin also names a committed baseline — one line per suite, distilled by the
harness from a run we were happy with — and audit_guard.py runs straight
after the gate and fails on any suite whose score no longer matches it, any
floor that was lowered, and any suite that stopped being scored. It can be
silenced by regenerating the baseline, which is the point: a move then arrives
as "score": 0.9630 becoming "score": 0.36 in a reviewed diff, rather than
as nothing at all.
A score that went up fails too, and the guard still will not adopt it.
An improvement nobody records is a bar nobody raised: the committed number
stays low, and every point of the improvement can be given back later with the
comparison calling it unchanged. So a rise stops the build exactly as a fall
does — labelled IMPROVEMENT rather than REGRESSION, because they do not
mean the same thing — and a person decides, in a commit, whether the better
number becomes the new bar. Nothing here ratchets by itself in either
direction, and a test pins that the guard never writes to the baseline.
A suite that was not scored, and now is. multilingual checks that a
response came back in the language it was asked in. It sat disabled because
the harness pinned at the time recognised English and Spanish only, while a
third of Cairn's evidence is Arabic — and it called an unrecognised language a
configuration error rather than scoring evidence it could not read, which is
right. Dropping the Arabic to make the suite runnable would have hidden the
language the interface exists to prove it supports. Cairn consumes Plumbline
at a pin and pushes nothing to it, so all Cairn could do was refuse to let the
gap read as coverage: declared in plumbline/target.toml, printed by the
guard beside every gate result, held there by a test, and written out with the
exact fix. Plumbline has since shipped Arabic recognition by script; bumping
the pin and enabling the suite scored it 1.0000 across all 26 items. The story
is kept in DESIGN.md,
because a consumer finding a real gap in its own auditor and saying so until
it got fixed is the interlock working.
And a second gap, the same way. One item, ck-022, is answered from the
housing document's deadline paragraph instead of the one with the amount in
it — and thirteen suites passed it, each of them correctly. The answer is
grounded, in a real passage; the citation resolves; the cited passage supports
the answer completely, because that is where the answer came from. Nothing
could say right document, wrong paragraph. Cairn wrote the case up, named
what a suite would need, and Plumbline built passage_attribution. The
evidence side of it is authored: plumbline/questions.toml now declares which
passage answers each question, because only a person who has read the question
and the corpus can say that, and cairn record refuses a question set where
an answer item does not. The suite scores 0.9412 over 17 items and fails
ck-022 by name — and is more precise than the write-up was, reporting it as
a retrieval failure, because the right passage never cleared the threshold
for composition to choose it. The behaviour has not changed; it is scored now
instead of only documented.
Everything above grades a bundle. A bundle is bytes on disk; the thing it is a
recording of is code that changes. So the same questions also get asked over
HTTP, against a running cairn serve, by the pinned harness's own live-target
recorder:
$ ./plumbline-gate.sh # resolves the harness; the only thing that does
$ ./plumbline-live.sh
PLUMBLINE LIVE: serving cairn on 127.0.0.1:8766
recorded: 27 responses
verdict: PASS
LIVE: MATCH — http://127.0.0.1:8766/ask, recorded 2026-08-16T…
27 answers over HTTP, byte-identical to the recorded evidence the gate grades.
the audited interface snapshot is the page being served.Wiring it up found something on the first run. Pointed at the served
answer text, citation_validity scored 0.0000 — on a system the offline audit
scores 1.0000. The inline citation markers existed only inside cairn record:
/ask returned the sources as structured metadata and the answer text with
none in it, so the audit's perfect citation score described a string no
consumer of the served interface could get, and any plain-text client got an
answer with no sources. Answer.cited_text is one definition of that shape
now, used by the recorder and returned by the API. The bundle came out
byte-identical; only who can produce it changed.
This is an addition, not the gate. The merge gate stays audit: offline,
deterministic, grading committed bytes, with no socket anywhere in it.
plumbline.pin does not name the live config, ./plumbline-live.sh
deliberately cannot resolve the harness — it uses the checkout the gate
verified, so grading a running server is never the act that installs its own
auditor — and the drift check itself runs in the core test suite against a
loopback server with no harness and no network at all.
site/index.html is a committed static page holding a
refusal, the cross-language answer the audit scores as a failure, and the
committed baseline. It is served by GitHub Pages at
https://chelseakr.github.io/cairn/. Pages was enabled in repository
settings on 2026-08-16 (the first successful pages deploy is from that
day), and the URL was checked to return 200 on 2026-08-22. A sentence here
used to say the URL was a 404; it was true when written and stayed in the
README for six days after it stopped being true, which is the kind of drift
nothing in this repository can test for — whether a URL serves is a fact
about GitHub's side, not about a checkout.
Nothing on it is written by hand. site_build.py renders it from
plumbline/bundle and plumbline/baseline.json, and
tests/test_site.py holds it to them two ways: it
re-renders and diffs, which catches a hand edit and a rebuild that never
happened, and separately it parses the committed HTML and compares the text it
finds to the JSONL, which is the half that catches a generator printing
something friendlier than Cairn said. The deploy workflow uploads the file and
refuses to publish if the first check fails; both checks also run offline in
core, so a drift fails the pull request before it can reach a deploy.
CITATION.cff, which GitHub renders as a "Cite this
repository" panel. The version in it, in pyproject.toml, in
cairn.__version__ and in CHANGELOG.md are held together by a
test, because a version recorded in four places is a version that will
disagree with itself.
This repository is held to a shared set of portfolio engineering standards. Every standard gets a row whether it applies or not, and a row that is an obligation rather than a passing result says so. Three of these rows are gaps, and they are here because leaving them out is the failure mode the rest of this page argues against.
| Standard | State | Evidence |
|---|---|---|
| Responsible-Tech Framework | Applies — the harm this project is built against is a confident wrong answer to somebody asking a public agency a question that matters to them. The design answer is refusal: no passage over the threshold means no answer, and the refusal is countable in the JSON output rather than being a phrase in a log. | DESIGN.md "Core stance" and "What is still open", where every open item is anchored to a test that fails if the item stops being accurate in either direction. |
| Code Quality | Applies — and two limits are configured rather than enforced, deliberately. make verify is the local gate: uv lock --check, ruff, mypy, and the suite under coverage with an 85% branch floor against 92% measured. mypy runs in its default mode, not --strict, which reports 44 findings on this tree today (28 before six features landed in one session — see WORKLOG.md sessions 11-16). The complexity limit of 10 is configured and the C90 rule is not switched on, because eight functions are over it, up from four the same session. Both are named in pyproject.toml at the point of configuration, and closing either is a refactor rather than a setting — see the good first issue label for bounded pieces of both. |
Makefile, pyproject.toml, uv.lock, .python-version |
| Security & Supply-Chain | Applies | SECURITY.md names the private channel and three project-specific vulnerability classes, including an ungrounded grounded answer. .github/workflows/security.yml runs gitleaks over the full history, Semgrep, and pip-audit, automatically and on a schedule, with no path that turns a skip into a green check. Every uses: is pinned to a commit SHA with a version comment, and .github/dependabot.yml raises the pins weekly with a cooldown. The runtime has no third-party dependencies at all. |
| CI/CD | Applies. Nine jobs: core (install, lint, test, and a fail-closed drill proving the gate exits non-zero when the harness is unreachable, matrixed across Python versions on Linux), core-windows and core-macos (the same install/lint/test/demo/independence steps, one canary job per non-Linux OS — the stdlib-only claim is a claim about the package, not about Ubuntu), image and package (build the container image and the sdist/wheel release.yml publishes, and run each against the demo corpus, on every change rather than first at release time), interface, audit, live, and gauntlet (a second, independent adversarial-suite interlock — ChelseaKR/gauntlet, pinned the same way Plumbline is, grading prompt-injection, refusal and grounding cases audit's own suites do not). Every workflow declares a top-level least-privilege permissions: block. The audit job is marked required in branch protection — the committed ruleset (.github/rulesets/main.json) was applied 2026-08-22 and verified two ways: a real pull request (#22) needed all nine required checks green to merge, and a throwaway pull request with one of them deliberately broken was refused a merge outright by GitHub, not just left red (.github/rulesets/README.md has both). ruleset-check.yml re-checks weekly that an active ruleset still exists on the live repository and opens a tracking issue if it is ever silently removed. Published to both PyPI and GHCR — v0.2.0's GitHub Release triggered release.yml for the first time; pip install cairn-assistant and docker pull ghcr.io/chelseakr/cairn both resolve to something real. See docs/release.md for how the one-time PyPI trusted-publisher setup was done, and the real bug its first run found. |
.github/workflows/ci.yml, .github/workflows/release.yml, .github/workflows/ruleset-check.yml, .github/rulesets/main.json, Dockerfile, tests/test_container.py, tests/test_rulesets.py (which fails if the ruleset stops naming the jobs it protects) |
| Release & Versioning | Applies — the version in CITATION.cff, pyproject.toml, cairn.__version__ and CHANGELOG.md is held together by a test, currently 0.2.0. Two tagged releases exist: v0.1.0 (2026-08-16) and v0.2.0 (2026-08-22) — tagging and pushing tags is the maintainer's own action, not an agent's, per this repository's working rule; see docs/release.md. release.yml published v0.2.0's GitHub Release to both PyPI (trusted publishing, no stored token) and GHCR — the first real run of the publish half found and fixed a real bug in the workflow itself (an action pinned to a tag object's SHA rather than the commit it points to; docs/release.md has the story), re-run after the fix, and confirmed live on both registries rather than trusted from a green check alone. Not in place: signed tags. |
CHANGELOG.md, CITATION.cff, docs/release.md, tests/test_cli.py |
| Observability | Applies — this is a local tool with no service to instrument, so the observable surface is the evidence rather than telemetry. cairn record writes what the real engine answered, --explain attributes a bad answer to the stage that caused it, and the audit report and committed baseline make a score change visible in a diff. Nothing phones home and there is no analytics anywhere. |
plumbline/bundle, plumbline/baseline.json, audit_guard.py, site/index.html |
| Performance | Applies — the served page is one small static document with no external resource of any kind, and retrieval is lexical over a local index with no model call in the path. tests/test_performance.py, gated in make verify, budgets both: page weight exactly (deterministic — no timing, so the budget is tight against the ~21KB measured baseline) and demo-corpus query latency deliberately loosely (two orders of magnitude above the ~3.3ms measured, wide enough that CI runner noise cannot trip it while a real algorithmic regression still would). benchmark_index.py separately measures where query latency stops being "milliseconds" at larger-than-demo corpus scale — unbudgeted and not gated, because an absolute number at scale does not survive running somewhere else (see DESIGN.md, "Measured, not only asserted"). |
tests/test_performance.py, benchmark_index.py, cairn.toml bounding the retrieval work itself. |
| Accessibility | Applies — WCAG 2.2 AA as behaviour rather than attributes, checked in two layers: tests/test_ui.py for markup, semantics, and computed contrast offline, and tests/browser/ for what only a browser can confirm, including axe-core's WCAG 2.2 AA rule set in light, dark, and right-to-left. No person has driven this page with a screen reader, that session has not happened, and no automated check here stands in for it. |
tests/test_ui.py, tests/browser/, cairn/ui/contrast.py |
| Internationalization | Applies — four interface languages ship (en, es, ar, fr), one of them right to left, with script-aware tokenizing, bidi isolation, and a language selector that mirrors the whole layout. French ships with no bundled corpus content, deliberately — see "Four languages" above and docs/I18N.md, which declares the scope beyond these four: a corpus document may be in any language with no code change, an interface language is a messages.py catalogue plus three tests, and a right-to-left code beyond the interface set is one table entry — each tier's flip condition stated, not left implicit. |
cairn/language.py, cairn/messages.py, docs/I18N.md, tests/test_multilingual.py |
| AI Evaluation | N/A — there is no model. Retrieval is deterministic lexical scoring over a corpus the operator supplies, and answers are passages quoted verbatim rather than generated, so there is no prompt, no sampling, and nothing to evaluate as a model. | The runtime has zero dependencies, which makes the no-model claim mechanically checkable; tests/test_answering.py holds every answer to its source text. |
| Documentation | Applies — and the pages are tested, which is the part that matters. tests/test_docs.py executes every command block in docs/demo.md and holds its output byte for byte, and executes the README's blocks under a looser rule that still forbids showing a word the command never printed. |
This README, DESIGN.md, CONTRIBUTING.md, SECURITY.md, CHANGELOG.md, CITATION.cff, WORKLOG.md, docs/I18N.md, docs/authoring.md, docs/onboarding.md, docs/deployment.md, docs/embedding.md, docs/refusal-analytics.md, docs/followup.md, docs/compliance.md, docs/screen-reader-test-script.md, docs/release.md, docs/pilot-usagov.md, docs/pilot-ca.md, docs/pilot-ca-elicitation.md, and the ADR log at docs/adr/. |
| Quality & Metrics | Applies — the floors are measured rather than aspirational, and a floor that differs from the auditor's own default must carry a written reason that audit_guard.py enforces against the pinned harness's source. The guard also catches what a floor cannot: a score that moved without breaching one, in either direction. |
plumbline/target.toml, audit_guard.py, tests/test_audit_guard.py, and the 85% branch-coverage floor in pyproject.toml. |
| AI Development Measurement | Applies — no AI-development baseline is recorded in this repository, and no activity counter is tracked or gated. The gates that exist are outcome-side: make verify locally, and an external auditor grading recorded behaviour at merge. |
Makefile, .github/workflows/ci.yml |
| Incident Response | Applies — private reporting with a seven-day acknowledgement expectation, and a scope section that names what is and is not a report for a tool with no deployment. No incident has been recorded, so there is no docs/incidents/ directory yet. |
SECURITY.md, docs/compliance.md |
| Data Governance | Applies — the corpus belongs to the operator and never leaves their machine: there is no upload, no telemetry, no external resource on the served page, and a default-src 'none' policy so the browser enforces that rather than this README claiming it. The corpus shipped here is synthetic and the README says so where it is used. Two opt-in server features hold real data past that default — --refusal-stats (aggregate counts only) and --followup-store (real contact information, by explicit consent) — and neither has a built-in retention period; docs/compliance.md states that plainly for a records-retention review. |
"The demo corpus is synthetic" above, corpus/, cairn/server.py, docs/compliance.md |
Apache-2.0 — see LICENSE.