Skip to content

Audit progress signal misleads + invalid-id handling is brittle (feedback from a real open-audit sweep) #105

Description

@andy-esch

Feedback gathered while running a full open-audit sweep (triage every open finding into a task, then close fully-triaged audits) against a real planning tree. Version: tskflwctl v0.14.1-0.20260707002012-89afc6655ae2+dirty. Bundled into one issue for context; happy to split into separate ones.

# Priority Area One-liner
P1 High audit show / audit list Progress number contradicts its own bar and hides triage state
P2 High id validation Non-Crockford id → wrong error, aborts the whole command, no repair path
P3 Medium audit lint Legacy finding-status debt has no --fix; was masked by P2
P4 Low task new --json created.id is the slug; real minted id only in created.path
P5 Low lifecycle No hard-delete for a mistaken/duplicate create

P1 — The audit progress signal contradicts itself and hides triage state

The N% in audit show / audit list counts only fixed/landed, but the bar glyphs encode a 3-level gradient ( fixed, in-progress, superseded). So the number and the bar disagree in the same widget:

findings: ▒▒▒▒▒▒▒▒▒▒ 0%  0/5   # all 5 superseded (fully offloaded to tasks) — reads as "0% done"
findings: ▓▓▓▓▓▒▒▒▒▒ 0%  0/4   # 2 in-progress + 2 superseded — bar says "half", number says "0%"
findings: ██████████ 100% 2/2  # all fixed

Consequences:

  • An audit that is 100% triaged (0 open, every finding superseded to a real task → correctly closeable) is indistinguishable at the number from an audit nobody has touched — both show 0%.
  • This directly caused a false alarm: "0%" was read as "nothing handled / the counter missed findings," when in fact every finding was parsed and superseded.

Suggested fixes (any one helps):

  • Label what the number measures (e.g. 0% resolved), or
  • Show a stacked count instead of a single % (e.g. 2 superseded · 2 in-progress · 0 fixed), or
  • Make the headline number track the same gradient the bar already computes.
  • Reconcile with the audit list open column so there's one consistent notion of "progress."

P2 — Invalid-id handling is brittle and misdiagnosed

Some audit files were generated with a non-Crockford char in the id (an l; Crockford base32 excludes i/l/o/u). Result:

  1. Wrong error message. not an entity file: "<id>-<date>-<area>.md" has no leading id — move it to meta/ or delete it. The file has a leading id — it's just non-canonical. Correct message: id "<id>" contains non-Crockford-base32 char 'l' (i/l/o/u disallowed).
  2. One bad file aborts the whole command. audit list, audit findings, and audit lint print the good rows and then exit 11 — so you can neither trust the exit code nor get clean output, even to query an unrelated audit by slug.
  3. No repair path. lint --fix "backfills missing ids" but cannot repair an invalid id — chicken-and-egg: the file isn't discoverable, so the fixer never sees it. (Removing the id + renaming to a bare date-area slug doesn't help either — a no-id filename is hard-rejected the same way.)

The manual fix that worked: substitute the illegal char with its Crockford decode-alias (l1, i1, o0) in both filename and id: frontmatter — same decoded value, canonical char, preserves sort order.

Suggested fixes:

  • Precise error naming the offending char / rule.
  • Skip-with-warning instead of aborting the whole command (or gate the abort behind --strict).
  • lint --fix should repair/re-mint an invalid id, not only backfill a missing one.
  • Validate ids at write time so a bad id can never persist (root cause: a generator used a different base32 alphabet than the CLI).

P3 — Legacy finding-status debt has no migration path (and was masked by P2)

  • audit lint enforces a strict vocabulary (deferred/fixed/in-progress/landed/open/superseded/wontfix), but older audits use emoji (✅/⏳/⛔) or words (tracked, declined), and the oldest predate the Status: convention entirely.
  • lint --fix repairs tasks/epics but there is no audit lint --fix to normalize finding statuses (strip emoji, map declined→wontfix / tracked→superseded, backfill missing).
  • The P2 abort hid this: whole-tree audit lint never completed until the invalid-id files were fixed, so ~10 audits' worth of status debt was invisible.
  • schema audit documents the Status-line format but not the allowed vocabulary — I had to learn it from lint errors.

Suggested fixes:

  • Add audit lint --fix covering the emoji/legacy-word normalizations.
  • List the finding-status vocabulary in schema audit.

P4 — task new --json output is confusing

The created object is under created, and created.id is the slug; the real minted id is only embedded in created.path (tasks/<minted-id>-<slug>.md). Scripting created.id expecting the id yields the slug.

Suggested fix:

  • Expose the minted id as its own top-level field distinct from slug.

P5 — No hard-delete for a mistaken create

A duplicate task new can only be undone with deprecate (leaves a tombstone) or by rm-ing the file (bypasses the CLI, against the "drive the lifecycle via tskflwctl" principle).

Suggested fix:

  • A task rm (and audit rm) for never-committed mistakes.

Working well (so this isn't all complaints)

--body-file, --dry-run, multi-slug audit close, schema <kind>, -c column selection, the finding→task supersede model, and the status-grouped tree in audit show are all great. The bar's glyph gradient is a genuinely good idea — it just needs a headline number that agrees with it (P1).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions