Newer
Older
openstack-caracal-dc-dc / docs / audit / grounding-audit-charter.md

PROMPT FOR CODE -- Project grounding audit and record-system restructure

0. Mission and framing

The operator's assessment, which the evidence supports: the project's record-keeping system has become the primary work generator. Sessions open by reconciling records, find discrepancies, file findings about the discrepancies, and thereby create more reconciliation surface. The last working day was spent in circles at the end of the Office1 stage without reaching a stable point on either goal (trustworthy records, or a repeatable stable environment). Details are getting lost. This audit exists to stop that structurally, not to fix the current crop of defects faster.

Measured evidence (verified 2026-07-18 against the pushed repo at 43b29d7):

  • docs/ holds 135 files; 95 are changelogs (31 filed on 2026-07-09 alone; sustained 6-14/day since).
  • Nine D-numbers ruled in four days (D-121..D-129). Several are operational choices, not architecture: D-126 (an SSH-forward option), D-127 (an autostart setting), D-128 (an operating model note), D-129 (a plugin list). The decision register is being used as a diary, which dilutes its authority and multiplies reconciliation surface.
  • docs/dc0-deploy-readiness.md carries 5/2/6 at line 37 and 5/0/6 at line 59 -- two contradictory plan counts, both live, in the deploy gate document.
  • The D-127 changelog asserts plan outcomes that are false as written (per the 2026-07-18 session's own findings). README pins tofu 1.12.3; the binary is 1.12.4.
  • docs/session-ledger.md is 1,125 lines and growing without a rotation rule.
  • Root cause pattern: status lives as hand-written copies in many surfaces (workflow-doc stage rows, ledger, changelogs, readiness doc, README, runbook gates), each rotting at its own rate. Every fix to one copy creates drift in another.

What is NOT broken, and must be preserved: fence discipline (validated OK), the adversarial-review process (ran 2026-07-16, R3-F01..F13, repairs landed), gated mutation discipline, honest retractions, refusal to fabricate. This session's epistemics are good. The audit repairs the SYSTEM, not the operator relationship or the discipline culture.

1. Ground rules (binding for the whole audit)

  • FULL FREEZE. No fixes -- not one typo, not one doc correction -- until Phase 5. No new D-numbers (audit findings use GA-NNN). No new changelogs. No module edits. No state writes. No cloud mutation. The 2026-07-18 issue list (D-130 candidates, platform-traps, doc-integrity items, env reconcile) is EVIDENCE for this audit, not a work queue. It gets re-verified and folded into the findings register.
    • Sole exemption: the two read-only verifications already identified (v0.9.8 volume-replace-under-running-domain on a throwaway domain; ignore_changes suppression check). They are audit evidence-gathering and are read-only / throwaway-scoped by construction.
  • Fresh session. This prompt is the entire charter of a NEW session. Do not resume the circling session; its context carries every half-fix as live state.
  • Model tiering: token budget is explicitly NOT a constraint. The operator has authorized maximum effort. Ground-truth author, adjudicator, and all fresh-context probe agents run on the HIGHEST tier available (Fable / Opus-class). Do not economize by shortening reads: every document a phase touches is read in FULL, not grepped for keywords. The deterministic layer stays deterministic regardless of budget -- scripts, not agents, for anything provable.
  • Verify-before-assert. Every claim in every audit output carries its evidence (path:line, command output, or commit). A claim without evidence is not written.
  • ASCII-only, LF, all standing hard rules apply.

1a. Operator-endorsed leans (recorded 2026-07-18; binding as PRIORS, not rulings)

The operator has reviewed the audit design and endorses the following. Record them accurately; do not strengthen or weaken them.

  • Primary success metric = the fresh-agent grounding test (Phase 3). The operator confirms: the audit succeeds when a cold session can stand up from the repo alone and know where it is. Findings volume is not the metric.
  • GA-R3 (D-number admission) and GA-R5 (one ruling per operator exchange, exact utterance quoted) are the operator's highest-priority structural rulings. Draft them first and draft them tight. Endorsement of DIRECTION is not ratification of TEXT: every GA-R still goes to the operator individually for gating.
  • D-130 mechanism -- the operator's provisional lean is (a) lifecycle { ignore_changes } on the seed volume, EXPLICITLY CONTINGENT on verification 8 confirming it actually suppresses the force-replacement diff. Rationale to record with the ruling: (a) is in-repo and environment-independent -- the Roosevelt-shaped property -- while (b) $TMPDIR depends on every future operator's shell being correctly configured, which is fragile in exactly the way that bites a bare-metal redeploy under pressure. If verification 8 refutes the suppression, the lean is DEAD -- do not rescue it. Fall back to (b) plus a runbook-enforced env guard, and say plainly that the fragility concern stands and is being accepted. This is a lean pending evidence, NOT a pre-ruling; the D-130 decision text must quote the verification output it rests on.
  • Sequencing stands as written: D-130 is ruled inside Phase 5, after CURRENT-STATE.md exists. The deploy is already stopped by its own gate; ruling against a contradictory record base is how the last batch went wrong.

2. Phase 1 -- deterministic inventory (scripts, not judgment)

Build and run (dry, read-only) a scripts/record-audit.py that produces docs/audit/record-inventory.md:

  1. Doc census. Every file in docs/: size, last-commit date, class (changelog / decision / ledger / runbook / readiness / finding / reference / patch), and whether anything else in the repo references it (grep). Unreferenced changelogs are consolidation candidates by default.
  2. Status-claim extractor. Grep every doc for the claim classes that have already been caught lying: plan counts (N add / N change / N destroy and N/N/N forms), stage statuses (READY / BLOCKED / CLOSED / COMPLETE / HELD), version pins (tofu/MAAS/Juju/OPNsense/NetBox version strings), and gate assertions ("blocking", "unblocked", "imported", "verified"). Emit a table: claim | file:line | value. Every place two rows disagree is a Phase-1 finding, mechanically, with zero judgment spent.
  3. D-number ledger check. Full ledger-scan.sh including --fences; reconcile the D-115-class false-positive fix (R3-F09) if still pending.
  4. Reference-integrity pass. Every doc that cites another doc, a commit, a line number, or a D-number: verify the target exists and says what the citer claims. Broken or stale references are findings.
  5. Repo-vs-live divergence snapshot (read-only): tofu plan captured to a file (this is the 7/2/7 vs 5/0/6 question made durable), virsh list --all + autostart table, git status. The plan OUTPUT FILE becomes the only citable source for plan counts from here forward.

3. Phase 2 -- ground-truth rebuild (highest-tier, fresh context)

One agent, highest tier, whose ONLY inputs are: Phase-1 outputs, the live environment (read-only), git history, and as-executed logs. Explicitly NOT inputs: changelogs' prose claims, the ledger's narrative, memory of prior sessions.

Deliverable: docs/CURRENT-STATE.md -- the single source of truth. Contents:

  • Where the project IS: stage, what is APPLIED (from tofu state + live env, not from docs), what is AUTHORED-BUT-NOT-APPLIED, what is RULED-BUT-NOT-BUILT.
  • Every open gate, each with: what closes it, who owns it, evidence of current state.
  • The definitive current plan count, cited to the Phase-1 captured plan file.
  • Version pin table, cited to --version command output captured in Phase 1.
  • Open decision queue (D-130 mechanism ruling; netem D-100 sub-item; anything else found), each stated as the exact question the operator must answer, with the verified evidence needed to answer it attached.

Rule going forward (proposed for ratification in Phase 4): no status claim may be hand-written anywhere else. Other docs POINT at CURRENT-STATE.md. CURRENT-STATE.md cites captured command output. Two sources for the same fact is the defect class this whole audit exists to kill.

4. Phase 3 -- the fresh-agent grounding test (this is the metric)

Deploy THREE clean-context agents, highest tier, no session history, no memory, no this-audit context. Each receives ONLY the repo (post-Phase-2, with CURRENT-STATE.md present) and this question set:

  1. What stage is this project in, and what is the very next action?
  2. What is applied to the live environment vs authored-only? Name three of each.
  3. What gates are currently open, and what closes each?
  4. What would you run before any tofu apply, and what must its output read?
  5. What are the three most dangerous known traps in this environment?
  6. Where do you look to find whether a decision D-NNN is ADOPTED, and can you trust a changelog's claim about it?
  7. You have just been asked to redeploy DC0 from scratch. Which document do you open, and does it get you there without leaving the repo?

Score each answer against ground truth. Every wrong or unanswerable item is a grounding hole, mapped to the artifact that should have answered it. This converts "Code lost grounding" from a feeling into a measured defect list -- and it is the direct test of the real requirement: that a future session can stand up cold and work. The holes map IS the Phase-5 work queue for documentation.

Run the probes INDEPENDENTLY (no shared scratchpad). If all three agree and are wrong the same way, that is a repo defect, not agent error -- weight it double.

5. Phase 4 -- structural rulings (proposed to the operator, individually gated)

Draft each of the following as a decision for the operator to gate. These are the recurrence-stoppers; without them the audit buys two weeks.

  • GA-R1: Single source of truth. CURRENT-STATE.md owns all status. Every other surface demotes to pointer or history. The workflow doc's stage table keeps only stage IDENTITY and links; the readiness doc becomes a checklist that cites the plan capture; README drops every pin that CURRENT-STATE.md owns.
  • GA-R2: Changelog consolidation. Per-stage consolidation on stage close (the proven v1 do-doc pattern): the 95 changelogs collapse into per-stage consolidated records; individual files preserved in git history via the consolidation commit; working set target: under 25 files in docs/. Standing rule: changelogs are session-scoped scratch, consolidated at stage close, and are NEVER citable as status authority (Phase 1 proved they lie).
  • GA-R3: D-number admission criteria. A D-number requires: architectural consequence beyond one stage, OR a Roosevelt-delta implication, OR supersession of an existing D. Operational choices (tool options, service settings, plugin lists, process notes) get a changelog line or a runbook edit, not a D-number. Include a one-time review: which of D-121..D-129 would not meet the bar (candidates: D-126, D-127, D-128, D-129) -- they KEEP their numbers (append-only), but the register gains a class tag so architectural Ds are findable among operational ones.
  • GA-R4: Ledger rotation. Sessions close with a bounded summary (hard cap, suggest 15 lines) written to the ledger; the body of a closed session moves to docs/archive/. The ledger's live section stays under a size a session can actually read in full (suggest 300 lines). Fence discipline unchanged.
  • GA-R5: Ruling gate discipline. Extend the existing individually-gated rule from mutations to RULINGS: one decision ruled per operator exchange, quoting the exact operator utterance in the status line. No batch adoptions. (This is R3-F01's lesson made standing policy.)
  • GA-R6: Executable definition of done. A stage is CLOSED only when a named script/gauntlet target passes, and the closing commit records its output. For the current gate specifically: DC0 readiness == the captured outer plan reads exactly 5 add / 0 change / 6 destroy AND the fresh-agent test question 7 passes. Prose cannot close a stage.

6. Phase 5 -- repair sweep (only now do fixes happen)

  • Consolidated findings register docs/audit/grounding-audit-YYYYMMDD.md: GA-F01..GA-Fnn, each with severity, evidence, proposed fix, and which GA-R ruling or grounding hole it traces to. Fold in and re-verify the 2026-07-18 issue list (D-130 staging trap, platform-traps items a/b, doc-integrity items, autostart state reconcile, seed-volume diff) and the adversarial review's unapplied residuals (R3-F12 Octavia headroom; netem sub-item; any other open R3-F).
  • The operator gates the sweep. Fixes are applied in traceable batches; every fix commit message names its GA-F number. A fix without a finding number does not ship. This is the churn-killer: it makes "found another thing, fixed it inline" impossible by construction.
  • Skill sweep (mandatory Phase-5 item): implement the RATIFIED GA-R rulings in the openstack-cloud-ops operating skill, per the standing stage-close skill-sweep discipline (commit 80502e9). The skill is what every future session loads first -- if the rulings do not land there, they decay. Proposed additions are in the operator's companion note (skill-additions-proposed.md); reconcile against the ratified GA-R text, not the proposal, where they differ. Keep the skill TIGHT: it is also record surface, and a bloated skill is this disease in its most expensive location.
  • The D-130 mechanism ruling happens HERE, informed by the two exempted verifications, as a single operator-gated decision -- after CURRENT-STATE.md exists, so the ruling lands on a trustworthy base.

7. Phase 6 -- exit criteria (executable, not prose)

The audit is DONE when, in order:

  1. Phase-1 extractor re-run reports ZERO contradictory status-claim pairs.
  2. docs/ working set is at the GA-R2 target with history preserved.
  3. The fresh-agent test re-runs with a NEW trio of clean agents and scores 7/7 on at least two of three agents, with no question failed by all three.
  4. The captured outer plan reads exactly 5 add / 0 change / 6 destroy (the standing gate; if the audit legitimately changes the expected counts, CURRENT-STATE.md records the new expected triple and WHY before the re-plan).
  5. The operator reads CURRENT-STATE.md top to bottom and signs it as matching their own understanding -- and where it does not match, the discrepancy is resolved in the document, not in chat.

Then, and only then, the DC0 apply path resumes from its gate.

8. Conduct notes

  • Do not perform contrition. The operator needs the system fixed, not apologized to. Findings about your own prior work are stated exactly like findings about anything else: evidence, severity, fix.
  • Where an audit finding implicates a prior operator ruling, say so plainly and propose the amendment -- the operator has explicitly asked for pushback and best-practice debate with sourced rationale.
  • If at any point the audit itself starts generating more record surface than it retires, STOP and report -- that is the disease presenting inside the cure, and the operator should see it immediately.