diff --git a/docs/audit/grounding-audit-charter.md b/docs/audit/grounding-audit-charter.md new file mode 100644 index 0000000..1979e7e --- /dev/null +++ b/docs/audit/grounding-audit-charter.md @@ -0,0 +1,247 @@ +# PROMPT FOR CODE -- Project grounding audit and record-system restructure + +## 0. Mission and framing + +The operator's assessment, which the evidence supports: the project's record-keeping +system has become the primary work generator. Sessions open by reconciling records, +find discrepancies, file findings about the discrepancies, and thereby create more +reconciliation surface. The last working day was spent in circles at the end of the +Office1 stage without reaching a stable point on either goal (trustworthy records, or +a repeatable stable environment). Details are getting lost. This audit exists to stop +that structurally, not to fix the current crop of defects faster. + +Measured evidence (verified 2026-07-18 against the pushed repo at `43b29d7`): + +- `docs/` holds 135 files; **95 are changelogs** (31 filed on 2026-07-09 alone; + sustained 6-14/day since). +- **Nine D-numbers ruled in four days** (D-121..D-129). Several are operational + choices, not architecture: D-126 (an SSH-forward option), D-127 (an autostart + setting), D-128 (an operating model note), D-129 (a plugin list). The decision + register is being used as a diary, which dilutes its authority and multiplies + reconciliation surface. +- `docs/dc0-deploy-readiness.md` carries **5/2/6 at line 37 and 5/0/6 at line 59** -- + two contradictory plan counts, both live, in the deploy gate document. +- The D-127 changelog asserts plan outcomes that are false as written (per the + 2026-07-18 session's own findings). README pins tofu 1.12.3; the binary is 1.12.4. +- `docs/session-ledger.md` is 1,125 lines and growing without a rotation rule. +- Root cause pattern: **status lives as hand-written copies in many surfaces** + (workflow-doc stage rows, ledger, changelogs, readiness doc, README, runbook gates), + each rotting at its own rate. Every fix to one copy creates drift in another. + +What is NOT broken, and must be preserved: fence discipline (validated OK), the +adversarial-review process (ran 2026-07-16, R3-F01..F13, repairs landed), gated +mutation discipline, honest retractions, refusal to fabricate. This session's +epistemics are good. The audit repairs the SYSTEM, not the operator relationship or +the discipline culture. + +## 1. Ground rules (binding for the whole audit) + +- **FULL FREEZE.** No fixes -- not one typo, not one doc correction -- until Phase 5. + No new D-numbers (audit findings use `GA-NNN`). No new changelogs. No module edits. + No state writes. No cloud mutation. The 2026-07-18 issue list (D-130 candidates, + platform-traps, doc-integrity items, env reconcile) is EVIDENCE for this audit, + not a work queue. It gets re-verified and folded into the findings register. + - **Sole exemption:** the two read-only verifications already identified (v0.9.8 + volume-replace-under-running-domain on a throwaway domain; ignore_changes + suppression check). They are audit evidence-gathering and are read-only / + throwaway-scoped by construction. +- **Fresh session.** This prompt is the entire charter of a NEW session. Do not + resume the circling session; its context carries every half-fix as live state. +- **Model tiering: token budget is explicitly NOT a constraint.** The operator has + authorized maximum effort. Ground-truth author, adjudicator, and all fresh-context + probe agents run on the HIGHEST tier available (Fable / Opus-class). Do not + economize by shortening reads: every document a phase touches is read in FULL, not + grepped for keywords. The deterministic layer stays deterministic regardless of + budget -- scripts, not agents, for anything provable. +- Verify-before-assert. Every claim in every audit output carries its evidence + (path:line, command output, or commit). A claim without evidence is not written. +- ASCII-only, LF, all standing hard rules apply. + +## 1a. Operator-endorsed leans (recorded 2026-07-18; binding as PRIORS, not rulings) + +The operator has reviewed the audit design and endorses the following. Record them +accurately; do not strengthen or weaken them. + +- **Primary success metric = the fresh-agent grounding test (Phase 3).** The operator + confirms: the audit succeeds when a cold session can stand up from the repo alone + and know where it is. Findings volume is not the metric. +- **GA-R3 (D-number admission) and GA-R5 (one ruling per operator exchange, exact + utterance quoted) are the operator's highest-priority structural rulings.** Draft + them first and draft them tight. Endorsement of DIRECTION is not ratification of + TEXT: every GA-R still goes to the operator individually for gating. +- **D-130 mechanism -- the operator's provisional lean is (a) `lifecycle + { ignore_changes }` on the seed volume, EXPLICITLY CONTINGENT on verification 8 + confirming it actually suppresses the force-replacement diff.** Rationale to + record with the ruling: (a) is in-repo and environment-independent -- the + Roosevelt-shaped property -- while (b) `$TMPDIR` depends on every future + operator's shell being correctly configured, which is fragile in exactly the way + that bites a bare-metal redeploy under pressure. **If verification 8 refutes the + suppression, the lean is DEAD -- do not rescue it. Fall back to (b) plus a + runbook-enforced env guard, and say plainly that the fragility concern stands and + is being accepted.** This is a lean pending evidence, NOT a pre-ruling; the D-130 + decision text must quote the verification output it rests on. +- **Sequencing stands as written:** D-130 is ruled inside Phase 5, after + CURRENT-STATE.md exists. The deploy is already stopped by its own gate; ruling + against a contradictory record base is how the last batch went wrong. + +## 2. Phase 1 -- deterministic inventory (scripts, not judgment) + +Build and run (dry, read-only) a `scripts/record-audit.py` that produces +`docs/audit/record-inventory.md`: + +1. **Doc census.** Every file in `docs/`: size, last-commit date, class (changelog / + decision / ledger / runbook / readiness / finding / reference / patch), and + whether anything else in the repo references it (grep). Unreferenced changelogs + are consolidation candidates by default. +2. **Status-claim extractor.** Grep every doc for the claim classes that have already + been caught lying: plan counts (`N add / N change / N destroy` and `N/N/N` forms), + stage statuses (READY / BLOCKED / CLOSED / COMPLETE / HELD), version pins + (tofu/MAAS/Juju/OPNsense/NetBox version strings), and gate assertions + ("blocking", "unblocked", "imported", "verified"). Emit a table: + claim | file:line | value. **Every place two rows disagree is a Phase-1 finding, + mechanically, with zero judgment spent.** +3. **D-number ledger check.** Full `ledger-scan.sh` including `--fences`; reconcile + the D-115-class false-positive fix (R3-F09) if still pending. +4. **Reference-integrity pass.** Every doc that cites another doc, a commit, a line + number, or a D-number: verify the target exists and says what the citer claims. + Broken or stale references are findings. +5. **Repo-vs-live divergence snapshot** (read-only): `tofu plan` captured to a file + (this is the 7/2/7 vs 5/0/6 question made durable), `virsh list --all` + + autostart table, git status. The plan OUTPUT FILE becomes the only citable source + for plan counts from here forward. + +## 3. Phase 2 -- ground-truth rebuild (highest-tier, fresh context) + +One agent, highest tier, whose ONLY inputs are: Phase-1 outputs, the live environment +(read-only), git history, and as-executed logs. Explicitly NOT inputs: changelogs' +prose claims, the ledger's narrative, memory of prior sessions. + +Deliverable: `docs/CURRENT-STATE.md` -- the single source of truth. Contents: + +- Where the project IS: stage, what is APPLIED (from tofu state + live env, not from + docs), what is AUTHORED-BUT-NOT-APPLIED, what is RULED-BUT-NOT-BUILT. +- Every open gate, each with: what closes it, who owns it, evidence of current state. +- The definitive current plan count, cited to the Phase-1 captured plan file. +- Version pin table, cited to `--version` command output captured in Phase 1. +- Open decision queue (D-130 mechanism ruling; netem D-100 sub-item; anything else + found), each stated as the exact question the operator must answer, with the + verified evidence needed to answer it attached. + +Rule going forward (proposed for ratification in Phase 4): **no status claim may be +hand-written anywhere else.** Other docs POINT at CURRENT-STATE.md. CURRENT-STATE.md +cites captured command output. Two sources for the same fact is the defect class this +whole audit exists to kill. + +## 4. Phase 3 -- the fresh-agent grounding test (this is the metric) + +Deploy THREE clean-context agents, highest tier, no session history, no memory, no +this-audit context. Each receives ONLY the repo (post-Phase-2, with CURRENT-STATE.md +present) and this question set: + +1. What stage is this project in, and what is the very next action? +2. What is applied to the live environment vs authored-only? Name three of each. +3. What gates are currently open, and what closes each? +4. What would you run before any `tofu apply`, and what must its output read? +5. What are the three most dangerous known traps in this environment? +6. Where do you look to find whether a decision D-NNN is ADOPTED, and can you trust + a changelog's claim about it? +7. You have just been asked to redeploy DC0 from scratch. Which document do you open, + and does it get you there without leaving the repo? + +Score each answer against ground truth. **Every wrong or unanswerable item is a +grounding hole, mapped to the artifact that should have answered it.** This converts +"Code lost grounding" from a feeling into a measured defect list -- and it is the +direct test of the real requirement: that a future session can stand up cold and +work. The holes map IS the Phase-5 work queue for documentation. + +Run the probes INDEPENDENTLY (no shared scratchpad). If all three agree and are +wrong the same way, that is a repo defect, not agent error -- weight it double. + +## 5. Phase 4 -- structural rulings (proposed to the operator, individually gated) + +Draft each of the following as a decision for the operator to gate. These are the +recurrence-stoppers; without them the audit buys two weeks. + +- **GA-R1: Single source of truth.** CURRENT-STATE.md owns all status. Every other + surface demotes to pointer or history. The workflow doc's stage table keeps only + stage IDENTITY and links; the readiness doc becomes a checklist that cites the + plan capture; README drops every pin that CURRENT-STATE.md owns. +- **GA-R2: Changelog consolidation.** Per-stage consolidation on stage close (the + proven v1 do-doc pattern): the 95 changelogs collapse into per-stage consolidated + records; individual files preserved in git history via the consolidation commit; + working set target: under 25 files in docs/. Standing rule: changelogs are + session-scoped scratch, consolidated at stage close, and are NEVER citable as + status authority (Phase 1 proved they lie). +- **GA-R3: D-number admission criteria.** A D-number requires: architectural + consequence beyond one stage, OR a Roosevelt-delta implication, OR supersession of + an existing D. Operational choices (tool options, service settings, plugin lists, + process notes) get a changelog line or a runbook edit, not a D-number. Include a + one-time review: which of D-121..D-129 would not meet the bar (candidates: + D-126, D-127, D-128, D-129) -- they KEEP their numbers (append-only), but the + register gains a class tag so architectural Ds are findable among operational ones. +- **GA-R4: Ledger rotation.** Sessions close with a bounded summary (hard cap, + suggest 15 lines) written to the ledger; the body of a closed session moves to + docs/archive/. The ledger's live section stays under a size a session can actually + read in full (suggest 300 lines). Fence discipline unchanged. +- **GA-R5: Ruling gate discipline.** Extend the existing individually-gated rule + from mutations to RULINGS: one decision ruled per operator exchange, quoting the + exact operator utterance in the status line. No batch adoptions. (This is R3-F01's + lesson made standing policy.) +- **GA-R6: Executable definition of done.** A stage is CLOSED only when a named + script/gauntlet target passes, and the closing commit records its output. For the + current gate specifically: DC0 readiness == the captured outer plan reads exactly + 5 add / 0 change / 6 destroy AND the fresh-agent test question 7 passes. Prose + cannot close a stage. + +## 6. Phase 5 -- repair sweep (only now do fixes happen) + +- Consolidated findings register `docs/audit/grounding-audit-YYYYMMDD.md`: + GA-F01..GA-Fnn, each with severity, evidence, proposed fix, and which GA-R ruling + or grounding hole it traces to. Fold in and re-verify the 2026-07-18 issue list + (D-130 staging trap, platform-traps items a/b, doc-integrity items, autostart + state reconcile, seed-volume diff) and the adversarial review's unapplied + residuals (R3-F12 Octavia headroom; netem sub-item; any other open R3-F). +- The operator gates the sweep. Fixes are applied in traceable batches; **every fix + commit message names its GA-F number. A fix without a finding number does not + ship.** This is the churn-killer: it makes "found another thing, fixed it inline" + impossible by construction. +- **Skill sweep (mandatory Phase-5 item):** implement the RATIFIED GA-R rulings in + the `openstack-cloud-ops` operating skill, per the standing stage-close + skill-sweep discipline (commit `80502e9`). The skill is what every future session + loads first -- if the rulings do not land there, they decay. Proposed additions + are in the operator's companion note (`skill-additions-proposed.md`); reconcile + against the ratified GA-R text, not the proposal, where they differ. Keep the + skill TIGHT: it is also record surface, and a bloated skill is this disease in + its most expensive location. +- The D-130 mechanism ruling happens HERE, informed by the two exempted + verifications, as a single operator-gated decision -- after CURRENT-STATE.md + exists, so the ruling lands on a trustworthy base. + +## 7. Phase 6 -- exit criteria (executable, not prose) + +The audit is DONE when, in order: + +1. Phase-1 extractor re-run reports ZERO contradictory status-claim pairs. +2. `docs/` working set is at the GA-R2 target with history preserved. +3. The fresh-agent test re-runs with a NEW trio of clean agents and scores 7/7 on + at least two of three agents, with no question failed by all three. +4. The captured outer plan reads exactly 5 add / 0 change / 6 destroy (the standing + gate; if the audit legitimately changes the expected counts, CURRENT-STATE.md + records the new expected triple and WHY before the re-plan). +5. The operator reads CURRENT-STATE.md top to bottom and signs it as matching their + own understanding -- and where it does not match, the discrepancy is resolved in + the document, not in chat. + +Then, and only then, the DC0 apply path resumes from its gate. + +## 8. Conduct notes + +- Do not perform contrition. The operator needs the system fixed, not apologized to. + Findings about your own prior work are stated exactly like findings about anything + else: evidence, severity, fix. +- Where an audit finding implicates a prior operator ruling, say so plainly and + propose the amendment -- the operator has explicitly asked for pushback and + best-practice debate with sourced rationale. +- If at any point the audit itself starts generating more record surface than it + retires, STOP and report -- that is the disease presenting inside the cure, and + the operator should see it immediately. diff --git a/docs/audit/skill-additions-proposed.md b/docs/audit/skill-additions-proposed.md new file mode 100644 index 0000000..30f47cd --- /dev/null +++ b/docs/audit/skill-additions-proposed.md @@ -0,0 +1,36 @@ +# Proposed additions to the openstack-cloud-ops skill (GATED: apply in audit +# Phase 5, reconciled against the RATIFIED GA-R text, not this draft) + +Add one section. Keep it under ~35 lines in the skill -- the skill is itself record +surface, and every session pays its length. + +--- + +## Record authority and session discipline (GA rulings) + +- **`docs/CURRENT-STATE.md` is the ONLY status authority.** Stage, applied-vs- + authored, gates, plan counts, version pins live THERE and nowhere else. Never + hand-write a status claim in any other document; point at CURRENT-STATE.md. + Update it in the same commit as the change that alters status. +- **Plan counts and versions are quoted only from captured command output** + (a committed plan capture, a `--version` capture) -- never from memory, never + from another document's prose. +- **Changelogs are session-scoped scratch.** They are consolidated per-stage at + stage close and are NEVER citable as status or decision authority. If a changelog + and CURRENT-STATE.md disagree, the changelog is wrong by definition. +- **One ruling per operator exchange.** A decision's ADOPTED status line must quote + the exact operator utterance that ruled it. Never batch-adopt. Never adopt from + an inferred or reconstructed ruling -- an operator ruling is a value under hard + rule 2: if it would be inferred, STOP and ask the A/B question in one line. +- **D-number admission bar:** a D-number requires architectural consequence beyond + one stage, OR a Roosevelt-delta implication, OR supersession of an existing D. + Tool options, service settings, plugin lists, and process notes get a changelog + line or a runbook edit, not a D-number. +- **Session bookends.** OPEN: read CURRENT-STATE.md in full before any other doc. + CLOSE: write a ledger summary of at most 15 lines; commit and push; a session + that ends without its bookend is treated as NOT durable by the next session. +- **Reconciliation circuit-breaker.** If you have been reconciling records -- + fixing docs to match docs -- for more than one cycle without touching the actual + deliverable, STOP and report to the operator. Record-churn presenting as + progress is this project's known failure mode; escalating it is the correct + move, not continuing.