diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 09e89a9..1f6b979 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -808,7 +808,7 @@ ONE IDENTITY SERVES ONE PRINCIPAL TYPE, enforced by the matrix `principal` column, so the SEC-020 conflation becomes machine-detectable and needs no separate D-number. **ALL FIVE SUB-RULINGS RULED -> D-137 is ADOPTED 2026-07-26 and implementation is - UNBLOCKED** (Status line in design-decisions.md is the authority). Expect the first + UNBLOCKED, with the build spec at `docs/D-137-implementation-plan.md`** (Status line in design-decisions.md is the authority). Expect the first run to be RED by design: `admin` serves both a human and a service row, which is the defect the invariant names. Research capture: `docs/audit/creds-creation-points-20260725.md`. diff --git a/docs/D-137-implementation-plan.md b/docs/D-137-implementation-plan.md new file mode 100644 index 0000000..b14549f --- /dev/null +++ b/docs/D-137-implementation-plan.md @@ -0,0 +1,157 @@ +# D-137 implementation plan -- credential matrix build spec + +**Status of this document.** Build spec for D-137, ADOPTED 2026-07-26 with all five +sub-rulings recorded. `docs/design-decisions.md` D-137 is the ruling authority; this file +is the executable detail. Written 2026-07-26 so the build survives a fresh session -- +the design originated in a plan file outside the repo, which the GA-R4 bookend path would +not surface. Precedent: `docs/D-068-vault-migration-plan-draft.md`. + +**NOTHING IN HERE IS BUILT YET.** No matrix file, no checker, no `--render`, no preflight +`Pn`, no SEC-009 demotion. + +## Why (the short version) + +`creds-audit` is DECLARATION-based: the manifest IS the declaration, so an undeclared +secret is structurally invisible. It reported CLEAN on 2026-07-15 while four region-VM +secrets minted 2026-07-13 sat undeclared; `admin.pass` surfaced 12 days later only +because someone looked (SEC-020). The operator's insight closed the other half: **a +discovery sweep can never detect a credential that was never minted at all** -- absence is +invisible to discovery. Hence a forward EXPECTED-state register (the matrix), reconciled +against measured state. + +## The five ruled decisions (authority: D-137 Status block) + +1. **Enforcement = blocking in preflight.** The check runs as a new `Pn` in + `scripts/preflight.sh` and HARD-FAILS (exit 1) on expected-but-absent, undeclared, or + per-DC-asymmetric credentials. No PreToolUse guard. Advisory was rejected as the + posture that demonstrably failed. +2. **Manifests are DERIVED from the matrix.** Matrix is single source; `--render` + regenerates `creds-manifests/*.manifest`; the gauntlet FAILS if checked-in differs from + rendered. ACCEPTED COST: manifests become generated, so their governance prose (SEC + linkage, mint dates, off-manifest carve-outs, un-consolidated findings) must become + MATRIX FIELDS or be regenerated into the rendered header -- it must not be dropped. That + prose is load-bearing; `vr1-dc1.manifest:18-20` is what recorded the dc0 backfill debt. +3. **`--remote` is bounded to declared locations.** New `creds-manifests/vm-secret-locations` + (` `); never walks outside it. No tenant surface, D-069 preserved. + The list MUST include (faithful-implementation note in the ruling): per-site creds + folders on the jumphost AND the headend (SEC-022 shadow stores), the region's + `/root/maas-secrets/` (SEC-020), and the three dirs outside the SEC-009 convention + (the Vault init dir, the Octavia PKI dir, the per-tenant dirs -- capture FINDING 2), + plus `overlays/octavia-pki.yaml` in the clone. Adding a row is DoD for any new mint site. +4. **D-137 is the policy authority.** SEC-009's convention block demotes to a POINTER; + the ledger returns to being purely an exposure/rotation register. The demotion MOVES + the rule text -- SEC-009's founding history (the NetBox-token miss) stays as history. +5. **Identity folded in.** Invariant: **one identity serves one principal type.** The + `principal` column enforces it -- a credential serving both `human` and `service` is a + FAIL. Absorbs the SEC-020 conflation question; no separate D-number. + +## Matrix schema (logical keys ONLY -- SEC-004, repo is currently PUBLIC) + +`creds-matrix.tsv`, positional records in the manifests' idiom (do NOT invent a format +language; `scripts/site-ssh-config.sh:44-50` uses the same shape): + + id | cardinality | site-key | host-role | filename | access-type | principal | mint-stage | mint-ref | sec-ref + +- `cardinality`: `singleton` | `per-site` | `per-DC` | `per-node` | `per-tenant`. + REQUIRED for Roosevelt transfer -- without it the matrix cannot express "9 nodes". +- `site-key`: region-qualified ONLY (`vr1-dc0`), bare `dc0` REJECTED -- import the + discipline at `scripts/lib-net.sh:160-162` / `lib-hosts.sh:153-155`. +- `host-role`: role token (`jumphost`, `headend`, `rack`, `edge`) -- NOT `voffice1` or an + IP. This still expresses "the copy is on the wrong role" without publishing topology. +- `access-type`: `gui` | `api` | `ssh` | `cli-profile` | `console` | `none`. +- `principal`: `human` | `service`. Ruling 5's invariant keys off this. +- `mint-ref`: MUST admit THREE provenance kinds -- `script:line`, `runbook:step`, and + `operator-terminal`. Capture FINDING 1: `ssh-keygen` returns ZERO hits repo-wide, so 12 + declared secrets have NO mint command; those rows are the DEBT the matrix exists to + surface and must be admitted, not treated as parse errors. +- NO grants, hostnames, IPs, custody detail, or values (ruling: logical keys only). + +## Checker: `scripts/creds-matrix.py` + +Model on `scripts/provider-bundle-check.py` -- the repo's canonical declarative gate: +expectations as top-of-file constants keyed by a region-qualified slug, `--dc` selector +with a safe default, `oks`/`fails` accumulators, `[ok]`/`[FAIL]` lines, ONE verdict line, +`return 1 if fails`, and checks that SELF-SKIP with an explicit ok when a precondition is +absent (`:281`) rather than silently passing -- needed for sites not yet deployed. + +Import the BOTH-BOUNDS discipline from `netbox/sandbox-fidelity-check.py:131-143`, which +records the identical false-green in-repo: *"EXPECTED_NEW was an upper bound masquerading +as an assertion. It must be BOTH bounds."* A credential matrix has the same failure mode +one level up: enumerate no sites -> nothing undeclared -> false CLEAN. Reuse its four-way +drift vocabulary -- MISSING / UNEXPECTED-EXTRA / EXPECTED-BUT-ABSENT / FIELD DRIFT. +Provenance drift is a FIELD mismatch, not a presence mismatch. + +### Three tiers, increasing cost + +1. **STATIC** (offline, gauntlet): matrix parses; every row has a manifest row; rendered + manifests == checked-in (ruling 2); every `script:line`/`runbook:step` `mint-ref` + resolves to a real location; per-DC rows are SYMMETRIC; principal invariant (ruling 5). +2. **EXISTENCE** (local + remote, preflight `Pn`): every expected credential present at + the expected host-role with expected mode; nothing undeclared at any declared location + INCLUDING remote ones (ruling 3). +3. **VALIDITY** (stage close, `--probe`): provenance digests match across hosts; per-row + behavioral probes where defined. Motivated by: `creds-audit` PARSES provenance and + NEVER verifies it (`fsrc` used only in the MISSING message), so the SEC-020 stale-trap + warning is unchecked today. + +## Hard constraints inherited + +- **Metadata only, enforced by source-grep.** `tests/creds-audit/run-tests.sh:71-73` fails + on any `cat|head|tail|less|more|od|xxd|hexdump` in the audit source. Remote provenance + verification MUST compare `sha256sum` digests over ssh -- never transfer content. A new + checker should carry the same guard test. +- **Gauntlet registration is pure glob** over `tests/*/run-tests.sh` + (`scripts/run-tests-all.sh:22`); only `tail -1` is shown, so the LAST line must + self-describe. Failure-detail excerpting matches `^\s*(FAIL|\[XX\]|MISS|LEAK|COUNT)` -- + it does NOT match `[FAIL]`, which is why creds-audit failures currently surface with no + detail. Lead detail lines with bare `FAIL`. +- **preflight** aggregation is `note $?` worst-exit (1 dominates 2 dominates 0); add one + banner + one call honouring the 0/1/2 contract. +- **repo-lint L1**: new `design-decisions.md` text must be pure ASCII. **L10**: any commit + touching `docs/audit/`, a `**Status:` line, or a gate row must touch CURRENT-STATE in the + SAME commit. +- Harness style: mutation-of-a-real-artifact fixtures + `rc` AND output-regex assertions, + one case per invariant, each named for the real defect it encodes + (`tests/provider-bundle-check/run-tests.sh`). + +## Files + +NEW: `creds-matrix.tsv`, `creds-manifests/vm-secret-locations`, `scripts/creds-matrix.py`, +`tests/creds-matrix/run-tests.sh`. +MODIFY: `scripts/preflight.sh` (new `Pn`), `scripts/creds-audit.sh` (`--all`, `--remote`, +VERIFY provenance, widen sprawl globs -- currently miss `admin.pass`, `*.apikey`, `*.key`, +`*.pem`, `*_ed25519`, `admin-openrc`), `creds-manifests/*.manifest` (become generated), +`docs/security-ledger.md` (SEC-009 demotion per ruling 4), `docs/CURRENT-STATE.md` (C1). + +## Verification / acceptance + +- `bash tests/creds-matrix/run-tests.sh` -- every invariant FAILS when violated. +- **ACCEPTANCE TEST:** `scripts/creds-matrix.py --all` must reproduce SEC-021/-022/-023 and + the openrc case as NAMED failures on today's tree. A checker that goes green on the + current tree is wrong. +- `bash scripts/run-tests-all.sh` ALL GREEN; `bash scripts/repo-lint.sh` 0-fail; + `bash scripts/preflight.sh` exercises the new `Pn`. + +## EXPECTED RED BY DESIGN -- do not "fix" by weakening the check + +The first full run SHOULD fail, and these are the defects, not bugs in the checker: +- `admin` serves a `human` GUI row AND a `service` api row -> ruling-5 invariant FAIL. + Remediation is the `operator`/`admin` split already begun 2026-07-25, i.e. migrate the + 19 `maas admin` call sites to a service identity. +- 12 rows carry `mint-ref=operator-terminal` -> reproducibility debt to convert. +- SEC-021 (dc0 `opnsense-api.txt` ABSENT; power-key name/host divergence), SEC-022 + (headend shadow stores), SEC-023 (sprawl blind spots + predicted `~/admin-openrc`). + +Going green is a REMEDIATION project separate from the build. Do not scope-creep it into +the build; land the checker red, record the failures, then gate remediation per row. + +## Fresh-session start + +1. Normal bootstrap: `git pull`, `bash scripts/repo-lint.sh`, read + `docs/CURRENT-STATE.md` in full, then `docs/session-ledger.md` + + `bash scripts/ledger-scan.sh`. +2. Read `docs/design-decisions.md` D-137 (the ruling authority -- all five sub-rulings and + their verbatim operator selections), then this file. +3. Read the research capture `docs/audit/creds-creation-points-20260725.md` -- it is the + matrix SEED (55 MINT sites classified by host, destination, stage, human/service). +4. Build tier 1 first (offline, no live access needed), harness it, then tier 2, then tier 3. diff --git a/docs/changelog-20260725-maas-admin-recovery.md b/docs/changelog-20260725-maas-admin-recovery.md index 28a3cd4..7f53e79 100644 --- a/docs/changelog-20260725-maas-admin-recovery.md +++ b/docs/changelog-20260725-maas-admin-recovery.md @@ -240,3 +240,24 @@ row -- that IS the defect). **Revert:** flip the D-137 Status line back to PROPOSED and drop the five sub-ruling blocks; restore CURRENT-STATE item 9 and the ledger open-decision count to 5. + +## Item 12 -- D-137 build spec made repo-durable (handoff to a fresh session) + +**What:** new `docs/D-137-implementation-plan.md` -- the executable build detail for the +ADOPTED D-137: the five ruled decisions and their consequences, the matrix schema +(logical keys only per the SEC-004 ruling), the checker model and precedents with +file:line, the three tiers, the inherited hard constraints, the file list, the acceptance +test, and an explicit "EXPECTED RED BY DESIGN" section. CURRENT-STATE item 9 and the +ledger addendum now point at it. Also updated the `creds-folder-convention` auto-memory +pointer: policy authority is now D-137 (ruling 4), with a verify-first caveat that +`creds-audit` CLEAN is not evidence of completeness and the matrix is not built. +**Why:** the design and committee review originated in a plan file under +`~/.claude/plans/`, which is OUTSIDE the repo -- so a fresh session bootstrapping via the +GA-R4 path (CURRENT-STATE -> session-ledger -> ledger-scan) would never surface it, and +the repo-is-authoritative rule would leave the build spec undiscoverable. Precedent for a +D-NNN plan doc in-repo: `docs/D-068-vault-migration-plan-draft.md`. Operator directed the +build to a fresh session, which makes this durability gap load-bearing rather than tidy. +**Note:** the spec deliberately tells the next session NOT to make the first run green -- +the acceptance test is that the checker REPRODUCES SEC-021/-022/-023 and the openrc case +as named failures. A checker that passes on today's tree is wrong. +**Revert:** `git rm docs/D-137-implementation-plan.md` and drop the two pointers. diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 4875ce9..40e0cfb 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -406,6 +406,8 @@ - **SEC-021/-022/-023 OPENED** (rows 16 -> 19): a consolidated dc0 credential ABSENT from its recorded location; two UNAUDITED shadow *-creds/ stores on the headend; sprawl-glob blind spots incl. a PREDICTED Stage-5 admin-openrc exposure. All logged-not-actioned. -- NEXT: build per the approved plan (matrix + checker + --render + preflight Pn + SEC-009 - demotion), first run expected RED by design. Plan: whimsical-wandering-spindle. +- NEXT (fresh session, operator-directed): build per **`docs/D-137-implementation-plan.md`** + (the repo-carried build spec -- the design originated in an out-of-repo plan file, which + this bookend path would not surface). First run expected RED BY DESIGN; going green is a + separate remediation project, do not scope-creep it into the build. - Status ONLY in CURRENT-STATE.md. Detail: docs/changelog-20260725-maas-admin-recovery.md.