Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / D-137-implementation-plan.md
@JANeumatrix JANeumatrix 14 hours ago 9 KB D-137: repo-carried build spec for the fresh-session handoff
Raw Blame History

D-137 implementation plan -- credential matrix build spec

Status of this document. Build spec for D-137, ADOPTED 2026-07-26 with all five sub-rulings recorded. docs/design-decisions.md D-137 is the ruling authority; this file is the executable detail. Written 2026-07-26 so the build survives a fresh session -- the design originated in a plan file outside the repo, which the GA-R4 bookend path would not surface. Precedent: docs/D-068-vault-migration-plan-draft.md.

NOTHING IN HERE IS BUILT YET. No matrix file, no checker, no --render, no preflight Pn, no SEC-009 demotion.

Why (the short version)

creds-audit is DECLARATION-based: the manifest IS the declaration, so an undeclared secret is structurally invisible. It reported CLEAN on 2026-07-15 while four region-VM secrets minted 2026-07-13 sat undeclared; admin.pass surfaced 12 days later only because someone looked (SEC-020). The operator's insight closed the other half: a discovery sweep can never detect a credential that was never minted at all -- absence is invisible to discovery. Hence a forward EXPECTED-state register (the matrix), reconciled against measured state.

The five ruled decisions (authority: D-137 Status block)

  1. Enforcement = blocking in preflight. The check runs as a new Pn in scripts/preflight.sh and HARD-FAILS (exit 1) on expected-but-absent, undeclared, or per-DC-asymmetric credentials. No PreToolUse guard. Advisory was rejected as the posture that demonstrably failed.
  2. Manifests are DERIVED from the matrix. Matrix is single source; --render regenerates creds-manifests/*.manifest; the gauntlet FAILS if checked-in differs from rendered. ACCEPTED COST: manifests become generated, so their governance prose (SEC linkage, mint dates, off-manifest carve-outs, un-consolidated findings) must become MATRIX FIELDS or be regenerated into the rendered header -- it must not be dropped. That prose is load-bearing; vr1-dc1.manifest:18-20 is what recorded the dc0 backfill debt.
  3. --remote is bounded to declared locations. New creds-manifests/vm-secret-locations (<host-role> <path>); never walks outside it. No tenant surface, D-069 preserved. The list MUST include (faithful-implementation note in the ruling): per-site creds folders on the jumphost AND the headend (SEC-022 shadow stores), the region's /root/maas-secrets/ (SEC-020), and the three dirs outside the SEC-009 convention (the Vault init dir, the Octavia PKI dir, the per-tenant dirs -- capture FINDING 2), plus overlays/octavia-pki.yaml in the clone. Adding a row is DoD for any new mint site.
  4. D-137 is the policy authority. SEC-009's convention block demotes to a POINTER; the ledger returns to being purely an exposure/rotation register. The demotion MOVES the rule text -- SEC-009's founding history (the NetBox-token miss) stays as history.
  5. Identity folded in. Invariant: one identity serves one principal type. The principal column enforces it -- a credential serving both human and service is a FAIL. Absorbs the SEC-020 conflation question; no separate D-number.

Matrix schema (logical keys ONLY -- SEC-004, repo is currently PUBLIC)

creds-matrix.tsv, positional records in the manifests' idiom (do NOT invent a format language; scripts/site-ssh-config.sh:44-50 uses the same shape):

id | cardinality | site-key | host-role | filename | access-type | principal | mint-stage | mint-ref | sec-ref
  • cardinality: singleton | per-site | per-DC | per-node | per-tenant. REQUIRED for Roosevelt transfer -- without it the matrix cannot express "9 nodes".
  • site-key: region-qualified ONLY (vr1-dc0), bare dc0 REJECTED -- import the discipline at scripts/lib-net.sh:160-162 / lib-hosts.sh:153-155.
  • host-role: role token (jumphost, headend, rack, edge) -- NOT voffice1 or an IP. This still expresses "the copy is on the wrong role" without publishing topology.
  • access-type: gui | api | ssh | cli-profile | console | none.
  • principal: human | service. Ruling 5's invariant keys off this.
  • mint-ref: MUST admit THREE provenance kinds -- script:line, runbook:step, and operator-terminal. Capture FINDING 1: ssh-keygen returns ZERO hits repo-wide, so 12 declared secrets have NO mint command; those rows are the DEBT the matrix exists to surface and must be admitted, not treated as parse errors.
  • NO grants, hostnames, IPs, custody detail, or values (ruling: logical keys only).

Checker: scripts/creds-matrix.py

Model on scripts/provider-bundle-check.py -- the repo's canonical declarative gate: expectations as top-of-file constants keyed by a region-qualified slug, --dc selector with a safe default, oks/fails accumulators, [ok]/[FAIL] lines, ONE verdict line, return 1 if fails, and checks that SELF-SKIP with an explicit ok when a precondition is absent (:281) rather than silently passing -- needed for sites not yet deployed.

Import the BOTH-BOUNDS discipline from netbox/sandbox-fidelity-check.py:131-143, which records the identical false-green in-repo: "EXPECTED_NEW was an upper bound masquerading as an assertion. It must be BOTH bounds." A credential matrix has the same failure mode one level up: enumerate no sites -> nothing undeclared -> false CLEAN. Reuse its four-way drift vocabulary -- MISSING / UNEXPECTED-EXTRA / EXPECTED-BUT-ABSENT / FIELD DRIFT. Provenance drift is a FIELD mismatch, not a presence mismatch.

Three tiers, increasing cost

  1. STATIC (offline, gauntlet): matrix parses; every row has a manifest row; rendered manifests == checked-in (ruling 2); every script:line/runbook:step mint-ref resolves to a real location; per-DC rows are SYMMETRIC; principal invariant (ruling 5).
  2. EXISTENCE (local + remote, preflight Pn): every expected credential present at the expected host-role with expected mode; nothing undeclared at any declared location INCLUDING remote ones (ruling 3).
  3. VALIDITY (stage close, --probe): provenance digests match across hosts; per-row behavioral probes where defined. Motivated by: creds-audit PARSES provenance and NEVER verifies it (fsrc used only in the MISSING message), so the SEC-020 stale-trap warning is unchecked today.

Hard constraints inherited

  • Metadata only, enforced by source-grep. tests/creds-audit/run-tests.sh:71-73 fails on any cat|head|tail|less|more|od|xxd|hexdump in the audit source. Remote provenance verification MUST compare sha256sum digests over ssh -- never transfer content. A new checker should carry the same guard test.
  • Gauntlet registration is pure glob over tests/*/run-tests.sh (scripts/run-tests-all.sh:22); only tail -1 is shown, so the LAST line must self-describe. Failure-detail excerpting matches ^\s*(FAIL|\[XX\]|MISS|LEAK|COUNT) -- it does NOT match [FAIL], which is why creds-audit failures currently surface with no detail. Lead detail lines with bare FAIL.
  • preflight aggregation is note $? worst-exit (1 dominates 2 dominates 0); add one banner + one call honouring the 0/1/2 contract.
  • repo-lint L1: new design-decisions.md text must be pure ASCII. L10: any commit touching docs/audit/, a **Status: line, or a gate row must touch CURRENT-STATE in the SAME commit.
  • Harness style: mutation-of-a-real-artifact fixtures + rc AND output-regex assertions, one case per invariant, each named for the real defect it encodes (tests/provider-bundle-check/run-tests.sh).

Files

NEW: creds-matrix.tsv, creds-manifests/vm-secret-locations, scripts/creds-matrix.py, tests/creds-matrix/run-tests.sh. MODIFY: scripts/preflight.sh (new Pn), scripts/creds-audit.sh (--all, --remote, VERIFY provenance, widen sprawl globs -- currently miss admin.pass, *.apikey, *.key, *.pem, *_ed25519, admin-openrc), creds-manifests/*.manifest (become generated), docs/security-ledger.md (SEC-009 demotion per ruling 4), docs/CURRENT-STATE.md (C1).

Verification / acceptance

  • bash tests/creds-matrix/run-tests.sh -- every invariant FAILS when violated.
  • ACCEPTANCE TEST: scripts/creds-matrix.py --all must reproduce SEC-021/-022/-023 and the openrc case as NAMED failures on today's tree. A checker that goes green on the current tree is wrong.
  • bash scripts/run-tests-all.sh ALL GREEN; bash scripts/repo-lint.sh 0-fail; bash scripts/preflight.sh exercises the new Pn.

EXPECTED RED BY DESIGN -- do not "fix" by weakening the check

The first full run SHOULD fail, and these are the defects, not bugs in the checker:

  • admin serves a human GUI row AND a service api row -> ruling-5 invariant FAIL. Remediation is the operator/admin split already begun 2026-07-25, i.e. migrate the 19 maas admin call sites to a service identity.
  • 12 rows carry mint-ref=operator-terminal -> reproducibility debt to convert.
  • SEC-021 (dc0 opnsense-api.txt ABSENT; power-key name/host divergence), SEC-022 (headend shadow stores), SEC-023 (sprawl blind spots + predicted ~/admin-openrc).

Going green is a REMEDIATION project separate from the build. Do not scope-creep it into the build; land the checker red, record the failures, then gate remediation per row.

Fresh-session start

  1. Normal bootstrap: git pull, bash scripts/repo-lint.sh, read docs/CURRENT-STATE.md in full, then docs/session-ledger.md + bash scripts/ledger-scan.sh.
  2. Read docs/design-decisions.md D-137 (the ruling authority -- all five sub-rulings and their verbatim operator selections), then this file.
  3. Read the research capture docs/audit/creds-creation-points-20260725.md -- it is the matrix SEED (55 MINT sites classified by host, destination, stage, human/service).
  4. Build tier 1 first (offline, no live access needed), harness it, then tier 2, then tier 3.