diff --git a/creds-matrix-notes.md b/creds-matrix-notes.md new file mode 100644 index 0000000..66a833c --- /dev/null +++ b/creds-matrix-notes.md @@ -0,0 +1,245 @@ +# creds-matrix-notes.md -- `notes-ref` targets for creds-matrix.tsv (D-137) + +The `notes-ref` column of `creds-matrix.tsv` points here. Free text lives HERE and never +in the TSV, so the TSV stays logical-keys-only (SEC-004) and greppable. Added by the +12-column amendment 2026-07-26 (`docs/D-137-implementation-plan.md`). + +**Parse contract.** One `## ` heading per note; the body is every line until the next +heading. `scripts/creds-matrix.py --render` emits a note as the manifest comment above the +rows that reference it -- this is how D-137 ruling 2's "prose must not be dropped" +obligation is met for per-row governance text. + +**No secret values, no paths, no hostnames** -- the same discipline as the TSV. Where a +note must refer to a location it names the role token and the register row that holds the +detail (`docs/security-ledger.md` remains the custody/exposure register of record). + +--- + +## n-maas-region-admin +The MAAS region `admin` superuser. Password 28 characters, alphanumeric, NO trailing +newline -- that format spec is the verify-by-format basis; never verify by printing. +The REGION copy is source-of-record and the jumphost copy is a byte-identical working +copy (sha256-verified 2026-07-25). SEC-020 ruling: `admin`'s password was deliberately +NOT rotated, so **any future rotation MUST update both copies in one operation or the +region copy becomes a stale trap**. +This identity is also the automation identity, which is why it carries both a `human` +and a `service` row -- see `n-maas-cli-profile`. That is the ruling-5 violation D-137 +exists to make machine-detectable, not a data-entry error. + +## n-maas-admin-apikey +Deliberately NOT consolidated (SEC-020(i)): trivially regenerable, and a second copy is +pure added surface. `not-consolidated-ruled` records that this is a settled decision, so +`--remote` must not report it as an undeclared find. + +## n-maas-cli-profile +The installer logs the CLI in as profile `admin`; 19 `maas admin ...` call sites across +4 scripts hardcode that profile (SEC-020(b)). ROOT CAUSE of the SEC-020 miss: automation +only ever needed the API key, so nobody ever needed the password, so nothing forced it +into the folder. Remediation is the `operator`/`admin` split begun 2026-07-25 -- migrate +those call sites to a service identity, which is what clears the ruling-5 FAIL. + +## n-maas-region-operator +Minted 2026-07-25 for HUMAN GUI login so a person no longer authenticates as the service +identity. 32-char base64, generated on the jumphost, set via **stdin -- never argv** +(`createadmin --password` would expose it in argv, the shape the installer uses). +The first human-only MAAS identity. + +## n-region-infra-carveout +Region infrastructure, ruled OUT of the consolidation task's scope (SEC-020(i)). Present +at a declared location by decision, not by oversight. + +## n-rack-enroll +Side effect of `maas init region+rack`; consumed by each DC rack at enrollment via a +0600 copy. Mint and consume are different stages -- minted at region standup, consumed at +each DC standup. + +## n-no-mint-command +**Research FINDING 1 -- reproducibility debt.** `ssh-keygen` returns ZERO hits repo-wide; +there is likewise no `mkpasswd`, `htpasswd`, `pwgen`, `uuidgen`, nor any Terraform +`random_password`/`tls_private_key`. This credential's only provenance record is a +manifest comment, so **it is not reproducible**: if the jumphost is rebuilt, no repo +artifact states how to recreate it. That is a Roosevelt-transfer defect, not merely +hygiene. `mint-ref=operator-terminal` rows are the debt to convert, and must be admitted +by the parser rather than treated as errors. + +## n-edge-api +OPNsense REST API key+secret, minted ON the edge via the vendor model (D-113(a2)) into a +0600 temp file that is then wiped, and retrieved to the jumphost creds folder by the +bootstrap driver. + +## n-third-party +Operator-supplied from a third-party control server. Correctly has NO repo mint command -- +this is the one FINDING 1 row that is not a defect. + +## n-netbox-upstream +Minted on the upstream NetBox apex. SEC-006 records this credential as BURNED; the +rotation obligation stands at v1 close. + +## n-netbox-sandbox +**The founding SEC-009 miss.** Minted on the NetBox VM and never pulled into the creds +folder; it went unnoticed until the fidelity re-check needed it. That case is why +`creds-audit` exists at all, and why the source-of-record vs consolidated distinction is +modelled: a VM-minted secret has no forcing function to reach the folder. + +## n-netbox-vm-secrets +Held on the NetBox VM, outside the jumphost creds folder. `netbox-secret-key` was rotated +off netbox-docker's PUBLIC default. The admin password is a human GUI login and is a +separate identity from the service SECRET_KEY -- do not merge these ids. + +## n-tofu-apikey +Office1 MAAS API key consumed by OpenTofu as `TF_VAR_maas_api_key`. + +## n-tfstate-plaintext +DOCFIX-175: the same API key reaches `terraform.tfstate` in PLAINTEXT. A secret-at-rest +location with no mint of its own -- it exists only as a second copy, which is exactly what +the (credential, location) row shape is for. Any rotation of `n-tofu-apikey` must account +for this copy. + +## n-office1-env +Environment file carrying the edge root-password hash and an SSH key path. One of the +three files whose loose, 0664 placement in the home directory WAS SEC-009. + +## n-readme +NOT a credential -- the human index of the creds folder, declared in the manifest so the +folder's undeclared-file check does not fire on it. `principal` is `-` deliberately; the +ruling-5 invariant must skip rows with no principal rather than fault them. + +## n-edge-key +Edge service SSH keypair for the D-112(c) console bootstrap. The runbook describes the +bootstrap step but does NOT contain the key generation itself -- see `n-no-mint-command`. + +## n-dc0-edge-api-absent +**SEC-021, tier-1 detectable.** dc1 declares an edge API credential and dc0 does not, so +this expected row has no manifest row to satisfy it. This is an EXPECTED-BUT-ABSENT +failure, not a matrix error: do not delete the row to make the checker green. + +## n-dc0-power-key-divergence +**SEC-021, per-DC asymmetry.** dc1's power key is consolidated on the jumphost under a +DC-qualified name; dc0's equivalent exists under a DIFFERENT name and on a DIFFERENT +host-role (the headend shadow store, SEC-022). Both the name and the location diverge from +the per-DC pattern, so the expected jumphost rows fail EXPECTED-BUT-ABSENT while the +divergent copy shows up at a declared remote location. + +## n-snap-key-fragility +The private half is installed into the region MAAS snap so MAAS's own power operations can +dial the rack. **The snap path is per-revision and may not survive a snap refresh -- it +must be re-asserted after any refresh** (a standing DC-standup item). Per-DC isolation is +mandatory here: NEVER a cross-DC power key (SEC-012 dc0, SEC-016 dc1). + +## n-dc1-power-key +Dedicated dc1 MAAS->libvirt power key, minted under the SEC-016 ruling "Mint a dedicated +dc1 power key" (per-DC isolation). Roosevelt analog: per-DC IPMI/BMC credentials. + +## n-dc-maas-apikey +Per-DC MAAS admin API key backing the Juju cloud credential. Generated FROM the jumphost +so it never persisted on the region -- which is why its provenance is local rather than a +VM path. Verified by FORMAT ONLY (parts=3, 70 chars); never printed, echoed, or placed in +argv. GRANT is admin-scoped MAAS API. The owning MAAS superuser's password is random and +unstored by ruling -- see `n-juju-user-nopassword`. + +## n-juju-cred-store +A runtime copy lives in the Juju client credential store on the HEADEND, a consequence of +running the Juju client there per D-128. The jumphost file stays canonical. Recorded +off-manifest in the manifests today (SEC-018/-019); this row is what makes it visible to +the checker instead. + +## n-juju-user-nopassword +`createadmin` is invoked with NO `--password`, so the value is stored NOWHERE. Kept random +and unstored BY RULING (2026-07-25): the API key is what is load-bearing, and the +`operator` identity removes any need to log in as this user. `filename` is `-` because +there is no host artifact -- existence checks must skip such rows rather than fail them. + +## n-headend-shadow-store +**SEC-022 -- a scope gap in the control itself, not a one-off miss.** An entire parallel +`*-creds/` store exists on the headend holding real key material. `creds-audit.sh:29-31` +resolves the folder against `$HOME` on whatever host it runs on, and the script contains +no ssh/scp at all, so these stores are structurally invisible to every check that exists +today. The D-128 operating model puts Plane-2 execution on the headend, so headend-resident +credentials are EXPECTED to keep appearing -- this is why ruling 3's declared-locations +list must include the headend, and why adding a location row is DoD for any new mint site. + +## n-outside-convention +**Research FINDING 2.** A credential directory outside the SEC-009 `*-creds/` convention +and outside `creds-audit` entirely -- the audit checks the per-site folders plus a loose +FILE glob in the home directory, and never inspects these DIRECTORIES. Named in CLAUDE.md +as a secret location, so it is KNOWN; it is simply outside the control. Materially extends +SEC-023. The Vault directory holds the cloud's entire root of trust and its mint is an +irreversible one-shot. + +## n-overlay-in-clone +Contains base64 CA cert/key blobs PLUS the issuing-CA passphrase in PLAINTEXT, written +**inside the repo clone** and gitignored. Given SEC-004 (this repo is currently PUBLIC), +that gitignore is the only thing standing between a CA private key and publication. + +## n-tenant-dir +Per-tenant credential directory outside the `*-creds/` convention (research FINDING 2). +Hard tenant isolation applies: D-069 keeps custody off-repo, and D-137 ruling 3 bounds +`--remote` to declared locations precisely so no tenant surface is ever walked. The +runbook `tenant-onboarding-v2-DRAFT.md` documents the SAME credential set as the onboard +script -- two creation locations, ONE set; do not double-count them into extra rows. + +## n-charm-minted +Charm-minted at bundle apply with no repo mint site, then MATERIALIZED to a file by the +operator. In matrix scope per the operator-ruled materialization test (in-matrix if a +credential is ever written to a file, presented by a human, or consumed out-of-band, +regardless of who generated it) -- a generation-based test would have missed it entirely. + +## n-predicted-openrc +**PREDICTED, not present (SEC-023).** The script that writes it runs at a stage this +deployment has not reached, so the checker must report this row as not-yet-expected via an +explicit skip with an `[ok]`, never as a silent pass. Both readings of the acceptance test +apply: this row exercises the not-yet-expected path, and the harness separately proves the +widened sprawl glob CATCHES the file once it appears. + +## n-cloud-only +Lives in the cloud or on a unit rather than in a host file, but is consumed out-of-band +(captured by an operator command at a documented step), so the materialization test puts +it in scope. `filename` may be `-`; existence checks skip such rows. + +## n-ephemeral-unit +Written to a 0600 temp file ON the unit during a stage step. In scope because it is written +to a file, but it is short-lived by design -- absence outside its step is normal, so this +row must never drive an existence failure. + +--- + +## Checker behaviour worth knowing before you "fix" it + +**S5 compares `mint-stage` and `mint-ref` as part of the per-DC shape, deliberately.** +Today both DCs agree on both, so it does not fire on them. A future DC that legitimately +mints the same credential at a DIFFERENT stage -- one standing up after Stage 5, say -- +would trip S5 on a non-defect. That strictness is INTENDED: a per-DC credential appearing +at a different stage or from a different mint site is exactly the drift the register +exists to surface, and it should be argued about rather than absorbed. If a future DC +makes it genuinely wrong, narrow it with a recorded reason -- do not silently drop the two +fields from the comparison. + +**S3 is a partial gate on ruling 2, not a full one.** It compares parsed row fields for +rows present on both sides; it does not compare manifest header prose, and `--render` +emits only the consolidated jumphost rows. Byte-equality becomes enforceable once the +manifests are actually replaced by rendered output, which is operator-gated because it +changes what `creds-audit` demands at three sites. + +--- + +## Deliberate EXCLUSIONS (recorded so the decision is not silent) + +These credentials appear in the research capture but are NOT matrix rows, because they +fail the materialization test -- never written to a file, never presented to a human, +never consumed out-of-band: + +- Per-service DB / RabbitMQ / Ceph keyring credentials -- charm relation data (the capture + confirms 56 apps, zero hardcoded secret options in the bundle). +- Vault-issued per-service TLS certificates -- they cascade from the charm-pki-local root; + the verify script checks them, nothing mints them. +- The charm-pki-local root CA itself -- created inside Vault and never materialized. +- The short-lived (10-minute TTL) child Vault token and the charm authorization token -- + printed once, deliberately so the root token never reaches juju logs. +- CAPI/CAPO per-cluster CAs and bootstrap tokens, and per-cluster trustee users/trusts -- + in-cluster Kubernetes Secrets and cloud-side objects, minted at every cluster create. +- Throwaway acceptance-run credentials -- created and deleted within the same run. +- Tenant end-user passwords and client-minted CI application credentials -- these live on + CLIENT infrastructure, outside this cloud's custody entirely. + +If any of these later becomes materialized to a file, it acquires a row at that moment -- +adding one is DoD for the change that materializes it. diff --git a/creds-matrix.tsv b/creds-matrix.tsv new file mode 100644 index 0000000..c056e56 --- /dev/null +++ b/creds-matrix.tsv @@ -0,0 +1,123 @@ +# creds-matrix.tsv -- D-137 expected-state credential register (ADOPTED 2026-07-26). +# +# THE SINGLE SOURCE. creds-manifests/*.manifest are DERIVED from this file +# (D-137 ruling 2: `scripts/creds-matrix.py --render`); do not hand-edit a manifest. +# +# WHY A FORWARD REGISTER. creds-audit is DECLARATION-based, so an undeclared secret is +# structurally invisible -- and no discovery sweep can ever detect a credential that was +# never minted at all. Absence is invisible to discovery. This file states what SHOULD +# exist so absence becomes a detectable FAIL. +# +# LOGICAL KEYS ONLY (SEC-004 -- this repo is currently PUBLIC). No values, no hostnames, +# no IPs, no grants, no paths. `host-role` is a role token, never a VM name. +# +# FORMAT. Whitespace-separated positional records in the manifests' idiom (the same shape +# creds-audit.sh:49 reads). `#` comments and blank lines ignored. NO FIELD MAY CONTAIN +# WHITESPACE. `-` means not-applicable. +# +# A ROW IS ONE (credential, location) PAIR -- not one credential (12-column amendment, +# 2026-07-26). Copies of one credential share an `id`, one row per place it lives. `id` is +# therefore the IDENTITY, which is what makes ruling 5 checkable: if the rows sharing an id +# carry more than one `principal`, that identity serves two principal types -> FAIL. +# +# columns: +# id logical identity (rows sharing an id are copies/forms of one identity) +# cardinality singleton | per-site | per-DC | per-node | per-tenant +# site-key region-qualified (vr1-dc0), bare `dc0` REJECTED; `-` = not site-scoped +# host-role jumphost|headend|rack|edge|netbox|cloud|unit|client (NEVER a VM name) +# filename basename at that location; `-` = no host artifact (skips existence checks) +# access-type gui|api|ssh|cli-profile|console|none +# principal human|service|- (ruling 5 keys off this) +# custody consolidated|source-of-record|off-manifest-known|not-consolidated-ruled +# mint-stage stage2..stage6 | vr0-phaseNN | tenant-onboard | adhoc | - +# mint-ref script:: | runbook:: | operator-terminal | - +# sec-ref SEC-NNN | - +# notes-ref key into creds-matrix-notes.md | - +# +# EXPECTED RED. This register is deliberately not satisfiable by today's tree; the +# failures are the defects D-137 exists to surface (see the notes file, and the +# "EXPECTED RED BY DESIGN" section of docs/D-137-implementation-plan.md). +# +# id cardinality site-key host-role filename access-type principal custody mint-stage mint-ref sec-ref notes-ref + +# ---------------------------------------------------------------- vr1-office1 (headend region + jumphost) +maas-region-admin singleton vr1-office1 headend admin.pass gui human source-of-record stage2 script:scripts/site-headend-install.sh:452 SEC-020 n-maas-region-admin +maas-region-admin singleton vr1-office1 jumphost maas-admin-password gui human consolidated stage2 script:scripts/site-headend-install.sh:452 SEC-020 n-maas-region-admin +maas-region-admin singleton vr1-office1 headend admin.apikey api service not-consolidated-ruled stage2 script:scripts/site-headend-install.sh:453 SEC-020 n-maas-admin-apikey +maas-region-admin singleton vr1-office1 headend maas-cli-profile cli-profile service off-manifest-known stage2 script:scripts/site-headend-install.sh:455 SEC-020 n-maas-cli-profile +maas-region-operator singleton vr1-office1 jumphost maas-operator-password gui human consolidated adhoc operator-terminal SEC-020 n-maas-region-operator +maas-region-db singleton vr1-office1 headend db.pass none service not-consolidated-ruled stage2 script:scripts/site-headend-install.sh:430 SEC-020 n-region-infra-carveout +maas-region-lxd-trust singleton vr1-office1 headend lxd-trust.pass none service not-consolidated-ruled stage2 script:scripts/site-headend-install.sh:485 SEC-020 n-region-infra-carveout +maas-rack-enroll-secret singleton vr1-office1 headend maas-enroll-secret none service off-manifest-known stage2 script:scripts/site-headend-install.sh:445 - n-rack-enroll +office1-svc-key per-site vr1-office1 jumphost office1_svc_ed25519 ssh service consolidated stage2 operator-terminal SEC-007 n-no-mint-command +office1-svc-key per-site vr1-office1 jumphost office1_svc_ed25519.pub none service consolidated stage2 operator-terminal SEC-007 n-no-mint-command +office1-edge-api per-site vr1-office1 jumphost opnsense-api.txt api service consolidated stage2 script:scripts/opnsense-bootstrap-apikey.sh:89 - n-edge-api +office1-edge-root per-site vr1-office1 jumphost opnsense-root-password gui human consolidated stage2 operator-terminal SEC-007 n-no-mint-command +office1-edge-root per-site vr1-office1 jumphost opnsense-root-hash none human consolidated stage2 operator-terminal SEC-007 n-no-mint-command +office1-tailscale-authkey singleton vr1-office1 jumphost tailscale-authkey.txt api service consolidated stage2 operator-terminal SEC-008 n-third-party +netbox-upstream-token singleton vr1-office1 jumphost vr1-netbox.env api service consolidated stage2 operator-terminal SEC-006 n-netbox-upstream +netbox-sandbox-token singleton vr1-office1 netbox api.token api service source-of-record stage2 runbook:runbooks/dc-dc-phase1-office1-standup.md:643 SEC-009 n-netbox-sandbox +netbox-sandbox-token singleton vr1-office1 jumphost vr1-netbox-sandbox.env api service consolidated stage2 runbook:runbooks/dc-dc-phase1-office1-standup.md:643 SEC-009 n-netbox-sandbox +netbox-secret-key singleton vr1-office1 netbox netbox-secret-key none service off-manifest-known stage2 operator-terminal - n-netbox-vm-secrets +netbox-admin singleton vr1-office1 netbox netbox-admin-password gui human off-manifest-known stage2 operator-terminal - n-netbox-vm-secrets +office1-tofu-maas-apikey singleton vr1-office1 jumphost vr1-stage1.env api service consolidated stage2 runbook:runbooks/dc-dc-phase1-office1-standup.md:464 SEC-009 n-tofu-apikey +office1-tofu-maas-apikey singleton vr1-office1 jumphost terraform.tfstate api service off-manifest-known stage2 runbook:runbooks/dc-dc-phase1-office1-standup.md:464 SEC-009 n-tfstate-plaintext +office1-env singleton vr1-office1 jumphost vr1-office1.env none service consolidated stage2 operator-terminal SEC-009 n-office1-env +creds-folder-readme per-site vr1-office1 jumphost README none - consolidated - - - n-readme + +# ---------------------------------------------------------------- vr1-dc0 +dc0-svc-key per-DC vr1-dc0 jumphost vr1-dc0_svc_ed25519 ssh service consolidated stage3 operator-terminal - n-no-mint-command +dc0-svc-key per-DC vr1-dc0 jumphost vr1-dc0_svc_ed25519.pub none service consolidated stage3 operator-terminal - n-no-mint-command +dc0-svc-key per-DC vr1-dc0 headend vr1-dc0_svc_ed25519 ssh service off-manifest-known stage3 operator-terminal SEC-022 n-headend-shadow-store +dc0-svc-key per-DC vr1-dc0 headend vr1-dc0_svc_ed25519.pub none service off-manifest-known stage3 operator-terminal SEC-022 n-headend-shadow-store +dc0-edge-key per-DC vr1-dc0 jumphost vr1-dc0-edge_ed25519 ssh service consolidated stage3 operator-terminal - n-edge-key +dc0-edge-key per-DC vr1-dc0 jumphost vr1-dc0-edge_ed25519.pub none service consolidated stage3 operator-terminal - n-edge-key +dc0-edge-api per-DC vr1-dc0 jumphost opnsense-api.txt api service consolidated stage3 script:scripts/opnsense-bootstrap-apikey.sh:89 SEC-021 n-dc0-edge-api-absent +dc0-maas-power-key per-DC vr1-dc0 jumphost vr1-dc0-maas-power_ed25519 ssh service consolidated stage3 operator-terminal SEC-021 n-dc0-power-key-divergence +dc0-maas-power-key per-DC vr1-dc0 jumphost vr1-dc0-maas-power_ed25519.pub none service consolidated stage3 operator-terminal SEC-021 n-dc0-power-key-divergence +dc0-maas-power-key per-DC vr1-dc0 headend maas-virsh_ed25519 ssh service off-manifest-known stage3 operator-terminal SEC-021 n-dc0-power-key-divergence +dc0-maas-power-key per-DC vr1-dc0 headend maas-snap-power-key ssh service off-manifest-known stage3 runbook:runbooks/dc-dc-phase2-tofu-dc-substrate.md:770 SEC-012 n-snap-key-fragility +dc0-maas-apikey per-DC vr1-dc0 jumphost maas-api-key.txt api service consolidated stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:136 SEC-018 n-dc-maas-apikey +dc0-maas-apikey per-DC vr1-dc0 headend juju-credentials api service off-manifest-known stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:141 SEC-018 n-juju-cred-store +dc0-juju-maas-user per-DC vr1-dc0 - - api service off-manifest-known stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:128 SEC-020 n-juju-user-nopassword + +# ---------------------------------------------------------------- vr1-dc1 +dc1-svc-key per-DC vr1-dc1 jumphost vr1-dc1_svc_ed25519 ssh service consolidated stage3 operator-terminal - n-no-mint-command +dc1-svc-key per-DC vr1-dc1 jumphost vr1-dc1_svc_ed25519.pub none service consolidated stage3 operator-terminal - n-no-mint-command +dc1-svc-key per-DC vr1-dc1 headend vr1-dc1_svc_ed25519 ssh service off-manifest-known stage3 operator-terminal SEC-022 n-headend-shadow-store +dc1-svc-key per-DC vr1-dc1 headend vr1-dc1_svc_ed25519.pub none service off-manifest-known stage3 operator-terminal SEC-022 n-headend-shadow-store +dc1-edge-key per-DC vr1-dc1 jumphost vr1-dc1-edge_ed25519 ssh service consolidated stage3 operator-terminal SEC-015 n-edge-key +dc1-edge-key per-DC vr1-dc1 jumphost vr1-dc1-edge_ed25519.pub none service consolidated stage3 operator-terminal - n-edge-key +dc1-edge-api per-DC vr1-dc1 jumphost opnsense-api.txt api service consolidated stage3 script:scripts/opnsense-bootstrap-apikey.sh:89 - n-edge-api +dc1-maas-power-key per-DC vr1-dc1 jumphost vr1-dc1-maas-power_ed25519 ssh service consolidated stage3 operator-terminal SEC-016 n-dc1-power-key +dc1-maas-power-key per-DC vr1-dc1 jumphost vr1-dc1-maas-power_ed25519.pub none service consolidated stage3 operator-terminal SEC-016 n-dc1-power-key +dc1-maas-power-key per-DC vr1-dc1 headend maas-snap-power-key ssh service off-manifest-known stage3 runbook:runbooks/dc-dc-phase2-tofu-dc-substrate.md:770 SEC-016 n-snap-key-fragility +dc1-maas-apikey per-DC vr1-dc1 jumphost maas-api-key.txt api service consolidated stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:136 SEC-019 n-dc-maas-apikey +dc1-maas-apikey per-DC vr1-dc1 headend juju-credentials api service off-manifest-known stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:141 SEC-019 n-juju-cred-store +dc1-juju-maas-user per-DC vr1-dc1 - - api service off-manifest-known stage5 runbook:runbooks/dc-dc-phase4-juju-bundle-per-dc.md:128 SEC-020 n-juju-user-nopassword + +# ---------------------------------------------------------------- outside the SEC-009 *-creds/ convention (research FINDING 2) +vault-unseal-shares singleton - jumphost vault-init none service off-manifest-known vr0-phase02 runbook:runbooks/phase-02-vault-bringup.md:71 SEC-023 n-outside-convention +vault-root-token singleton - jumphost vault-init none service off-manifest-known vr0-phase02 runbook:runbooks/phase-02-vault-bringup.md:71 SEC-023 n-outside-convention +octavia-issuing-ca-passphrase singleton - jumphost octavia-issuing-ca-passphrase none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:299 SEC-023 n-outside-convention +octavia-issuing-ca-key singleton - jumphost octavia-issuing-ca-key none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:302 SEC-023 n-outside-convention +octavia-issuing-ca-cert singleton - jumphost octavia-issuing-ca-cert none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:305 SEC-023 n-outside-convention +octavia-controller-ca-pass singleton - jumphost octavia-controller-ca-passphrase none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:321 SEC-023 n-outside-convention +octavia-controller-ca-key singleton - jumphost octavia-controller-ca-key none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:324 SEC-023 n-outside-convention +octavia-controller-ca-cert singleton - jumphost octavia-controller-ca-cert none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:327 SEC-023 n-outside-convention +octavia-controller-key singleton - jumphost octavia-controller-key none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:343 SEC-023 n-outside-convention +octavia-controller-bundle singleton - jumphost octavia-controller-bundle none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:375 SEC-023 n-outside-convention +octavia-pki-overlay singleton - jumphost octavia-pki.yaml none service off-manifest-known vr0-phase01 runbook:runbooks/phase-01-bundle-deploy.md:391 SEC-004 n-overlay-in-clone +tenant-domain-admin per-tenant - jumphost domain-admin-cred.txt gui human off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:64 SEC-023 n-tenant-dir +tenant-cluster-user per-tenant - jumphost cluster-cred.txt api human off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:90 SEC-023 n-tenant-dir +tenant-svc-user per-tenant - jumphost svc-cred.txt api service off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:90 SEC-023 n-tenant-dir +tenant-svc-appcred per-tenant - jumphost svc-appcred.txt api service off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:141 SEC-023 n-tenant-dir +tenant-nova-key per-tenant - jumphost tenant-key.pem ssh service off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:167 SEC-023 n-tenant-dir + +# ---------------------------------------------------------------- in scope by the materialization ruling (charm/platform-minted, then materialized) +keystone-admin singleton - cloud - api service off-manifest-known vr0-phase01 - - n-charm-minted +keystone-admin singleton - jumphost admin-openrc none service off-manifest-known vr0-phase03 script:scripts/phase-03-admin-openrc.sh:52 SEC-023 n-predicted-openrc +magnum-domain-admin singleton - cloud - api service off-manifest-known vr0-phase07 script:scripts/phase-07-conductor-graft.sh:90 - n-cloud-only +capi-mgmt-kubeconfig singleton - cloud kube-config none service off-manifest-known vr0-phase06 script:scripts/phase-06-capi-stack.sh:91 - n-cloud-only +rbd-mirror-peer-token per-DC - unit rbd-mirror-bootstrap-token none service off-manifest-known stage6 script:scripts/dc-dc-rbd-mirror.sh:88 - n-ephemeral-unit +rgw-multisite-system-key singleton - unit - api service off-manifest-known stage6 script:scripts/dc-dc-radosgw-multisite.sh:172 - n-cloud-only diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index fb96c21..042d927 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -815,6 +815,32 @@ per-row inventory (55 MINT rows with file:line, host, destination, stage and human/service classification), transcribed in-repo 2026-07-26 so the matrix SEED does not depend on a session transcript. + **TIER 1 BUILT 2026-07-26** (offline/STATIC half only; tiers 2-3 NOT built, and the + preflight `Pn` of ruling 1 is NOT wired -- see the sequencing question below). Shipped: + `creds-matrix.tsv` (72 rows), `creds-matrix-notes.md`, `scripts/creds-matrix.py` + (S1 schema / S2 manifest coverage both-bounds / S3 render drift / S4 mint-ref + resolution / S5 per-DC symmetry / S6 ruling-5 principal invariant / S7 notes + integrity), harness `tests/creds-matrix/run-tests.sh` **24/24**; gauntlet **ALL GREEN + (80)**, repo-lint 0-fail. **SCHEMA AMENDED 10 -> 12 columns** (operator-ruled + 2026-07-26, "go with the 12-column amendment"): `custody` + `notes-ref` added and rows + re-keyed to (credential, location), because a read-first round-trip check proved the + 10-column form could not carry what ruling 2 forbids dropping -- and because ruling 3 + puts four deliberate, reasoned credential copies INSIDE `--remote`'s declared + locations, where `creds-audit.sh:63-67` would report every one as UNDECLARED. Detail + + rationale in `docs/D-137-implementation-plan.md`; OPS under GA-R3, the five sub-rulings + are untouched. **RED BY DESIGN AND CORRECTLY SO -- 5 findings, do not "fix" by deleting + rows:** the ruling-5 identity conflation on `maas-region-admin` (its own SEC-020 + defect), 3x EXPECTED-BUT-ABSENT for SEC-021 (dc0 declares neither an edge API + credential nor a jumphost-consolidated power key, both of which dc1 declares), and the + S5 asymmetry of dc0's divergently-named headend power key. 27 rows carry + `mint-ref=operator-terminal` = the research FINDING 1 reproducibility debt, admitted and + counted, not faulted. **Acceptance test PARTIALLY met** (corrected from the plan's + original text): SEC-021's DECLARATION half is tier-1 detectable and is reproduced; + SEC-022/-023 and SEC-021's on-disk half need tier 2. **OPEN SEQUENCING QUESTION for the + operator:** ruling 1 lands the check as a BLOCKING preflight `Pn` and the first run is + red, so wiring it hard-fails `preflight.sh` until the credential defects are remediated; + wiring-as-ruled is the default and deferring until after per-row remediation is the + departure. Not decided; nothing wired either way. ## 9. Additional defect found while authoring (FIXED in sweep Batch 0.3, 2026-07-19 -- wrap-aware exclusion, GA-F15; history below) diff --git a/docs/D-137-implementation-plan.md b/docs/D-137-implementation-plan.md index b14549f..8a84118 100644 --- a/docs/D-137-implementation-plan.md +++ b/docs/D-137-implementation-plan.md @@ -50,7 +50,12 @@ `creds-matrix.tsv`, positional records in the manifests' idiom (do NOT invent a format language; `scripts/site-ssh-config.sh:44-50` uses the same shape): - id | cardinality | site-key | host-role | filename | access-type | principal | mint-stage | mint-ref | sec-ref + id | cardinality | site-key | host-role | filename | access-type | principal | custody | mint-stage | mint-ref | sec-ref | notes-ref + +Whitespace-separated positional records -- the same shape `creds-audit.sh:49` already +reads, which is what "the manifests' idiom" means; the file is column-aligned for human +review and NO FIELD MAY CONTAIN WHITESPACE. `#` comments and blank lines ignored; `-` +means not-applicable. The doc renders the columns pipe-delimited for readability only. - `cardinality`: `singleton` | `per-site` | `per-DC` | `per-node` | `per-tenant`. REQUIRED for Roosevelt transfer -- without it the matrix cannot express "9 nodes". @@ -64,7 +69,53 @@ `operator-terminal`. Capture FINDING 1: `ssh-keygen` returns ZERO hits repo-wide, so 12 declared secrets have NO mint command; those rows are the DEBT the matrix exists to surface and must be admitted, not treated as parse errors. -- NO grants, hostnames, IPs, custody detail, or values (ruling: logical keys only). +- NO grants, hostnames, IPs, or values (ruling: logical keys only). + +### AMENDMENT 2026-07-26 -- 10 columns -> 12; rows keyed by (credential, location) + +Operator-approved 2026-07-26 ("go with the 12-column amendment") after a read-first +round-trip check of the three manifests against the 10-column schema found it could not +carry what ruling 2 forbids dropping. This is ruling 2 being IMPLEMENTED, not departed +from -- its own words are that the prose "must become MATRIX FIELDS". OPS under GA-R3 +(mechanism detail of an adopted decision, no Roosevelt-delta of its own); D-137's five +sub-rulings are untouched and remain the authority. + +**A row is one (credential, location) pair, not one credential.** A credential with +copies in several places gets one row per place, sharing an `id`. This was already +`host-role`'s stated intent ("expresses that the copy is on the wrong role") and it is +what makes the SEC-020 stale-trap machine-checkable: tier 3 cannot compare digests across +hosts without knowing which locations hold the same `id` and which one is authoritative. + +**`custody`** (NEW) -- `consolidated` | `source-of-record` | `off-manifest-known` | +`not-consolidated-ruled`. A four-token logical enum, NOT the "custody detail" the original +bullet excludes: no paths, no hostnames, no holders. It exists because ruling 3 puts +`/root/maas-secrets/` and the headend shadow stores INSIDE `--remote`'s declared +locations, so `creds-audit.sh:63-67`'s undeclared-file check will visit four credential +copies that are deliberate, reasoned decisions and report every one as UNDECLARED: + - the Juju client credential store copies of both DCs' `maas-api-key.txt` + (`vr1-dc0.manifest:27-28`, `vr1-dc1.manifest:34-35`; also SEC-018); + - the MAAS snap copy of the dc1 power key (`vr1-dc1.manifest:27-28`; also SEC-016); + - `voffice1:/root/maas-secrets/{admin.apikey,db.pass,lxd-trust.pass}`, ruled + not-consolidated with per-file reasons (`vr1-office1.manifest:24-27`; SEC-020(i)). +Without the column the checker either fails forever on settled questions or the decisions +get deleted to silence it. `source-of-record` additionally carries SEC-020's ruling that +the REGION copy of the `admin` password is authoritative and the jumphost copy is the +working copy -- the fact the stale-trap warning depends on. + +**`notes-ref`** (NEW) -- an `id`-keyed pointer into `creds-matrix-notes.md`, or `-`. +Free text lives THERE, never in the TSV, so the TSV stays logical-keys-only and greppable. +`--render` emits the referenced note as the manifest comment above its row, which is how +ruling 2's "regenerated into the rendered header" obligation is met for per-row prose. +Load-bearing content this preserves: mint dates; the not-reproducible debt on the 12 +`operator-terminal` rows; format specifications used for verify-by-format (e.g. the +`admin` password's 28 chars, alphanumeric, NO trailing newline); and the reason text on +every `not-consolidated-ruled` / `off-manifest-known` row. + +**Knowledge-loss qualifier (measured, do not overstate the gap).** Most of this prose is +ALREADY duplicated in `docs/security-ledger.md` -- SEC-018 carries the Juju-store copy, +SEC-020 carries source-of-record, the stale-trap warning, and all three carve-out reasons. +The real defect the two columns fix is that the CHECKER had no machine-readable field for +facts that are otherwise well recorded, not that the facts were about to be lost. ## Checker: `scripts/creds-matrix.py` @@ -116,7 +167,8 @@ ## Files -NEW: `creds-matrix.tsv`, `creds-manifests/vm-secret-locations`, `scripts/creds-matrix.py`, +NEW: `creds-matrix.tsv`, `creds-matrix-notes.md` (the 2026-07-26 amendment's `notes-ref` +target), `creds-manifests/vm-secret-locations`, `scripts/creds-matrix.py`, `tests/creds-matrix/run-tests.sh`. MODIFY: `scripts/preflight.sh` (new `Pn`), `scripts/creds-audit.sh` (`--all`, `--remote`, VERIFY provenance, widen sprawl globs -- currently miss `admin.pass`, `*.apikey`, `*.key`, @@ -129,6 +181,20 @@ - **ACCEPTANCE TEST:** `scripts/creds-matrix.py --all` must reproduce SEC-021/-022/-023 and the openrc case as NAMED failures on today's tree. A checker that goes green on the current tree is wrong. + - **Tier boundary (clarified 2026-07-26; corrected after the matrix was seeded).** + SEC-022 and SEC-023 are pure EXISTENCE failures needing tier 2's remote reads across + three sites. **SEC-021 SPLITS across the tiers:** its DECLARATION asymmetry -- dc1 + declares an edge API credential and a jumphost-consolidated power key, dc0 declares + neither -- is caught at TIER 1 by the per-DC symmetry and manifest-coverage checks; only + its on-disk absence needs tier 2. So tier 1 + its harness partially satisfies the + acceptance test and must be reported that way -- neither as fully satisfying it nor as + unable to touch it. + - **The `admin-openrc` case resolves to BOTH readings, which are complementary.** + `~/admin-openrc` is PREDICTED, not present: `phase-03-admin-openrc.sh` runs at Stage 5, + not yet reached. So it is (a) a matrix row whose `mint-stage` is in the future, which + the checker reports as not-yet-expected via the explicit-skip-with-`[ok]` pattern + (`provider-bundle-check.py:281`) rather than a silent pass, AND (b) a harness fixture + proving the widened sprawl glob CATCHES it when it does appear. Build both. - `bash scripts/run-tests-all.sh` ALL GREEN; `bash scripts/repo-lint.sh` 0-fail; `bash scripts/preflight.sh` exercises the new `Pn`. @@ -154,4 +220,12 @@ their verbatim operator selections), then this file. 3. Read the research capture `docs/audit/creds-creation-points-20260725.md` -- it is the matrix SEED (55 MINT sites classified by host, destination, stage, human/service). + **55 mint SITES is not 55 rows** (clarified 2026-07-26): A1/A10 are generator helpers, + not credentials; A21-A25 are flagged in the capture itself as the SAME set as A11-A14 + ("do not double-count"); B30 is explicitly not a mint; A8+A9 are one credential minted + then retrieved; and `per-tenant`/`per-node`/`per-cluster` sets collapse to ONE row under + `cardinality`. Conversely the materialization ruling ADDS rows the inventory excluded as + MATERIALIZE (the Keystone `admin` password, `~/admin-openrc`), and the 12 no-mint + declared secrets of FINDING 1 are rows with `mint-ref=operator-terminal`. Derive the row + set from credential IDENTITY; never transcribe the table 1:1. 4. Build tier 1 first (offline, no live access needed), harness it, then tier 2, then tier 3. diff --git a/docs/changelog-20260726-d137-tier1.md b/docs/changelog-20260726-d137-tier1.md new file mode 100644 index 0000000..716badd --- /dev/null +++ b/docs/changelog-20260726-d137-tier1.md @@ -0,0 +1,158 @@ +# Session changelog 2026-07-26 -- D-137 credential matrix, tier 1 (STATIC) + +Session-scoped scratch (GA-R2/D1). NOT a status or decision authority -- status lives only +in `docs/CURRENT-STATE.md`, rulings only in `docs/design-decisions.md`. Consolidated at +stage close. + +Branch `dc-dc-stage4-phase3-maas-deploy`. No cloud state touched: every item below is +offline repo work. Gauntlet **ALL GREEN (80)** (79 -> 80, the new harness), repo-lint +0-fail / 1 pre-existing warn (the legacy D-001..018 ASCII carve-out). + +--- + +## Item 1 -- D-137 matrix schema amended 10 -> 12 columns (operator-ruled) + +**What.** `docs/D-137-implementation-plan.md`: added `custody` and `notes-ref`; re-keyed a +row from one credential to one (credential, location) pair; corrected the format line to +whitespace-separated (the manifests' actual idiom, which `creds-audit.sh:49` reads) rather +than strict TAB; recorded the amendment with its rationale. + +**Why.** A read-first round-trip check of the three checked-in manifests against the +10-column schema found it could not carry three classes of governance prose that D-137 +ruling 2 forbids dropping: +- **off-manifest carve-outs** -- four deliberate, reasoned credential copies (both DCs' + Juju client-store keys, the dc1 MAAS-snap power key, and the three ruled + not-consolidated region files). Ruling 3 puts their locations INSIDE `--remote`'s + declared set, so `creds-audit.sh:63-67` will visit every one and report it UNDECLARED. + Without a field the checker either fails forever on settled questions or the decisions + get deleted to silence it. +- **multi-copy credentials** -- one credential, several locations, DIFFERENT filenames + (`admin.pass` vs `maas-admin-password`), one designated authoritative copy. Tier 3's + digest comparison cannot know what to compare without it. +- **verification-relevant detail** -- format specs, mint dates, the not-reproducible debt. + +Ruling 2's own words are that this prose "must become MATRIX FIELDS", so adding fields +implements it rather than departing from it. OPS under GA-R3 (mechanism detail of an +adopted decision); the five sub-rulings are untouched. + +**Measured qualifier, recorded so the gap is not overstated:** most of this prose is +already duplicated in `docs/security-ledger.md` (SEC-018 carries the Juju-store copy; +SEC-020 carries source-of-record, the stale-trap warning, and all three carve-out +reasons). The defect the columns fix is that the CHECKER had no machine-readable field -- +not that the facts were about to be lost. + +**Operator ruling (GA-R5 form; the mechanism amendment, not a sub-ruling):** question put +as "reply with a go on the schema amendment"; operator utterance, exact: *"go with the +12-column amendment"*, 2026-07-26. + +**Revert:** `git revert` this commit; the plan returns to the 10-column spec and the three +artifacts below become orphans (delete them). + +## Item 2 -- `creds-matrix.tsv` + `creds-matrix-notes.md` (the register) + +**What.** 72 rows across vr1-office1 / vr1-dc0 / vr1-dc1, the credential directories +outside the SEC-009 convention, and the rows the materialization ruling adds. 32 note keys. + +**Why the row count is not 55.** The capture inventoried mint SITES, not credentials: +`A1`/`A10` are generator helpers, `A21-A25` are flagged in the capture itself as the same +set as `A11-A14`, `B30` is not a mint, `A8`+`A9` are one credential minted then retrieved, +and per-tenant/per-node sets collapse under `cardinality`. Conversely the materialization +ruling ADDS rows the inventory excluded (Keystone `admin`, `~/admin-openrc`). Derived from +identity, never transcribed 1:1. + +**Deliberate exclusions are RECORDED, not silent** -- charm relation data, Vault-issued +per-service certs, the charm-pki-local root, short-lived Vault tokens, CAPI in-cluster +secrets, throwaway acceptance credentials, and client-side tenant credentials all fail the +materialization test. The reasoning is in the notes file so a later reader can challenge +it. + +**Revert:** delete both files. + +## Item 3 -- `scripts/creds-matrix.py` tier 1 (STATIC) + +**What.** Seven check groups in the `provider-bundle-check.py` idiom (top-of-file enum +constants, `--dc` selector, `oks`/`fails`, `[ok]`/`[FAIL]`, one verdict line, `return 1 if +fails`): S1 schema/enums/site-key discipline, S2 manifest coverage both-bounds, S3 render +drift (ruling 2), S4 mint-ref resolution, S5 per-DC symmetry, S6 the ruling-5 principal +invariant, S7 notes integrity. `--render` prints derived manifests; it does NOT write. + +**Two design decisions worth challenging at review:** +- The manifest **mode column is DERIVED** (`.pub` -> 644, else 600) rather than becoming a + 13th column. That rule reproduces all three checked-in manifests exactly. +- **`--render` deliberately does not write.** Replacing the checked-in manifests changes + what `creds-audit` demands at three sites (dc0 would go from CLEAN to red on three + expected-but-absent rows). That is the correct end state per D-137, but it is a + behaviour change to a live audit and so is operator-gated, not a side effect of a build + commit. `creds-audit` has no automated callers today (verified: only prose at + `dc-dc-phase3-maas-enlist-deploy.md:498`), so nothing is destabilised by waiting. + +**Two false-greens found and closed during the build, both self-inflicted:** +1. **S2 enumerated only the sites the MATRIX mentions**, so an empty or site-truncated + matrix examined no manifest, found nothing undeclared, and reported "both bounds clean". + That is precisely the defect `netbox/sandbox-fidelity-check.py:131-143` records in-repo + ("an upper bound masquerading as an assertion. It must be BOTH bounds") reproduced one + level up. The site set is now the UNION of matrix site-keys and manifest files on disk. + Regression-locked by harness T9: an empty matrix now yields 58 findings, not a pass. +2. The uniqueness key was (site, host-role, filename), which false-failed two DISTINCT + identities sharing one file -- the Vault init artifact holds both the unseal shares and + the root token. Key now includes `id`. Regression-locked by T6. + +**Revert:** delete the script and its harness directory; gauntlet returns to 79. + +## Item 4 -- `tests/creds-matrix/run-tests.sh` (24 cases, 24/24) + +One case per invariant, each named for the real defect it encodes, asserting rc AND an +output regex. Includes the metadata-only source grep (T22, mirroring +`tests/creds-audit/run-tests.sh:71-73`) and **T24, which asserts the real tree stays RED** +-- a guard against someone making the register green by deleting the rows that fail. + +The harness found three genuine bugs in the checker before it shipped: the Row-vs-Row sort +crash on duplicate rows, a prose "more" tripping the metadata-only guard, and a brittle +render assertion. All three fixed. + +**Revert:** delete `tests/creds-matrix/`. + +## Item 5 -- corrected a wrong `sec-ref` before it could be cited as authority + +`dc1-edge-api` carried `SEC-015`. Verified against `docs/security-ledger.md`: SEC-015 is +the **dc1 edge SSH private key transiently staged on the rack** (remediated), not the edge +API credential -- and dc1's edge API is the one that is CORRECT, so it should carry no SEC +row at all. Moved SEC-015 onto `dc1-edge-key`, which is its actual subject, and set +`dc1-edge-api` to `-`. Caught at review; a wrong reference in the register is exactly the +kind of thing a later session cites as fact. + +Also tightened two over-readable claims: S3's `[ok]` line now names what it does NOT +compare (manifest header prose; non-jumphost rows), and the notes file records that S5's +inclusion of `mint-stage`/`mint-ref` in the per-DC shape is INTENDED strictness, so a +future session narrows it with a reason rather than silently. + +--- + +## EXPECTED RED -- the 5 findings are the deliverable, not a defect in it + +`python3 scripts/creds-matrix.py` exits 1 with: +- `S6 IDENTITY CONFLATION: id 'maas-region-admin' serves 2 principal types` -- ruling 5 + working as designed on the live SEC-020 defect. Clears when the 19 `maas admin` call + sites migrate to a service identity. +- `S2 ... EXPECTED-BUT-ABSENT` x3 (dc0 `opnsense-api.txt`, `vr1-dc0-maas-power_ed25519`, + `.pub`) -- SEC-021's declaration half. +- `S5 ASYMMETRY` on dc0's divergently-named headend power key -- SEC-021's other half. + +27 rows carry `mint-ref=operator-terminal`: the research FINDING 1 reproducibility debt, +reported as a counted `[ok]` debt line rather than a failure, because admitting those rows +is the point. + +Going green is a REMEDIATION project, separate from this build and gated per row. + +## NOT done in this session (stated so the boundary is unambiguous) + +- Tier 2 (EXISTENCE) and tier 3 (VALIDITY) -- not built. `--all` names them as unbuilt + with an explicit `[ok]`; they never pass silently. +- `creds-manifests/vm-secret-locations` (ruling 3) -- not written. Two `--render` source + fields depend on it and are explicitly skipped as pending, not silently matched. +- The preflight `Pn` of ruling 1 -- **not wired**, and this is an OPEN QUESTION for the + operator, not an omission: wiring it blocking (the default, since it is what was ruled) + hard-fails `preflight.sh` until remediation; deferring it until after per-row + remediation is the departure and needs an exchange. +- SEC-009's demotion to a pointer (ruling 4) -- not done; it belongs with tier 2. +- No manifest was rewritten; no cloud state was touched. diff --git a/scripts/creds-matrix.py b/scripts/creds-matrix.py new file mode 100644 index 0000000..2ab7baa --- /dev/null +++ b/scripts/creds-matrix.py @@ -0,0 +1,462 @@ +#!/usr/bin/env python3 +""" +creds-matrix.py -- D-137 credential-matrix checker (ADOPTED 2026-07-26). + +The matrix (`creds-matrix.tsv`) is the EXPECTED-state register of every credential this +cloud is supposed to have. This script reconciles it against what the repo actually +declares. It exists because `creds-audit` is DECLARATION-based -- the manifest IS the +declaration, so an undeclared secret is structurally invisible -- and because no discovery +sweep can EVER detect a credential that was never minted at all. Absence is invisible to +discovery; only a forward register makes it a FAIL. + +THREE TIERS, increasing cost (plan: docs/D-137-implementation-plan.md): + 1. STATIC offline; runs in the gauntlet <-- IMPLEMENTED HERE + 2. EXISTENCE local + remote reads; runs as a preflight gate <-- NOT BUILT YET + 3. VALIDITY provenance digests + behavioral probes <-- NOT BUILT YET +Tiers 2 and 3 SELF-SKIP with an explicit [ok] naming them as unbuilt. They never pass +silently -- a silent pass is the false-green this whole design exists to prevent. + +METADATA ONLY. This script reads the matrix, the manifests, and the notes file. It never +opens a credential, and it must never gain a content-reading verb (its harness greps the +source to enforce that, mirroring tests/creds-audit/run-tests.sh:71-73). + +EXPECTED RED BY DESIGN. A clean run against today's tree would mean the checker is wrong. +See the plan's "EXPECTED RED BY DESIGN" section: the ruling-5 identity conflation, the 12 +operator-terminal reproducibility-debt rows, and SEC-021's per-DC divergence are DEFECTS +the register surfaces, not bugs to silence. Do not delete a row to go green. + +Usage: + python3 scripts/creds-matrix.py # tier 1 (static) -- the gauntlet entry + python3 scripts/creds-matrix.py --all # every implemented tier + skip notices + python3 scripts/creds-matrix.py --dc vr1-dc1 # narrow the per-DC checks to one DC + python3 scripts/creds-matrix.py --render # print manifests rendered from the matrix +Exit: 0 clean, 1 findings, 2 usage/IO error. ASCII-only output. +""" +import sys, os, re, argparse + +COLUMNS = ("id", "cardinality", "site-key", "host-role", "filename", "access-type", + "principal", "custody", "mint-stage", "mint-ref", "sec-ref", "notes-ref") +NCOL = len(COLUMNS) + +CARDINALITY = {"singleton", "per-site", "per-DC", "per-node", "per-tenant"} +HOST_ROLE = {"jumphost", "headend", "rack", "edge", "netbox", "cloud", "unit", "client", "-"} +ACCESS_TYPE = {"gui", "api", "ssh", "cli-profile", "console", "none"} +PRINCIPAL = {"human", "service", "-"} +CUSTODY = {"consolidated", "source-of-record", "off-manifest-known", + "not-consolidated-ruled"} + +# site-key discipline imported from scripts/lib-net.sh:160-162 / lib-hosts.sh:153-155: +# region-qualified only. A bare `dc0` is REJECTED -- it is the drifting form that has +# repeatedly meant two different things across the two DCs. +SITE_KEY_RE = re.compile(r"^(vr1-dc[01]|vr1-office1)$") +MINT_REF_RE = re.compile(r"^(script|runbook):([^:]+):(\d+)$") +DC_TOKEN_RE = re.compile(r"dc[01]") + +# The manifest mode column is DERIVED, not a matrix field: a public key half is 0644, +# everything else 0600. That rule reproduces all three checked-in manifests exactly, so +# it stays a derivation rather than a 13th column. +def derived_mode(filename): + return "644" if filename.endswith(".pub") else "600" + + +class Row(object): + def __init__(self, fields, lineno): + self.lineno = lineno + for name, value in zip(COLUMNS, fields): + setattr(self, name.replace("-", "_"), value) + + def key(self): + # `id` is part of the key on purpose: two DIFFERENT identities may legitimately + # share one file (the Vault init artifact holds both the unseal shares and the + # root token). What is never legitimate is the SAME identity declared twice at + # the same location. + return (self.id, self.site_key, self.host_role, self.filename) + + def has_artifact(self): + # `filename == "-"` means the credential has no host artifact (it lives in the + # cloud, on a unit, or -- the juju-user case -- nowhere at all). Existence-type + # checks must SKIP such rows, never fault them. + return self.filename != "-" + + +def load_matrix(path, fails): + rows = [] + try: + with open(path) as fh: + lines = fh.readlines() + except IOError as exc: + sys.stderr.write("ERROR: cannot read matrix %s: %s\n" % (path, exc)); sys.exit(2) + for n, line in enumerate(lines, 1): + text = line.split("#", 1)[0].strip() + if not text: + continue + fields = text.split() + if len(fields) != NCOL: + fails.append("matrix:%d has %d field(s), want %d -- a field with whitespace " + "in it would also land here: %s" % (n, len(fields), NCOL, text[:60])) + continue + rows.append(Row(fields, n)) + return rows + + +def load_manifest(path): + """Return {filename: (mode, source)} for a checked-in manifest, or None if absent.""" + if not os.path.exists(path): + return None + out = {} + with open(path) as fh: + for line in fh: + text = line.split("#", 1)[0].strip() + if not text: + continue + parts = text.split() + if len(parts) >= 3: + out[parts[0]] = (parts[1], parts[2]) + elif len(parts) == 2: + out[parts[0]] = (parts[1], "-") + return out + + +def load_notes(path): + keys = set() + if not os.path.exists(path): + return None + with open(path) as fh: + for line in fh: + if line.startswith("## "): + keys.add(line[3:].strip()) + return keys + + +# --------------------------------------------------------------------------- tier 1 +def s1_schema(rows, oks, fails): + bad = 0 + for r in rows: + where = "matrix:%d %s" % (r.lineno, r.id) + if r.cardinality not in CARDINALITY: + fails.append("%s cardinality '%s' invalid" % (where, r.cardinality)); bad += 1 + if r.site_key != "-" and not SITE_KEY_RE.match(r.site_key): + fails.append("%s site-key '%s' is not region-qualified (bare 'dc0' is REJECTED)" + % (where, r.site_key)); bad += 1 + if r.host_role not in HOST_ROLE: + fails.append("%s host-role '%s' invalid (must be a role token, never a VM name)" + % (where, r.host_role)); bad += 1 + if r.access_type not in ACCESS_TYPE: + fails.append("%s access-type '%s' invalid" % (where, r.access_type)); bad += 1 + if r.principal not in PRINCIPAL: + fails.append("%s principal '%s' invalid" % (where, r.principal)); bad += 1 + if r.custody not in CUSTODY: + fails.append("%s custody '%s' invalid" % (where, r.custody)); bad += 1 + seen = {} + for r in rows: + if r.has_artifact(): + if r.key() in seen: + fails.append("matrix:%d duplicate (id,site,host-role,filename) row -- also " + "at matrix:%d" % (r.lineno, seen[r.key()])); bad += 1 + seen[r.key()] = r.lineno + if not bad: + oks.append("S1 schema: %d rows, all enums valid, site-keys region-qualified, " + "no duplicate (id,site,host-role,filename)" % len(rows)) + + +def s2_manifest_coverage(rows, manifest_dir, oks, fails): + """BOTH BOUNDS. Imported from netbox/sandbox-fidelity-check.py:131-143, which records + the identical false-green in-repo: a subset-only check 'was an upper bound + masquerading as an assertion. It must be BOTH bounds.' Here the same failure is one + level up -- enumerate no expectations and nothing is ever undeclared, so the register + reads CLEAN while declaring nothing.""" + # BOTH BOUNDS AT THE SITE LEVEL TOO. Enumerating only the sites the MATRIX mentions + # is the same false-green one level up: an empty (or site-truncated) matrix would + # examine no manifest, find nothing undeclared, and report CLEAN while every declared + # credential sat unregistered. The site set is the UNION of both sides. + sites = {r.site_key for r in rows if r.site_key != "-"} + if os.path.isdir(manifest_dir): + for entry in os.listdir(manifest_dir): + if entry.endswith(".manifest"): + sites.add(entry[:-len(".manifest")]) + sites = sorted(sites) + total_missing = total_extra = total_drift = 0 + for site in sites: + manifest = load_manifest(os.path.join(manifest_dir, "%s.manifest" % site)) + if manifest is None: + fails.append("S2 %s: no manifest file -- every site in the matrix needs one" + % site) + continue + # A manifest declares exactly the CONSOLIDATED jumphost copies for its site. + expected = {r.filename: r for r in rows + if r.site_key == site and r.custody == "consolidated" + and r.host_role == "jumphost" and r.has_artifact()} + # bound 1: expected-but-absent + for fname, r in sorted(expected.items()): + if fname not in manifest: + total_missing += 1 + fails.append("S2 %s EXPECTED-BUT-ABSENT: '%s' (id %s%s) is expected by the " + "matrix but NOT declared in the manifest -- the credential is " + "either missing or undeclared" + % (site, fname, r.id, "" if r.sec_ref == "-" else ", " + r.sec_ref)) + # bound 2: declared but unexpected + for fname in sorted(manifest): + if fname not in expected: + total_extra += 1 + fails.append("S2 %s UNEXPECTED-EXTRA: manifest declares '%s' with no " + "matching matrix row -- add it to the register or remove it" + % (site, fname)) + # field drift on the rows present in both + for fname, r in sorted(expected.items()): + if fname not in manifest: + continue + want = derived_mode(fname) + if manifest[fname][0] != want: + total_drift += 1 + fails.append("S2 %s FIELD DRIFT: '%s' manifest mode %s, matrix derives %s" + % (site, fname, manifest[fname][0], want)) + # rows the matrix marks as deliberately NOT in the folder must not appear in it + for r in rows: + if (r.site_key == site and r.host_role == "jumphost" and r.has_artifact() + and r.custody != "consolidated" and r.filename in manifest): + total_drift += 1 + fails.append("S2 %s FIELD DRIFT: '%s' is custody=%s yet declared in the " + "manifest" % (site, r.filename, r.custody)) + if not (total_missing or total_extra or total_drift): + oks.append("S2 manifest coverage: both bounds clean across %d site(s)" % len(sites)) + + +def s3_render_drift(rows, manifest_dir, notes, oks, fails): + """D-137 ruling 2: manifests are DERIVED; checked-in must equal rendered.""" + sites = sorted({r.site_key for r in rows if r.site_key != "-"}) + drift = pending = 0 + for site in sites: + manifest = load_manifest(os.path.join(manifest_dir, "%s.manifest" % site)) + if manifest is None: + continue + for fname, mode, source, r in render_rows(rows, site): + if fname not in manifest: + continue # already reported by S2 + if manifest[fname][0] != mode: + drift += 1 + fails.append("S3 %s render drift: '%s' mode checked-in=%s rendered=%s" + % (site, fname, manifest[fname][0], mode)) + if source == "PENDING-TIER2": + # The source field for a VM-minted credential needs the declared PATH, + # which ruling 3 puts in creds-manifests/vm-secret-locations -- a tier-2 + # artifact. Skip the field EXPLICITLY rather than let it read as matching. + pending += 1 + continue + if manifest[fname][1] != source: + drift += 1 + fails.append("S3 %s render drift: '%s' source checked-in=%s rendered=%s" + % (site, fname, manifest[fname][1], source)) + if pending: + oks.append("S3 render: %d source field(s) SKIPPED pending tier 2 -- they need the " + "declared path from creds-manifests/vm-secret-locations (ruling 3), " + "which is not built yet" % pending) + if not drift: + # SCOPE, stated so this [ok] cannot be over-read: S3 compares + # PARSED ROW FIELDS (mode, source) for rows present on both sides. It does NOT + # compare manifest header prose or field-definition blocks, and render emits only + # consolidated jumphost rows -- so ruling 2's "checked-in == rendered" obligation + # is only PARTLY gated here. Full byte-equality becomes enforceable when the + # manifests are actually replaced by rendered output (operator-gated; see the + # 2026-07-26 changelog, item 3). + oks.append("S3 render drift: rendered row fields (mode, source) match checked-in; " + "header prose and non-jumphost rows are OUT OF SCOPE of this compare") + + +def render_rows(rows, site): + """Yield (filename, mode, source, row) for one site's rendered manifest.""" + src_of_record = {} + for r in rows: + if r.custody == "source-of-record": + src_of_record.setdefault(r.id, r) + out = [] + for r in rows: + if (r.site_key == site and r.custody == "consolidated" + and r.host_role == "jumphost" and r.has_artifact()): + source = "PENDING-TIER2" if r.id in src_of_record else "local" + out.append((r.filename, derived_mode(r.filename), source, r)) + # Sort on the rendered fields only -- Row is not orderable, and a duplicate row + # (T5) would otherwise reach a Row-vs-Row comparison and crash before S1's finding + # could be printed. + return sorted(out, key=lambda t: t[:3]) + + +def s4_mint_ref(rows, repo, oks, fails): + bad = 0 + terminal = 0 + for r in rows: + ref = r.mint_ref + if ref == "-": + continue + if ref == "operator-terminal": + terminal += 1 + continue + m = MINT_REF_RE.match(ref) + if not m: + fails.append("matrix:%d %s mint-ref '%s' is not script::, " + "runbook::, or operator-terminal" + % (r.lineno, r.id, ref)); bad += 1 + continue + _, path, line = m.groups() + full = os.path.join(repo, path) + if not os.path.exists(full): + fails.append("matrix:%d %s mint-ref target does not exist: %s" + % (r.lineno, r.id, path)); bad += 1 + continue + with open(full) as fh: + n = sum(1 for _ in fh) + if int(line) > n: + fails.append("matrix:%d %s mint-ref %s points past EOF (file has %d lines)" + % (r.lineno, r.id, ref, n)); bad += 1 + if not bad: + oks.append("S4 mint-ref: every script:/runbook: reference resolves to a real " + "location") + # Research FINDING 1 is a DEBT REPORT, not a failure: `ssh-keygen` returns zero hits + # repo-wide, so these rows genuinely have no mint command. Admitting them is the point. + if terminal: + oks.append("S4 provenance debt: %d row(s) are mint-ref=operator-terminal -- NOT " + "reproducible from the repo (research FINDING 1). Admitted by design; " + "converting them is remediation, not a checker fix." % terminal) + + +def s5_per_dc_symmetry(rows, dc_filter, oks, fails): + """Per-DC rows must be SYMMETRIC. sec-ref and notes-ref are excluded from the + comparison -- the two DCs legitimately carry different SEC rows for the same shape.""" + dcs = ["vr1-dc0", "vr1-dc1"] + if dc_filter: + oks.append("S5 per-DC symmetry: --dc %s narrows the run; symmetry needs BOTH DCs, " + "so the check is SKIPPED (explicitly, not passed)" % dc_filter) + return + shapes = {} + for dc in dcs: + shapes[dc] = set() + for r in rows: + if r.site_key != dc or r.cardinality != "per-DC": + continue + shapes[dc].add(( + DC_TOKEN_RE.sub("dcN", r.id), + DC_TOKEN_RE.sub("dcN", r.filename), + r.host_role, r.access_type, r.principal, r.custody, r.mint_stage, + DC_TOKEN_RE.sub("dcN", r.mint_ref), + )) + only0 = shapes["vr1-dc0"] - shapes["vr1-dc1"] + only1 = shapes["vr1-dc1"] - shapes["vr1-dc0"] + for label, extra, missing_from in (("vr1-dc0", only0, "vr1-dc1"), + ("vr1-dc1", only1, "vr1-dc0")): + for shape in sorted(extra): + fails.append("S5 ASYMMETRY: %s declares id=%s file=%s on %s (custody=%s) with " + "no counterpart in %s -- a per-DC credential must exist at BOTH " + "DCs in the same shape" + % (label, shape[0], shape[1], shape[2], shape[5], missing_from)) + if not (only0 or only1): + oks.append("S5 per-DC symmetry: %d shared per-DC shape(s), no asymmetry" + % len(shapes["vr1-dc0"])) + + +def s6_principal_invariant(rows, oks, fails): + """D-137 ruling 5: ONE IDENTITY SERVES ONE PRINCIPAL TYPE. Rows sharing an `id` are + forms/copies of one identity, so two differing principals across them IS the + conflation. Rows with principal '-' (non-credential artifacts) are skipped.""" + by_id = {} + for r in rows: + if r.principal == "-": + continue + by_id.setdefault(r.id, {}).setdefault(r.principal, []).append(r) + bad = 0 + for ident in sorted(by_id): + principals = by_id[ident] + if len(principals) > 1: + bad += 1 + detail = "; ".join( + "%s via %s" % (p, ", ".join(sorted({x.access_type for x in rs}))) + for p, rs in sorted(principals.items())) + fails.append("S6 IDENTITY CONFLATION: id '%s' serves %d principal types (%s) " + "-- ruling 5 requires one identity to serve one principal type" + % (ident, len(principals), detail)) + if not bad: + oks.append("S6 principal invariant: every identity serves exactly one principal " + "type (%d identities checked)" % len(by_id)) + + +def s7_notes_ref(rows, notes, notes_path, oks, fails): + if notes is None: + fails.append("S7 notes file missing: %s -- the 12-column amendment makes it the " + "home for every governance note ruling 2 forbids dropping" % notes_path) + return + bad = 0 + used = set() + for r in rows: + if r.notes_ref == "-": + continue + used.add(r.notes_ref) + if r.notes_ref not in notes: + fails.append("matrix:%d %s notes-ref '%s' has no '## %s' entry in the notes file" + % (r.lineno, r.id, r.notes_ref, r.notes_ref)); bad += 1 + orphans = sorted(k for k in notes if k.startswith("n-") and k not in used) + for k in orphans: + fails.append("S7 orphan note '%s' is referenced by no matrix row" % k) + if not bad and not orphans: + oks.append("S7 notes: %d note key(s) referenced, all resolve, none orphaned" + % len(used)) + + +def main(): + ap = argparse.ArgumentParser(description="D-137 credential-matrix checker") + here = os.path.dirname(os.path.abspath(__file__)) + repo = os.path.dirname(here) + ap.add_argument("--matrix", default=os.path.join(repo, "creds-matrix.tsv")) + ap.add_argument("--manifest-dir", default=os.path.join(repo, "creds-manifests")) + ap.add_argument("--notes", default=os.path.join(repo, "creds-matrix-notes.md")) + ap.add_argument("--repo", default=repo) + ap.add_argument("--dc", default=None, help="narrow per-DC checks to one region-qualified DC") + ap.add_argument("--all", action="store_true", help="run every implemented tier") + ap.add_argument("--render", action="store_true", + help="print manifests rendered from the matrix (does not write)") + args = ap.parse_args() + + if args.dc and not SITE_KEY_RE.match(args.dc): + sys.stderr.write("ERROR: --dc must be region-qualified (vr1-dc0 | vr1-dc1)\n") + return 2 + + oks, fails = [], [] + rows = load_matrix(args.matrix, fails) + notes = load_notes(args.notes) + + if args.render: + for site in sorted({r.site_key for r in rows if r.site_key != "-"}): + print("# %s.manifest -- GENERATED from creds-matrix.tsv (D-137 ruling 2)." % site) + print("# Do not hand-edit: regenerate with scripts/creds-matrix.py --render.") + for fname, mode, source, r in render_rows(rows, site): + if r.notes_ref != "-": + print("# [%s] see creds-matrix-notes.md ## %s" % (r.id, r.notes_ref)) + print("%-32s %s %s" % (fname, mode, source)) + print() + return 0 + + print("=== creds-matrix: tier 1 (STATIC) ===") + s1_schema(rows, oks, fails) + s2_manifest_coverage(rows, args.manifest_dir, oks, fails) + s3_render_drift(rows, args.manifest_dir, notes, oks, fails) + s4_mint_ref(rows, args.repo, oks, fails) + s5_per_dc_symmetry(rows, args.dc, oks, fails) + s6_principal_invariant(rows, oks, fails) + s7_notes_ref(rows, notes, args.notes, oks, fails) + + if args.all: + # SELF-SKIP WITH AN EXPLICIT [ok]: an unbuilt tier must be visible, never silent. + oks.append("tier 2 (EXISTENCE) NOT BUILT -- needs creds-manifests/vm-secret-locations " + "and creds-audit --remote; SEC-022/-023 and SEC-021's on-disk half cannot " + "be reproduced until it exists") + oks.append("tier 3 (VALIDITY) NOT BUILT -- needs cross-host sha256 provenance " + "comparison (never content transfer) and per-row behavioral probes") + + for o in oks: + print(" [ok] %s" % o) + for f in fails: + print(" [FAIL] %s" % f) + print("\n%s: creds-matrix tier 1 -- %d row(s), %d check group(s) clean, %d finding(s)" + % ("PASS" if not fails else "FAIL", len(rows), len(oks), len(fails))) + return 1 if fails else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/creds-matrix/run-tests.sh b/tests/creds-matrix/run-tests.sh new file mode 100644 index 0000000..d0d32a0 --- /dev/null +++ b/tests/creds-matrix/run-tests.sh @@ -0,0 +1,241 @@ +#!/usr/bin/env bash +# tests/creds-matrix/run-tests.sh -- harness for scripts/creds-matrix.py (D-137 tier 1). +# +# One case per invariant, each named for the REAL defect it encodes. Fixtures are +# mutations of a minimal valid artifact set, and every case asserts BOTH the exit code and +# an output regex -- a check that fires with the wrong message is a check nobody can act on. +# +# Gauntlet contract (scripts/run-tests-all.sh:22 registers by glob and shows only tail -1): +# the LAST line self-describes, and failure detail lines lead with a bare FAIL so the +# runner's `^\s*(FAIL|\[XX\]|MISS|LEAK|COUNT)` excerpt matches them. +set -uo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO="$(cd "$HERE/../.." && pwd)" +CHECK="$REPO/scripts/creds-matrix.py" +TMPROOT="${TMPDIR:-/tmp}/creds-matrix-tests.$$" +mkdir -p "$TMPROOT" +trap 'rm -rf "$TMPROOT"' EXIT + +P=0; F=0 +ok() { P=$((P+1)); echo " [ok] $1"; } +no() { F=$((F+1)); echo "FAIL $1"; } + +# ---- minimal VALID fixture set ------------------------------------------------------- +mkenv() { + local d; d="$(mktemp -d "$TMPROOT/env.XXXXXX")" + mkdir -p "$d/creds-manifests" + cat > "$d/matrix.tsv" <<'EOF' +alpha-key per-site vr1-office1 jumphost alpha_ed25519 ssh service consolidated stage2 operator-terminal - n-alpha +alpha-key per-site vr1-office1 jumphost alpha_ed25519.pub none service consolidated stage2 operator-terminal - n-alpha +EOF + cat > "$d/creds-manifests/vr1-office1.manifest" <<'EOF' +alpha_ed25519 600 local +alpha_ed25519.pub 644 local +EOF + cat > "$d/notes.md" <<'EOF' +## n-alpha +The alpha service keypair. +EOF + echo "$d" +} +run() { # run [extra args...] -> sets RC, writes $1/out + local d="$1"; shift + python3 "$CHECK" --matrix "$d/matrix.tsv" --manifest-dir "$d/creds-manifests" \ + --notes "$d/notes.md" --repo "$REPO" "$@" > "$d/out" 2>&1 + RC=$? +} +row() { printf '%s\n' "$1" >> "$2/matrix.tsv"; } +note() { printf '\n## %s\nfixture note.\n' "$1" >> "$2/notes.md"; } + +echo "=== creds-matrix harness ===" + +# T1 -- the baseline must actually PASS, or every other case proves nothing. +D=$(mkenv); run "$D" +[ "$RC" = 0 ] && grep -q '^PASS: creds-matrix' "$D/out" \ + && ok "T1 a consistent matrix/manifest/notes set -> PASS, exit 0" \ + || { no "T1 the valid baseline must pass (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T2 -- a field containing whitespace silently shifts every later column. +D=$(mkenv); row 'beta pass singleton vr1-office1 jumphost beta.txt api service consolidated stage2 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'field(s), want 12' "$D/out" \ + && ok "T2 a row with a whitespace-bearing field -> field-count FAIL" \ + || { no "T2 whitespace in a field must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T3 -- bare `dc0` is the drifting form that has meant two different things per DC. +D=$(mkenv); row 'beta per-DC dc0 jumphost beta.txt api service off-manifest-known stage3 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'not region-qualified' "$D/out" \ + && ok "T3 a bare 'dc0' site-key -> REJECTED (region-qualified discipline)" \ + || { no "T3 bare dc0 must be rejected (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T4 -- an invented custody token would silently escape every custody-conditioned rule. +D=$(mkenv); row 'beta singleton vr1-office1 jumphost beta.txt api service probably-fine stage2 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q "custody 'probably-fine' invalid" "$D/out" \ + && ok "T4 an unknown custody token -> enum FAIL" \ + || { no "T4 custody enum must be closed (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T5 -- the SAME identity declared twice at one location is a real duplication error. +D=$(mkenv); row 'alpha-key per-site vr1-office1 jumphost alpha_ed25519 ssh service consolidated stage2 operator-terminal - n-alpha' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'duplicate (id,site,host-role,filename)' "$D/out" \ + && ok "T5 one identity declared twice at one location -> duplicate FAIL" \ + || { no "T5 same-identity duplication must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T6 -- REGRESSION: two DIFFERENT identities may share one file (the Vault init artifact +# holds both the unseal shares and the root token). Keying dedup on location alone +# false-failed this. +D=$(mkenv) +row 'vault-shares singleton - jumphost vault-init none service off-manifest-known vr0-phase02 operator-terminal - -' "$D" +row 'vault-root singleton - jumphost vault-init none service off-manifest-known vr0-phase02 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 0 ] \ + && ok "T6 two identities sharing one file -> allowed (Vault init regression)" \ + || { no "T6 distinct identities may share a file (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T7 -- SEC-021's class: the register expects a credential the manifest never declared. +D=$(mkenv); row 'gamma per-site vr1-office1 jumphost gamma.txt api service consolidated stage2 operator-terminal SEC-021 -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q "EXPECTED-BUT-ABSENT: 'gamma.txt'" "$D/out" \ + && ok "T7 expected by the matrix, absent from the manifest -> EXPECTED-BUT-ABSENT (SEC-021 class)" \ + || { no "T7 expected-but-absent must fire (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T8 -- the undeclared-secret class SEC-009/-020 kept missing, now from the other side. +D=$(mkenv); printf 'delta.txt 600 local\n' >> "$D/creds-manifests/vr1-office1.manifest" +run "$D" +[ "$RC" = 1 ] && grep -q "UNEXPECTED-EXTRA: manifest declares 'delta.txt'" "$D/out" \ + && ok "T8 declared in the manifest, unknown to the matrix -> UNEXPECTED-EXTRA" \ + || { no "T8 unexpected-extra must fire (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T9 -- THE FALSE GREEN. An empty matrix examines no site, so a site-scoped loop finds +# nothing undeclared and reports CLEAN while every declared credential sits unregistered. +# Same defect netbox/sandbox-fidelity-check.py:131-143 records: an upper bound +# masquerading as an assertion. The site set must be the UNION of both sides. +D=$(mkenv); printf '# every row removed\n' > "$D/matrix.tsv" +run "$D" +[ "$RC" = 1 ] && grep -q 'UNEXPECTED-EXTRA' "$D/out" \ + && ok "T9 an EMPTY matrix against populated manifests -> FAILS (the false-green regression)" \ + || { no "T9 an empty matrix must not read clean (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T10 -- a mint-ref that points past EOF is stale provenance, the drift class tier 3 exists for. +D=$(mkenv); row 'beta singleton vr1-office1 headend beta.txt api service off-manifest-known stage2 script:scripts/creds-audit.sh:99999 - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'points past EOF' "$D/out" \ + && ok "T10 a mint-ref line past EOF -> stale-provenance FAIL" \ + || { no "T10 out-of-range mint-ref must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T11 -- a mint-ref naming a file that no longer exists (the rename/retire class). +D=$(mkenv); row 'beta singleton vr1-office1 headend beta.txt api service off-manifest-known stage2 script:scripts/no-such-script.sh:1 - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'mint-ref target does not exist' "$D/out" \ + && ok "T11 a mint-ref to a deleted file -> FAIL" \ + || { no "T11 missing mint-ref target must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T12 -- research FINDING 1: ssh-keygen has ZERO hits repo-wide, so 12 declared secrets +# have no mint command. Those rows are the DEBT the register exists to surface -- they +# must be ADMITTED and reported, never treated as parse errors. +D=$(mkenv); run "$D" +[ "$RC" = 0 ] && grep -q 'provenance debt: 2 row(s) are mint-ref=operator-terminal' "$D/out" \ + && ok "T12 operator-terminal rows -> admitted and reported as debt, not parse errors" \ + || { no "T12 operator-terminal must be admitted + counted (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T13 -- a per-DC credential existing at one DC only (the SEC-021 divergence shape). +D=$(mkenv) +row 'dc0-power per-DC vr1-dc0 jumphost vr1-dc0-power_ed25519 ssh service consolidated stage3 operator-terminal - -' "$D" +mkdir -p "$D/creds-manifests"; printf 'vr1-dc0-power_ed25519 600 local\n' > "$D/creds-manifests/vr1-dc0.manifest" +run "$D" +[ "$RC" = 1 ] && grep -q 'S5 ASYMMETRY' "$D/out" \ + && ok "T13 a per-DC credential at one DC only -> ASYMMETRY FAIL (SEC-021 shape)" \ + || { no "T13 per-DC asymmetry must fire (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T14 -- narrowing to one DC cannot silently 'pass' a check that needs both. +D=$(mkenv); run "$D" --dc vr1-dc1 +[ "$RC" = 0 ] && grep -q 'symmetry needs BOTH DCs, so the check is SKIPPED' "$D/out" \ + && ok "T14 --dc narrows -> symmetry SKIPPED explicitly, never silently passed" \ + || { no "T14 --dc must skip symmetry out loud (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T15 -- D-137 ruling 5: ONE IDENTITY SERVES ONE PRINCIPAL TYPE. This is the live SEC-020 +# conflation (`admin` is both the human GUI login and the automation identity). +D=$(mkenv) +row 'shared-ident singleton vr1-office1 headend shared.pass gui human off-manifest-known stage2 operator-terminal - -' "$D" +row 'shared-ident singleton vr1-office1 headend shared.apikey api service off-manifest-known stage2 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q 'IDENTITY CONFLATION' "$D/out" \ + && ok "T15 one identity serving human AND service -> ruling-5 CONFLATION FAIL" \ + || { no "T15 ruling-5 invariant must fire (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T16 -- a non-credential index file (principal '-') must not fault the ruling-5 check. +D=$(mkenv) +row 'folder-readme per-site vr1-office1 jumphost README none - consolidated - - - -' "$D" +printf 'README 600 local\n' >> "$D/creds-manifests/vr1-office1.manifest" +run "$D" +[ "$RC" = 0 ] \ + && ok "T16 a principal='-' index file -> skipped by ruling 5, not faulted" \ + || { no "T16 principal '-' must be skipped (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T17 -- a notes-ref with no entry means ruling-2 governance prose was silently dropped. +D=$(mkenv); row 'beta singleton vr1-office1 headend beta.txt api service off-manifest-known stage2 operator-terminal - n-nonexistent' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q "notes-ref 'n-nonexistent' has no" "$D/out" \ + && ok "T17 a dangling notes-ref -> FAIL (ruling-2 prose must not be droppable)" \ + || { no "T17 dangling notes-ref must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T18 -- an orphan note is prose that survived the row it documented (the other direction). +D=$(mkenv); note 'n-orphan' "$D" +run "$D" +[ "$RC" = 1 ] && grep -q "orphan note 'n-orphan'" "$D/out" \ + && ok "T18 a note referenced by no row -> orphan FAIL" \ + || { no "T18 orphan notes must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T19 -- the mode column is DERIVED (.pub -> 644, else 600), so a hand-edited manifest +# mode is drift. This is what makes manifests generated artifacts per ruling 2. +D=$(mkenv); sed -i 's/^alpha_ed25519.pub 644/alpha_ed25519.pub 600/' "$D/creds-manifests/vr1-office1.manifest" +run "$D" +[ "$RC" = 1 ] && grep -qE 'FIELD DRIFT|render drift' "$D/out" \ + && ok "T19 a hand-edited manifest mode -> render/field DRIFT FAIL" \ + || { no "T19 mode drift must be caught (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T20 -- rows with no host artifact (filename '-') must skip existence-shaped checks +# rather than fault: the juju- superuser password is stored NOWHERE by ruling. +D=$(mkenv) +row 'nowhere-user singleton vr1-office1 - - api service off-manifest-known stage5 operator-terminal - -' "$D" +run "$D" +[ "$RC" = 0 ] \ + && ok "T20 a filename='-' row (credential with no host artifact) -> skipped, not faulted" \ + || { no "T20 no-artifact rows must be skipped (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T21 -- an unbuilt tier must announce itself. A silent pass is the exact false-green +# this design exists to prevent. +D=$(mkenv); run "$D" --all +[ "$RC" = 0 ] && grep -q 'tier 2 (EXISTENCE) NOT BUILT' "$D/out" \ + && grep -q 'tier 3 (VALIDITY) NOT BUILT' "$D/out" \ + && ok "T21 --all names tiers 2 and 3 as NOT BUILT -> no silent pass" \ + || { no "T21 unbuilt tiers must self-announce (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T22 -- METADATA ONLY. The checker must never gain a content-reading verb; it inspects +# existence, mode, and declared provenance. Mirrors tests/creds-audit/run-tests.sh:71-73. +grep -qE '\b(cat|head|tail|less|more|od|xxd|hexdump)\b' "$CHECK" \ + && no "T22 the checker contains a content-reading verb -- it must inspect metadata only" \ + || ok "T22 metadata-only: no cat/head/tail/od/xxd in the checker" + +# T23 -- --render must emit a GENERATED banner, so nobody hand-edits a derived manifest. +D=$(mkenv); run "$D" --render +[ "$RC" = 0 ] && grep -q 'GENERATED from creds-matrix.tsv' "$D/out" \ + && grep -qE 'alpha_ed25519\.pub +644 +local' "$D/out" \ + && ok "T23 --render emits a GENERATED banner and the derived modes" \ + || { no "T23 --render output shape (rc=$RC)"; sed 's/^/ /' "$D/out"; } + +# T24 -- the REAL tree must be RED, and red for the RIGHT reasons. A checker that goes +# green on today's tree is wrong (plan: EXPECTED RED BY DESIGN). Guards against someone +# "fixing" the register by deleting the rows that fail. +python3 "$CHECK" > "$TMPROOT/real.out" 2>&1; RRC=$? +[ "$RRC" = 1 ] && grep -q 'IDENTITY CONFLATION' "$TMPROOT/real.out" \ + && grep -q 'EXPECTED-BUT-ABSENT' "$TMPROOT/real.out" \ + && ok "T24 the real tree is RED with the ruling-5 conflation + SEC-021 absences intact" \ + || { no "T24 the real tree must stay RED by design (rc=$RRC)"; sed 's/^/ /' "$TMPROOT/real.out"; } + +echo +if [ "$F" = 0 ]; then echo "creds-matrix: $P/$P PASS"; exit 0; fi +echo "creds-matrix: FAILURES: $F (passed $P)"; exit 1