tfstate backup path built; E2/E3 closed live; a gate disagreement found and fixed ...

BACKUP PROVEN BY RESTORE, not by existence. Both PKI archives were extracted and compared
byte-for-byte against the live tree: 12/12 sha256 MATCH across both DCs, covering both encrypted
CA keys, both passphrases, both CA serials and both controller bundles. Archives are 600 inside
700 folders and differ between DCs, so per-DC independence survives into the backup.

E2/E3 CLOSED LIVE. Operator ran the chmod and the intermediate removal; octavia-pki.sh verify
now reads PASS 26/0 on BOTH DCs, workspaces 12 -> 10 files, and P5 on the headend fell 18 -> 8.

A GATE DISAGREEMENT, recorded as the pattern rather than the incident. verify read 26/0 while
creds-matrix E2 still reported the CA serial mode 664 on both DCs. The chmod list predated the
serial being declared, and verify's mode assertions covered six private files and three certs --
the serial was in neither. Two gates that can both see a file must not disagree about it. Fixed
in three places: verify now mode-checks the serial (integrity, not secrecy -- rewriting it forces
the next issuance to reuse a serial), the generator's chmod includes it, and T18 pins it. The
harness then caught its own fixture creating the serial at the inherited umask; baseline went red
until the fixture matched the corrected generator. 18/18.

TFSTATE BACKUP as dc-dc-phase2 step 13, 2 rows, notes key n-tfstate-backup. Register 99 rows;
schema, render-drift, mint-ref and notes checks clean.

Three deliberate differences from the PKI backup: tfstate is DYNAMIC so the serial is recorded
both sides and the step must be re-run after every apply (restoring a stale state is actively
dangerous -- tofu then believes everything created since does not exist); the pull block proves
RESTORE by decompressing to check the archive parses as JSON and reports a serial, because a
truncated gzip passes a size check; and mint-stage is stage3, which is REACHED, so P5 correctly
reports these EXPECTED-BUT-ABSENT until the step runs rather than deferring them.

Classification recorded honestly: the outer state is credential-bearing (DOCFIX-175, MAAS API key
in plaintext); the inner states show no credential-shaped key names, consistent with the inner
root using only libvirt over qemu+ssh. Strong evidence, NOT proof -- a name-based absence cannot
exclude material inside a value -- so they are registered and stored as if sensitive. Uncertainty
resolves toward more auditing, and an unaudited backup directory beside audited ones is how the
SEC-022 shadow stores happened.

Two of my own defects caught by the gates: repo-lint L9 rejected a hardcoded clone name in the
new step (the repo has been renamed once, D-110) -- now $REPO; and two mint-refs had drifted when
a later edit added lines above their anchors, which S4 cannot catch because it only checks
line <= EOF. Both re-anchored.

Also corrected: phase-01:605 said the overlay must be in the backup set. It is derivable from the
archived workspace by 1.0-GEN.d, so the line implied a gap that does not exist.

Known, not fixed: both live inner states are 664, group-writable, and they are the authority tofu
trusts. Tightening needs its own gated change plus proof the provider preserves the mode.

creds-matrix 65/65, octavia-pki 18/18, repo-lint 0 fail / 621 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Build the PKI backup path: three surfaces asserted a backup set that did not exist ...

Operator direction: back up to the per-DC jumphost creds folder BEFORE the cert cleanup, and
record that the pinned secrets-storage solution must carry a certificate/credential backup
procedure with this step folded into it.

THE GAP, MEASURED. phase-01:594 said the workspace must be "backed up securely", :605 said it
"MUST be in the per-DC backup set", and CURRENT-STATE said the inner tfstate should be added to
"the site backup set" -- while cloud-snapshot.sh, the only candidate, is a juju-layer capture
that mentions octavia, tfstate and terraform ZERO times. Both DCs' 10-year amphora trust roots
therefore sat in one place on one VM with no copy. Losing the issuing CA key means Octavia can
never sign another amphora certificate; losing the controller CA key means the controller
certificate can never be reissued, which F9 says it must be at Stage 7.

SHAPE: one gzipped archive per DC rather than a file-by-file tree copy -- 2 register rows instead
of ~20, atomic (a half-copied tree is the dangerous state), and sha256-verifiable. The live tree
on the headend stays what octavia-pki.sh verify asserts; the archive is recovery only.

COST STATED RATHER THAN GLOSSED: a second at-rest copy of both encrypted CA keys beside their
plaintext passphrases, so the archive's contents defeat encryption-at-rest. That is the trade
D-109 option (b) was refused for; the distinction is deliberate, because that ruling governed
where the authoritative artifact lives and which host the deploy reads. A recovery copy is not a
second source of truth.

Delivered: phase-01 step 1.0-GEN.e (build on the headend, pull to the per-DC creds folder,
compare sha256 against the source BEFORE removing the staging copy -- a truncated scp would
otherwise leave a verified-looking backup of nothing); 2 register rows, custody=consolidated
since the creds folder is the SEC-009 location; notes key n-pki-backup.

THE REGISTER CAUGHT AN INCOMPLETE CHANGE OF MINE: adding rows raised S2 EXPECTED-BUT-ABSENT
twice, because the per-site manifests are DERIVED from the matrix and I had not re-rendered them.
Rendered rows appended to both; S3 render drift clean, findings back to the pre-existing 7.

Standing forward requirement recorded in the register rather than as a runbook comment: the
pinned secrets-storage solution must include a documented process and procedure for certificate
and credential backup, and this step is one of the steps that must be folded into it. The
jumphost creds folder is the interim home only.

Still not backed up and out of scope here: the two inner terraform.tfstate files on the headend
-- gitignored, untracked, state-of-record for 20 VMs, and named as owed to the same backup set
that has just been shown not to exist.

Register 97 rows / 24 octavia rows. Gauntlet ALL GREEN (89), repo-lint 0 fail / 621 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

E3 resolved: declare the two artifacts that persist, delete the two intermediates at source ...

Operator-approved disposition after review. The four undeclared per-DC generator outputs were
separated by whether anything will read them again.

DECLARED. The controller certificate, because octavia-pki.sh verify reads it for the subject,
chain and SAN assertions -- a gate that depends on an undeclared artifact is incoherent. And
the CA serial file, because it is issuance STATE rather than residue: delete it and the next
-CAcreateserial starts a fresh sequence, so the same CA can issue a duplicate serial.
Reissuance is scheduled, not hypothetical -- the controller cert is 2-year and F9's D-106 work
will want new SANs.

DELETED AT SOURCE. The signing request is spent the moment the cert is issued. The openssl
config is fully derived from $DC plus the VIP overlay, so the repo already determines it, and
verify now asserts the SAN set on the CERT, which is where the F8 failure actually shows.

WHY NOT DECLARE ALL FOUR, which was lower-risk and less work: a register that accumulates rows
for artifacts that should not exist trains the reader to add a row rather than ask whether the
file belongs -- the register-as-rubber-stamp failure, the same family as this project's
false-green problems. Concretely, both intermediates are 664, so declaring them would have
produced four MORE world-readable findings for files nothing reads.

Also fixed at source, beyond the three requested steps and flagged as such: the generator now
chmod 600s all three certificates, so the next DC is correct by construction instead of by a
remembered follow-up.

Register 95 rows / 22 octavia rows; schema, mint-ref and notes checks clean; no new findings
(still 7 on the jumphost, the same pre-existing set). verify A2 extended 9 -> 10 so the register
and the gate agree on what should exist.

Harness 17/17 (was 15). T16 proves a missing serial FAILS. T17 proves a workspace with the
intermediates DELETED still PASSES -- without it, adding the serial to A2 would have silently
failed every clean workspace.

Gauntlet ALL GREEN (89), repo-lint 0 fail / 621 files.

Still owed, operator-executed: the one-off chmod on the six existing certs and removal of the
four existing intermediates. The guard blocks both for the session -- including the chmod, which
is worth recording as fork-1 evidence since it blocks an operation that strictly improves
posture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

Execute the ruled generation host: Step 1.0-GEN and the register move to the headend ...

Dependent work for the D-109 ruling note (b), "Generate on voffice1 (Recommended)". Repo-side
only -- no PKI has been minted.

All seven RUN markers inside Step 1.0-GEN now say voffice1, with $REPO stated to mean the
HEADEND clone. The 18 Octavia register rows move to host-role=headend, the Octavia entries in
vm-secret-locations become headend, and host-identity binds headend -> voffice1.

The Octavia locations are declared `local` rather than with the voffice1 ssh-target on purpose:
that routes them through the F6 host binding, so the checker refuses to measure them from the
wrong machine instead of quietly probing whatever filesystem it is standing on.

Measured both ways:
  on vcloud   -> all 8 locations report NOT PROBED, naming the host they actually live on
  on headend  -> probed for real, reporting not-yet-minted
Verdict unchanged at 7 findings on vcloud; tests/creds-matrix 65/65.

Consequence recorded rather than discovered later: P5's octavia rows are now judged ONLY on
voffice1, so the credential half of the Stage-5 entry gate must be read on the headend -- the
same host P3/P4 already require. A narrowing of where preflight is authoritative, and a direct
consequence of the ruled generation host.

repo-lint 0 fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

F2 BUILT: the Octavia PKI rows are per-DC, so P5 can now see a missing second CA set ...

The nine Octavia PKI ids were cardinality=singleton with site-key=- and
mint-stage=vr0-phase01. creds-matrix.py's s5_per_dc_symmetry enforces both-DC existence ONLY
for rows marked per-DC, so the register behind the BLOCKING P5 gate was structurally blind to
a missing second PKI set -- while R7 rules that there must be exactly two.

This turned out to be RULED work, not a judgment call: R13 Part 1 (D-137 sub-ruling 6, RULED
2026-07-27) already requires these rows staged to what Stage 5 actually mints, and names R7 as
making it more urgent -- "per-DC independent Octavia PKI means TWO CA mints where the register
expects none".

Nine ids -> 18 rows (9 x 2 DCs), per-DC, region-qualified site-keys, mint-stage=stage5, and
the overlay row's filename follows the F1 rename.

Mint-refs RE-DERIVED from the edited phase-01: s4_mint_ref only checks that a line number is
within EOF, so the F1 commit had silently left all nine pointing at wrong lines with no gate
able to notice.

PROVEN CAPABLE OF FAILING before being trusted -- the inverse of "a checker that cannot fail
is not a gate", which this repo established bites both ways:
  both DCs declared   -> 91 rows, 7 findings (the same 7 as the 82-row baseline: none added)
  one dc1 row removed -> 8 findings, S5 ASYMMETRY naming the missing vr1-dc1 counterpart
tests/creds-matrix 60/60 PASS. repo-lint 0 fail / 615 files.

stage5 deliberately STAYS pending in stages-reached: the rows are now correctly ATTRIBUTED,
not yet expected, and the flip belongs with the actual mint. Trap recorded for that flip --
stage5 is one coarse token also covering eight existing rows.

Remainder of R13 Part 1 NOT done: the vault-init and admin-openrc rows carry the same
mis-staging, and vault-init the same per-DC defect under D-109's original text. Deliberately
not bundled: its blast radius reaches a directory CLAUDE.md designates operator-only one-shot.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

creds consolidation batch: SEC-021(b) dc0 power key, SEC-025 NetBox GUI, V2 ruled deferral ...

Operator asked whether vcloud holds a consolidated set of login creds for every account that
exists, and what is missing / needs creating / was never moved in. Audited, then cleared the
consolidation batch on operator direction ("clear the whole consolidation batch first").

The audit's own headline: creds-audit read CLEAN on all three sites while the matrix returned
13 findings -- and the two most consequential gaps were in NEITHER surface. They were found by
enumerating the accounts that ACTUALLY EXIST. Absence of a ROW is invisible to the register,
which generalises D-137's founding argument one level up.

VERIFIED POSITIVE (both previously only asserted):
- MAAS account set COMPLETE: all 6 live accounts accounted for; admin + operator passwords on
  vcloud, juju-vr1-dc0/dc1 random+unstored BY RULING with API keys present, MAAS +
  maas-init-node MAAS-internal.
- tier-3 V1 now MEASURES maas-admin-password byte-identical to the headend source-of-record,
  so SEC-020's stale-trap risk is clear as of this date.

DONE:
- dc0 SEC-012 power key consolidated to vcloud + .pub DERIVED (SEC-021(b) as written).
  MEASURED FIRST, because the naming looked like a defect and was not: the headend
  maas-virsh_ed25519 and the snap's id_ed25519 are the SAME key; it IS dedicated (distinct
  from the dc0 service key, which is what SEC-012 requires); and dc0 using the snap's default
  identity is SEC-016's RULED design. So no re-mint, no live power path touched.
- dc1 svc .pub backfilled to the headend store (public-key material, sha256-verified).
- NetBox web-GUI admin password consolidated -- a HUMAN login that had never left the VM it
  was minted on, the third instance of the SEC-009/SEC-020 miss class. SEC-025 OPENED for the
  at-rest exposure the copy CREATES, not the gap it closed. Open rows 20 -> 21.
- V2 taught the ruled-deferral state: SEC-006's standing "revoke at completion of this
  deployment" ruling is now ACKNOWLEDGED (still naming the credential live and exposed)
  instead of failing every run. Reissuing the token would have CONTRAVENED that ruling, so the
  register was what needed to change. Match demands the cased state word AND explicit operator
  ruling attribution, and cannot cross a ledger table cell -- T58/T59 lock the evasion out.
- Added --ledger (default unchanged): V2 read the register via --repo, which the harness must
  point at the real repo for S4, so V2 had shipped with ZERO harness cases. T56-T59 added.

Findings 13 -> 7. Matrix 82 rows, harness 60/60 (was 56), creds-audit CLEAN x3, gauntlet ALL
GREEN (81), repo-lint 0-fail. The register is NOT green and is not expected to be.

RESIDUAL, all recorded: dc0-edge-api x2 (the opnsense-api.txt re-mint is a live edge mutation,
deliberately EXCLUDED from this batch), S5 x3 (RULED by SEC-016 -- needs a ruled-exception
mechanism, operator decision), S6 conflation x1 (the SEC-020 defect), E4 x2 (Stage-5/6 rows).

LOGGED NOT ACTIONED (hard rule 1): no registered root/console credential at EITHER DC edge
(measured absence of row/manifest/SEC row; what those passwords ARE is UNKNOWN and
deliberately unprobed per hard rule 2 -- vector is the LAN-reachable GUI and serial console,
not SSH, which is key-only and proven); plus the two structural blind spots that let it hide
-- S5 compares only cardinality=per-DC while all six per-site rows are office1-only, and
vm-secret-locations declares no rack/edge/cloud/unit/client location though the checker
accepts them.

Revert: per-item reverts in docs/changelog-20260727-creds-consolidation.md. The headend, snap
and NetBox VM source copies were never modified, so every file move is reversible by deletion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

D-137 tier 3 (VALIDITY) BUILT: cross-copy sha256 provenance ...

Closes the gap the plan names: creds-audit PARSES declared provenance and
never verifies it, so SEC-020's stale-trap warning has been unchecked prose
and its sha256 equality was a ONE-TIME manual act on 2026-07-25.

sha256sum runs as an EXTERNAL command on both sides so only a 64-hex digest
enters the checker -- hashing bytes in Python would meet the letter of "never
transfer content" and not the spirit.

RESULT: 4 groups compared, all byte-identical, INCLUDING the SEC-020 pair
(maas-region-admin, region source-of-record vs jumphost copy). Now
machine-verified instead of remembered. The [ok] line names the groups.

TWO FALSE POSITIVES IN MY OWN DRAFT, caught by verifying before reporting.
Keying digests by (role, site) let an identity's DISTINCT artifacts at one
scope overwrite each other, so it compared unrelated files and reported drift
on maas-region-admin and dc1-svc-key. Manual sha256sum showed both IDENTICAL.
Reporting that would have raised a false credential-rotation alarm.

Root cause was a schema gap: nothing expressed which rows are byte-copies.
custody gains `verbatim-copy` [OPS]. V1 now compares only rows sharing
(id, filename) across scopes, or a source-of-record row against declared
verbatim-copy rows -- SEC-020's exact shape. Everything else is skipped with
a reason: a pair we cannot justify comparing is not a pair we may report
drift on. verbatim-copy is manifest-bearing; it asserts byte-identity, it
does not move the file.

Harness 44 -> 52: T47/T48 regression-lock both false positives; T49 keeps
SEC-020's case from falling into T48's skip; T51 asserts the digest path uses
sha256sum and that no content-transfer verb exists anywhere in the checker.

NOT built: tier 3's behavioural probes. Liveness, expiry, revocation state
and grant/scope drift remain unverified.

Gauntlet ALL GREEN (80), repo-lint 0-fail, finding-class baseline unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

D-137 phase 3: re-measure the register; SEC-024 opened ...

~21 inferred filenames corrected against their own mint-refs. The session had
owned six; those were only the rows the first sweep physically touched.
Octavia's 8 real basenames plus its THREE SUBDIRECTORIES (the bare
~/octavia-pki/* pattern matched ZERO of the 8 artifacts, two of them CA
private keys); vault's init.txt; the tenant rows' <client>- instance prefix,
with a placeholder matcher so they can match a real basename at all;
kube-config -> config; .maas.cli -> .maascli.db (measured: ~/.maas.cli does
not exist).

sec-ref mis-attribution corrected: both juju-maas-user rows cited SEC-020;
the on-subject rows are SEC-018/-019. This is the SEC-015 class recurring --
one instance was fixed by hand earlier without checking whether it was
systemic. It was.

mint-ref corrected: admin.pass cited a CONSUMER (:452), not the mint (:450);
the RGW key cites a line that takes it as INPUT, so it is operator-terminal.

Rows added: the libvirt SSH power password (standing rotation obligation from
a 2026-06-26 plaintext exposure, in neither the matrix nor the exclusions),
vault-ca-root, and the tfstate siblings. ~/as-executed/ deliberately NOT
declared, reason recorded: it holds session logs, so declaring it made every
log a false UNDECLARED.

Four further defects in my own phase-2 changes, found by RUNNING the sweep:
shlex.quote broke ~ expansion; a non-existent parent read as unreadable
rather than absent; site-key `-` needed shared-location semantics; the
tfstate siblings needed rows.

SEC-024 OPENED -- new exposure found by the fixed checker on its first clean
run: opentofu/terraform.tfstate.backup is mode 0664 and carries the MAAS API
key in plaintext (DOCFIX-175); the live state file is 0600. Invisible to
every prior control because the world-readable check was custody-gated and
the siblings were undeclared. LOGGED, NOT ACTIONED (hard rule 1).

Findings 35 -> 13, all true. Matrix 81 rows. Harness 44/44, creds-audit
15/15, gauntlet ALL GREEN (80), repo-lint 0-fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

D-137 tier 2: privileged sweep completed; permission gap fixed; 6 inferred filenames corrected ...

The block was a PATTERN GAP, not a policy conflict. Bash(ssh * sudo *) was
already in the project ask list, but it needs a literal space before `sudo`
and the command was ssh <host> 'sudo ...' -- the quote meant no rule matched,
so it fell through to the classifier. Added the quoted variants plus a
targeted rule for the privileged invocation; all `ask`, never `allow`.

FLAGGED NOT FIXED (hard rule 1): Bash(ssh * virsh *) has the identical gap.

MEASURED (docs/audit/d137-tier2-privileged-20260726.txt): the region secrets
dir holds exactly admin.apikey/admin.pass/db.pass/lxd-trust.pass at 0600 --
precisely the SEC-020(i) carve-out list; netbox holds admin.pass/api.token/
secret_key at 0600; the SEC-022 shadow stores are confirmed and now audited.
ZERO undeclared files at any declared location: SEC-020 and SEC-022 are
accounted for.

OWNED: the first matrix draft carried SIX filenames I authored plausibly
instead of measuring -- a hard-rule-2 miss. The checker's undeclared-detection
caught every one on first contact with real hosts. Corrected to measured
values, real locations declared, and five colocated non-secret artifacts
declared so the check stays meaningful. Matrix 72 -> 77 rows.

9 findings remain, all true: SEC-021 (3 declaration + 2 on-disk), the ruling-5
conflation, and 3 S5 asymmetries -- dc0's snap power key is the generic
id_ed25519 while dc1's is the dedicated id_dc1_power, the SEC-012/SEC-016
per-DC-isolation divergence made visible.

Harness 35/35, gauntlet ALL GREEN (80), repo-lint 0-fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf

D-137 tier 1 (STATIC): credential matrix, checker, harness -- RED by design ...

Schema amended 10 -> 12 columns (operator-ruled 2026-07-26, "go with the
12-column amendment"): adds `custody` + `notes-ref` and re-keys a row to one
(credential, location) pair. A read-first round-trip check of the three
manifests proved the 10-column form could not carry what D-137 ruling 2
forbids dropping -- and ruling 3 puts four deliberate credential copies inside
--remote's declared locations, where creds-audit.sh:63-67 would report each as
UNDECLARED. Implements ruling 2 ("prose must become MATRIX FIELDS"); OPS under
GA-R3, the five sub-rulings untouched.

Ships: creds-matrix.tsv (72 rows), creds-matrix-notes.md (32 keys),
scripts/creds-matrix.py (S1 schema / S2 both-bounds manifest coverage / S3
render drift / S4 mint-ref resolution / S5 per-DC symmetry / S6 ruling-5
principal invariant / S7 notes integrity), tests/creds-matrix 24/24.
Gauntlet ALL GREEN (80, was 79); repo-lint 0-fail.

RED BY DESIGN, 5 findings -- do not go green by deleting rows: the ruling-5
identity conflation on maas-region-admin (the live SEC-020 defect), 3x
EXPECTED-BUT-ABSENT for SEC-021's declaration half, and the S5 asymmetry on
dc0's divergently-named headend power key. 27 operator-terminal rows are the
FINDING 1 reproducibility debt, counted not faulted.

Two self-inflicted false-greens found and closed during the build: S2
enumerated only matrix-mentioned sites (an empty matrix read CLEAN -- the
sandbox-fidelity-check.py:131-143 defect one level up; now the union of both
sides, regression-locked by T9), and the uniqueness key false-failed two
identities sharing one file (T6).

NOT done, stated so the boundary is unambiguous: tiers 2-3, vm-secret-locations,
the SEC-009 demotion, and the ruling-1 preflight Pn -- whose blocking-vs-deferred
sequencing is an OPEN operator question. No manifest rewritten; no cloud state
touched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HvCyrwvYTTcDYnRErfMsNf