diff --git a/creds-manifests/vr1-dc0.manifest b/creds-manifests/vr1-dc0.manifest index 3311f8d..44a719b 100644 --- a/creds-manifests/vr1-dc0.manifest +++ b/creds-manifests/vr1-dc0.manifest @@ -54,3 +54,5 @@ # [octavia-pki-backup] see creds-matrix-notes.md ## n-pki-backup octavia-pki-vr1-dc0.tar.gz 600 local +# [octavia-reissue-backup] see creds-matrix-notes.md ## n-reissue-backup +vr1-dc0-octavia-pki-.tar.gz 600 local diff --git a/creds-manifests/vr1-dc1.manifest b/creds-manifests/vr1-dc1.manifest index fd4bc46..50a88ea 100644 --- a/creds-manifests/vr1-dc1.manifest +++ b/creds-manifests/vr1-dc1.manifest @@ -41,3 +41,5 @@ # [octavia-pki-backup] see creds-matrix-notes.md ## n-pki-backup octavia-pki-vr1-dc1.tar.gz 600 local +# [octavia-reissue-backup] see creds-matrix-notes.md ## n-reissue-backup +vr1-dc1-octavia-pki-.tar.gz 600 local diff --git a/creds-matrix-notes.md b/creds-matrix-notes.md index 1ae7430..a2a5ab4 100644 --- a/creds-matrix-notes.md +++ b/creds-matrix-notes.md @@ -295,6 +295,47 @@ place. Related: R13 Part 2's reproducibility debt, and the 12 declared secrets with no mint command anywhere in the repo. +## n-reissue-backup +**Pre-reissue restore point for a per-DC Octavia controller LEAF rotation, added 2026-07-30 +(operator-directed, after the reissue was executed on both DCs).** One gzipped archive per +reissue, produced automatically by `scripts/octavia-pki.sh reissue` BEFORE the first byte is +minted, then pulled to that DC's jumphost creds folder by +`phase-01-bundle-deploy.md` step 1.0-REISSUE.4. + +**DISTINCT FROM `n-pki-backup`, and both must exist.** `n-pki-backup` is the GENERATION-time +archive of the whole workspace (1.0-GEN.e), taken once when the DC's CAs were created. +`n-reissue-backup` is taken per ROTATION and additionally contains the DEPLOY OVERLAY, which +the generation archive does not: a reissue changes the workspace AND `overlays/-octavia-pki.yaml`, +and restoring only the workspace would leave the half that actually reaches the charm +unrecoverable. Filename is templated (``) because each rotation writes its own. + +**WHAT IT COSTS.** The same trade `n-pki-backup` records, and it must be read the same way: a +further at-rest copy of BOTH encrypted CA private keys BESIDE their plaintext passphrases, so +the archive's own contents defeat encryption-at-rest. Treat it exactly as the live keys. It is +0600 and lives only in the SEC-009 creds folder; the headend staging copy is removed after a +sha256 comparison, because a truncated `scp` would otherwise leave a verified-looking backup of +nothing. + +**THE HAZARD THAT IS UNIQUE TO THIS ROW -- read before ever restoring one.** The archive +contains `controller-ca.cert.srl`, the CA's ISSUANCE STATE. **Never restore it over a workspace +that has issued since.** Doing so rolls the serial counter BACKWARDS, and the next mint then +issues a certificate carrying a serial the estate already holds -- two distinct certs with the +same issuer+serial, an RFC 5280 violation. If a post-promotion rollback is ever performed, RECORD +the burned serial and advance the counter past it under a separate gated step. Serials burned by +the 2026-07-30 reissue: `...250E` (vr1-dc0), `...674DE` (vr1-dc1); capture +`docs/audit/octavia-reissue-executed-20260730.txt`. + +**STANDING FORWARD REQUIREMENT -- PINNED, and it now covers this step too.** When the +secrets-storage solution is settled, its documented PROCESS AND PROCEDURE for certificate and +credential backup MUST absorb **both** backup steps and **all** the material this reissue path +touches: the per-DC issuing CA and controller CA (key, passphrase, cert), the controller LEAF +key/cert/bundle, the CA serial state, and the deploy overlay -- at BOTH DCs. The jumphost creds +folder is the SEC-009 convention location and is the INTERIM home ONLY. Operator direction +2026-07-30, recorded so the requirement is found by anyone reading the register rather than +rediscovered: the creds and certs produced by this step are to be included in the secrets +workflow planning. Related: `n-pki-backup`'s identical pin, `n-tfstate-backup`, R13 Part 2's +reproducibility debt, and the 12 declared secrets with no mint command anywhere in the repo. + ## n-tenant-dir Per-tenant credential directory outside the `*-creds/` convention (research FINDING 2). Hard tenant isolation applies: D-069 keeps custody off-repo, and D-137 ruling 3 bounds diff --git a/creds-matrix.tsv b/creds-matrix.tsv index 38328cd..21eb0e7 100644 --- a/creds-matrix.tsv +++ b/creds-matrix.tsv @@ -112,6 +112,7 @@ octavia-controller-ca-serial per-DC vr1-dc0 headend controller-ca.cert.srl none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:542 SEC-023 n-outside-convention substrate-tfstate-backup per-DC vr1-dc0 jumphost vr1-dc0-substrate.tfstate.gz none service consolidated stage3 runbook:runbooks/dc-dc-phase2-tofu-dc-substrate.md:783 SEC-023 n-tfstate-backup octavia-pki-backup per-DC vr1-dc0 jumphost octavia-pki-vr1-dc0.tar.gz none service consolidated stage5 runbook:runbooks/phase-01-bundle-deploy.md:658 SEC-023 n-pki-backup +octavia-reissue-backup per-DC vr1-dc0 jumphost vr1-dc0-octavia-pki-.tar.gz none service consolidated stage5 runbook:runbooks/phase-01-bundle-deploy.md:817 SEC-023 n-reissue-backup octavia-pki-overlay per-DC vr1-dc0 headend vr1-dc0-octavia-pki.yaml none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:609 SEC-004 n-overlay-in-clone octavia-issuing-ca-passphrase per-DC vr1-dc1 headend passphrase.txt none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:391 SEC-023 n-outside-convention octavia-issuing-ca-key per-DC vr1-dc1 headend issuing-ca.key.enc none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:395 SEC-023 n-outside-convention @@ -125,6 +126,7 @@ octavia-controller-ca-serial per-DC vr1-dc1 headend controller-ca.cert.srl none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:542 SEC-023 n-outside-convention substrate-tfstate-backup per-DC vr1-dc1 jumphost vr1-dc1-substrate.tfstate.gz none service consolidated stage3 runbook:runbooks/dc-dc-phase2-tofu-dc-substrate.md:783 SEC-023 n-tfstate-backup octavia-pki-backup per-DC vr1-dc1 jumphost octavia-pki-vr1-dc1.tar.gz none service consolidated stage5 runbook:runbooks/phase-01-bundle-deploy.md:658 SEC-023 n-pki-backup +octavia-reissue-backup per-DC vr1-dc1 jumphost vr1-dc1-octavia-pki-.tar.gz none service consolidated stage5 runbook:runbooks/phase-01-bundle-deploy.md:817 SEC-023 n-reissue-backup octavia-pki-overlay per-DC vr1-dc1 headend vr1-dc1-octavia-pki.yaml none service off-manifest-known stage5 runbook:runbooks/phase-01-bundle-deploy.md:609 SEC-004 n-overlay-in-clone tenant-domain-admin per-tenant - jumphost -domain-admin-cred.txt gui human off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:64 SEC-023 n-tenant-dir tenant-cluster-user per-tenant - jumphost -cluster-cred.txt api human off-manifest-known tenant-onboard script:scripts/tenant-onboard.sh:90 SEC-023 n-tenant-dir diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index f02e5e9..e7f887c 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2525,13 +2525,31 @@ - **The hardened re-run guard proved itself on the live tree:** re-running dc0 immediately after returned exit 4 "already correct", explicitly noting *"and verify returns clean"* -- the fix for the defect that had it refusing to repair certs `verify` was failing. - - **STILL OWED (operator-gated, NOT done):** backup custody. Each archive holds BOTH CA - private keys, BOTH passphrases and the overlay, and currently sits on the headend at - `~/octavia-pki/backups/`. Step 1.0-REISSUE.4 pulls them to the jumphost creds folder, - sha256-compares, then removes the headend copy. They are also an UNREGISTERED credential - class (no `creds-matrix.tsv` row, no retention rule, so P5 cannot see them) -- logged, not - actioned. And: **never restore one over a workspace that has issued since** (it carries the - CA's issuance state; burned serials today are `...250E` dc0 / `...674DE` dc1). + - **BACKUP CUSTODY DONE 2026-07-30 (operator-directed), and the archives are now REGISTERED.** + Step 1.0-REISSUE.4 executed: both archives pulled to the jumphost SEC-009 creds folders + (`~/vr1-dc0-creds/`, `~/vr1-dc1-creds/`), **sha256 compared and identical at both ends** + (`e30cab7f...3487` dc0, `17c831da...6fe3` dc1), 0600, each listing 15 entries including the + deploy overlay and the CA serial. Headend staging copies REMOVED (`~/octavia-pki/backups/` + gone); no `.reissue-*` staging residue. + **Register updated:** two new `octavia-reissue-backup` rows in `creds-matrix.tsv` (per-DC, + jumphost, custody `consolidated`, templated filename `-octavia-pki-.tar.gz`), + the derived manifests updated to match, and a new **`n-reissue-backup`** note. Measured + after: tier 1 = 101 rows, **the SAME 5 pre-existing findings**; tier 2 = the same 7 + (dc0-edge-api S2+E1, three power-key asymmetries, the principal conflation, E4's two + uncheckable rows) -- **no reissue-attributable finding, and tier 2 FOUND both archives at + their declared location.** `creds-matrix` harness 65/65. + **THE HAZARD UNIQUE TO THESE ARCHIVES, recorded in the note:** they contain + `controller-ca.cert.srl`, the CA's ISSUANCE STATE, so **restoring one over a workspace that + has issued since rolls the serial counter BACKWARDS** and the next mint reuses a serial the + estate already holds (two certs, same issuer+serial -- an RFC 5280 violation). Serials + burned 2026-07-30: `...250E` (dc0), `...674DE` (dc1). + - **PINNED FORWARD REQUIREMENT, operator-directed 2026-07-30:** the creds and certs this step + produced MUST be included in the secrets-storage workflow planning. Recorded in + `creds-matrix-notes.md` `## n-reissue-backup` (alongside `n-pki-backup`'s identical pin) so + it is found by anyone reading the register rather than rediscovered. Scope of what the + settled solution must absorb: per-DC issuing CA and controller CA (key, passphrase, cert), + the controller LEAF key/cert/bundle, the CA serial state, and the deploy overlay, at BOTH + DCs. **The jumphost creds folder is the INTERIM home only.** - The certs remain INERT by design -- `os-public-hostname` is still set nowhere (R5 refused setting it at Stage 5 as a D-019 repeat). The names are now correct IN ADVANCE of D-106's Stage-7 work arming them.