diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 7798d74..1f941e1 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2271,8 +2271,12 @@ **REMAINING (separate, NOT vault/ovn):** ceph-mon/2 + ceph-radosgw/0 stuck `allocating` (apt-wedge class) -> cascade to ceph-osd/glance/nova-compute; triage after the bookend. Full record: `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. Close sweep: - `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 first-surface). SESSION - CLOSE 2026-08-05 (GA-R4 bookend) -- vault init DONE + ovn-central RESOLVED; D-142 saved. + `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 first-surface). Body: + `docs/changelog-20260805-vault-init-ovn-resolved.md`. Root CA openssl-VERIFIED on the rack + (sha256 75:DF:33:97:...:35:A1; dates confirmed, superseding my earlier self-decode). D-142 + is scan-visible (`ledger-scan`: 4 PROPOSED/OPEN decisions, next-free D-143). voffice1 synced + to HEAD. SESSION CLOSE 2026-08-05 (GA-R4 bookend) -- vault init DONE + ovn-central RESOLVED; + D-142 saved. **>>> SESSION CLOSE 2026-08-03 (GA-R4 bookend). Stage 5 remains OPEN; this is a session bookend, NOT a stage close. <<<** DEPLOY IS MID-STAGE-5: bundle deployed + mostly converged (9 machines started, mysql ONLINE, vault init+unseal+root-CA, ~25 units active, 0 error), diff --git a/docs/audit/stage5-dc0-redeploy-asexec-20260804.txt b/docs/audit/stage5-dc0-redeploy-asexec-20260804.txt index 790ba66..a345b2e 100644 --- a/docs/audit/stage5-dc0-redeploy-asexec-20260804.txt +++ b/docs/audit/stage5-dc0-redeploy-asexec-20260804.txt @@ -260,9 +260,11 @@ 2.3 authorize + root CA: authorize-charm op1/task2 completed; generate-root-ca op3/task4 returned the root CA PEM. vault/0 -> active/idle "Unit is ready (active: true, mlock: disabled)"; vault-mysql-router/0 active. EXIT GATE MET. - ROOT CA (PUBLIC -- no secret custody; as-built metadata only): subject "Vault Root - Certificate Authority (charm-pki-local)"; validity 2026-08-05 02:05:57 -> 2036-08-02 - 01:06:27 UTC (~10y anchor). PEM body NOT stored (public but unnecessary). + ROOT CA (PUBLIC -- no secret custody; as-built metadata only; openssl-verified on the rack + from the pasted PEM, NOT a self-decode): subject "CN = Vault Root Certificate Authority + (charm-pki-local)"; notBefore Aug 5 02:05:57 2026 GMT; notAfter Aug 2 01:06:27 2036 GMT + (~10y anchor); sha256 75:DF:33:97:43:48:17:FE:85:97:1C:EC:60:DB:D5:4B:0D:FB:60:4A:0B:F4: + 60:08:92:E8:12:E5:29:69:35:A1. PEM body NOT stored (public but unnecessary). SECURITY HYGIENE NOTE: the child token printed by `vault token create -field=token` appeared in plaintext in the operator's paste (init keys + root token WERE redacted). It was ttl=10m, unset after use, and long expired by paste time -> dead/benign, no rotation. diff --git a/docs/changelog-20260805-vault-init-ovn-resolved.md b/docs/changelog-20260805-vault-init-ovn-resolved.md new file mode 100644 index 0000000..312b412 --- /dev/null +++ b/docs/changelog-20260805-vault-init-ovn-resolved.md @@ -0,0 +1,62 @@ +# Changelog 2026-08-05 -- vault init DONE + ovn-central RESOLVED; D-142 vault-init QoL saved + +Session body (the "what changed" record; status authority stays in `docs/CURRENT-STATE.md`). +Commits: `ca3c0d3` (engineering + as-built), `8305ceb` (GA-R4 close), plus this close-fix. +No cloud mutation by the agent this session -- the vault init one-shot was operator-run +(secret custody); the agent's changes are all docs. + +## What happened (live, operator-run -- recorded no-secrets) + +- Fresh read-only `phase-02-vault-preflight vr1-dc0` on the dc0 rack -> **PROCEED** (9 + machines started; mysql 3/3 active+ONLINE, 1 R/W; vault fresh; census 66 units, 0 + workload-error, 0 agent-error). +- Operator ran `runbooks/phase-02-vault-bringup.md` Steps 2.1-2.3 VERBATIM on the dc0 rack + (`-m vr1-dc0`, VR0 `-m openstack` head-substituted): init (5 shares + TOKEN_OK, + `2>&1|tee`, pre-init writability probe PROBE_OK, off-host save by hand), unseal 3/3 + (Sealed=false), authorize-charm + generate-root-ca -> **vault/0 active/idle "Unit is + ready"**. +- Root CA (PUBLIC anchor; openssl-verified on the rack from the pasted PEM): + subject `CN = Vault Root Certificate Authority (charm-pki-local)`; notBefore + `Aug 5 02:05:57 2026 GMT`; notAfter `Aug 2 01:06:27 2036 GMT`; + sha256 `75:DF:33:97:43:48:17:FE:85:97:1C:EC:60:DB:D5:4B:0D:FB:60:4A:0B:F4:60:08:92:E8:12:E5:29:69:35:A1`. +- **>>> ovn-central/3,4,5 ALL active/idle "Unit is ready"** (/4 leader ovnnb_db+ovnsb_db, + /5 northd active) -- the OVN NB/SB cluster FORMED. This closes the multi-session + ovn-central cert saga via BOTH fixes landed this cycle: `dc-node-etchosts.sh` Step 1.2b + (CN delivery) + the D-052 re-amendment `''`->metal-admin (reachability). Census 40->47 + active, 0 error. + +## Files changed (docs only) + +- **NEW** `docs/audit/vault-init-qol-proposal-20260805.md` -- full R1-R5 QoL engineering + + hidden-prompt safety analysis + tee-write residual + pre-init writability probe + R3 + harness constraints + operator's verbatim safety constraint. +- **NEW** `docs/audit/queued-findings-20260805-vault-init-close.txt` -- 3 first-surface items. +- `docs/design-decisions.md` -- **D-142 PROPOSED / OPEN** (vault-init workflow QoL sweep; + approved-in-principle, impl deferred; R2 transport unresolved). Distinct from D-068 + (substrate) / D-011.6 (unseal bar). +- `docs/runbook-fold-register.md` -- **F13** (D-138: `-m openstack`->`-m vr1-dc0` + + run-location = DC rack on `phase-02-vault-bringup.md`; scope-stretch stated; OPEN). +- `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt` -- appended the vault-init outcome + (no secrets), root CA as-built (openssl-verified), ovn-central resolution, and the + remaining ceph item. +- `docs/CURRENT-STATE.md` -- 2026-08-05 (CONTINUED) status block + close reference. +- `docs/session-ledger.md` -- GA-R4 close block. + +## Security hygiene + +The child token printed by `vault token create -field=token` appeared in plaintext in the +operator's otherwise-redacted paste (init keys + root token WERE redacted). It was +`-ttl=10m`, `unset` after use, and long expired by paste time -> dead/benign, no rotation. +NOT stored/echoed/committed. Lesson recorded: redact token-create output too. + +## Remaining (separate, NOT vault/ovn) + +`ceph-mon/2` + `ceph-radosgw/0` stuck `allocating` ("waiting for machine", apt-wedge class) +-> cascade to `ceph-osd x4` (Incomplete monitor) -> `glance` + `nova-compute x2` (Incomplete +storage-backend). Triage via appendix-A after this close. + +## Revert + +Docs-only. To revert: `git revert 8305ceb ca3c0d3 ` (or `git checkout d6975fb -- +` per file). No cloud state to unwind -- the vault init was operator-run and is not +reverted by touching the repo. diff --git a/docs/design-decisions.md b/docs/design-decisions.md index a4bd5b3..7dbe629 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -7686,9 +7686,9 @@ ## D-142: PROPOSED -- Vault-init workflow QoL sweep (interaction-shape, VR1-fit, save-gate) -**Status:** APPROVED IN PRINCIPLE, IMPLEMENTATION DEFERRED. Filed 2026-08-05. Operator -approved all of R1-R5 (exact utterance: *"Yes, I approve of all the recommendations so -far"*), then directed that the 2026-08-05 dc0 vault init run on the CURRENT (unchanged) +**Status:** PROPOSED / OPEN -- APPROVED IN PRINCIPLE, IMPLEMENTATION DEFERRED. Filed +2026-08-05. Operator approved all of R1-R5 (exact utterance: *"Yes, I approve of all the +recommendations so far"*), then directed that the 2026-08-05 dc0 vault init run on the CURRENT (unchanged) runbook and the QoL work be implemented + TESTED next opportunity (exact utterance: *"We will complete a vault init using current commands, take the qol and updated script/steps developed and we can test them on the next opportunity"*). **R2's off-host transport is diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 3ae9768..fbf93ab 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -302,5 +302,5 @@ - **D-142 PROPOSED -- vault-init workflow QoL sweep SAVED (approved-in-principle, IMPL DEFERRED per operator "run current commands now, test QoL next opportunity"; R2 off-host transport UNRESOLVED).** Full R1-R5 + hidden-prompt safety analysis + tee-write residual + pre-init writability probe + R3 harness constraints: `docs/audit/vault-init-qol-proposal-20260805.md`. Fold-register **F13** (D-138 `-m openstack`->`-m vr1-dc0`, scope-stretch stated). Distinct from D-068 (substrate) / D-011.6 (unseal bar). - **Hung-apps directive (fulfilled):** the "hung" set was mostly correctly-waiting (ovn certs, designate/octavia/rbd-mirror). GENUINELY stuck: **ceph-mon/2 + ceph-radosgw/0** `allocating` (apt-wedge class) -> cascade to ceph-osd/glance/nova-compute. Logged, not fixed (hard rule 1). - **Security hygiene:** child token printed in the operator's paste (init keys + root token WERE redacted) was ttl=10m/unset/expired -> benign, not stored/echoed/committed. Lesson: redact token-create output too. -- **Durability:** vcloud 0 uncommitted / 0 unpushed at close; repo-lint 0 fail / 1 legacy warn. Root CA + vault init recorded no-secrets in `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. -- **NEXT:** triage the ceph-mon/2 + ceph-radosgw/0 apt-wedge (appendix-A) -> ceph/storage-backend convergence -> continue Stage 5 (phase-03 core verify). Sweep: `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 items). Status ONLY in CURRENT-STATE.md. +- **Durability:** vcloud 0 uncommitted / 0 unpushed at close; repo-lint 0 fail / 1 legacy warn; **voffice1 PULLED to sync** (was at `3321c57`, 4 behind -> `8305ceb`); dc0 rack `~/repo-stage` UNAFFECTED (docs-only session, no staged script touched; `phase-02-vault-preflight.sh` sha `90910dfb` verified == repo today). Gauntlet not owed (docs-only, no script/harness change). Root CA + vault init recorded no-secrets (root CA openssl-verified on the rack) in `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. +- **NEXT:** triage the ceph-mon/2 + ceph-radosgw/0 apt-wedge (appendix-A) -> ceph/storage-backend convergence -> continue Stage 5 (phase-03 core verify). Sweep: `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 items). **Body: `docs/changelog-20260805-vault-init-ovn-resolved.md`.** Status ONLY in CURRENT-STATE.md.