diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 010c6b8..7798d74 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -2270,7 +2270,9 @@ (init keys + root token redacted) was ttl=10m/unset/expired -> benign, not stored. **REMAINING (separate, NOT vault/ovn):** ceph-mon/2 + ceph-radosgw/0 stuck `allocating` (apt-wedge class) -> cascade to ceph-osd/glance/nova-compute; triage after the bookend. - Full record: `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. + Full record: `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. Close sweep: + `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 first-surface). SESSION + CLOSE 2026-08-05 (GA-R4 bookend) -- vault init DONE + ovn-central RESOLVED; D-142 saved. **>>> SESSION CLOSE 2026-08-03 (GA-R4 bookend). Stage 5 remains OPEN; this is a session bookend, NOT a stage close. <<<** DEPLOY IS MID-STAGE-5: bundle deployed + mostly converged (9 machines started, mysql ONLINE, vault init+unseal+root-CA, ~25 units active, 0 error), diff --git a/docs/audit/queued-findings-20260805-vault-init-close.txt b/docs/audit/queued-findings-20260805-vault-init-close.txt new file mode 100644 index 0000000..9795217 --- /dev/null +++ b/docs/audit/queued-findings-20260805-vault-init-close.txt @@ -0,0 +1,28 @@ +QUEUED FINDINGS -- 2026-08-05 session close (vault init + ovn-central resolved + D-142 QoL saved) +First-surface items logged, NOT actioned this session (hard rule 1). Status authority = CURRENT-STATE.md. + +F1 [FIRST SURFACE / LIVE CONVERGENCE ITEM] ceph-mon/2 + ceph-radosgw/0 stuck `allocating` + ("waiting for machine") post-redeploy. Cascade: ceph-mon cannot reach 3-peer quorum -> + ceph-osd x4 "Incomplete relation: monitor" -> glance + nova-compute x2 "Incomplete + storage-backend". Likely the 08-03 apt-cacher-ng / cloud-init wedge class (same shape that + wedged mysql-innodb-cluster/1 -> remediated as mysql/3). NOT vault/ovn-related; vault init + and the ovn-central cert cascade completed independently. TRIAGE: appendix-A apt-wedge + procedure (proxy check + container remove/re-add) after this close. Separate from Stage-5 + completion gating only for OVN; ceph/storage-backend convergence is still owed. + +F2 [FIRST SURFACE / TOOLING GAP] `/background` is unavailable over Remote Control -- an + operator `/background` directive (dispatch a status-check agent) failed to execute and had + to be handled inline. Recording as a first-surface tooling limitation of the multi- + workstation / Remote Control path (see auto-memory multi-workstation-remote-control). Not + deploy-blocking; note for the operator's workflow. + +F3 [CAPTURED IN D-142, restated here for the sweep] R2 off-host transport for + `~/vault-init/init.txt` is UNDEFINED -- the file lives inside the ephemeral vault/0 + container. Surfaced verbally at the 2026-08-05 hand-off; operator saved by hand. Blocks + building the R2 save-gate until the operator's actual off-host practice is known. Home: + D-142 + docs/audit/vault-init-qol-proposal-20260805.md. + +SECURITY HYGIENE (not a finding, a note): the child token printed by `vault token create +-field=token` appeared in plaintext in the operator's otherwise-redacted paste. ttl=10m, +unset after use, long expired by paste time -> dead/benign, no rotation. Lesson recorded in +the as-exec: redact token-create output too. Never stored/echoed/committed. diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 3462314..3ae9768 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -293,3 +293,14 @@ - Gauntlet **ALL GREEN (99)**, repo-lint 0 fail. voffice1 PULLED to sync (`9b49b84`); dc0 rack `~/repo-stage` lacks the new script (redeploy prereq -- sweep F3). Live tests were reversible; model at its captured before-state (residue: certificates:145, /0 holds its proof cert -- sweep F2). - **OWNED:** called the root cause wrong THREE times before right; explained away a null resolvectl result (advisor caught it); two false-negative binding tests (didn't re-fire the relation); nearly wired the fix without confirming a server cert issues (advisor made that test blocking). - **NEXT:** we are at **STEP 3** -- Path M model teardown -> preflight -> phase-01 (with Step 1.2b) -> phase-02 vault init -> converge. Sweep: `docs/audit/queued-findings-20260804-ovn-cert-fix.txt` (4 FIRST SURFACE). Body: `docs/changelog-20260804-ovn-central-cert-fix.md`. Status ONLY in CURRENT-STATE.md. + +## SESSION CLOSE 2026-08-05 -- vault init DONE + ovn-central RESOLVED; D-142 vault-init QoL saved (bounded, GA-R4) + +- Branch `dc-dc-stage5-preconditions`, pushed `d6975fb..ca3c0d3`. NO stage opened/closed; Stage 5 remains OPEN. Scan: **4** PROPOSED/OPEN decisions (D-142 added), **SEC 29**, next-free **D-143 / DOCFIX-209 / BUNDLEFIX-053**. +- **>>> VAULT INIT COMPLETE (operator-run one-shot) + ovn-central RESOLVED end-to-end. <<<** Fresh `phase-02-vault-preflight vr1-dc0` -> PROCEED; operator ran phase-02-vault-bringup 2.1-2.3 VERBATIM on the dc0 rack (`-m vr1-dc0`): init (5 shares + TOKEN_OK, `2>&1|tee`), unseal 3/3, authorize + generate-root-ca -> vault active/idle. Root CA (public): valid 2026-08-05 02:05:57 -> 2036-08-02 01:06:27 UTC. +- **ovn-central/3,4,5 ALL active** (/4 leader ovnnb_db+ovnsb_db, /5 northd) -- OVN NB/SB cluster FORMED. Closes the multi-session saga via BOTH cycle fixes: dc-node-etchosts Step 1.2b (CN delivery) + D-052 re-amendment `''`->metal-admin (reachability). Census 40->47 active, 0 error. +- **D-142 PROPOSED -- vault-init workflow QoL sweep SAVED (approved-in-principle, IMPL DEFERRED per operator "run current commands now, test QoL next opportunity"; R2 off-host transport UNRESOLVED).** Full R1-R5 + hidden-prompt safety analysis + tee-write residual + pre-init writability probe + R3 harness constraints: `docs/audit/vault-init-qol-proposal-20260805.md`. Fold-register **F13** (D-138 `-m openstack`->`-m vr1-dc0`, scope-stretch stated). Distinct from D-068 (substrate) / D-011.6 (unseal bar). +- **Hung-apps directive (fulfilled):** the "hung" set was mostly correctly-waiting (ovn certs, designate/octavia/rbd-mirror). GENUINELY stuck: **ceph-mon/2 + ceph-radosgw/0** `allocating` (apt-wedge class) -> cascade to ceph-osd/glance/nova-compute. Logged, not fixed (hard rule 1). +- **Security hygiene:** child token printed in the operator's paste (init keys + root token WERE redacted) was ttl=10m/unset/expired -> benign, not stored/echoed/committed. Lesson: redact token-create output too. +- **Durability:** vcloud 0 uncommitted / 0 unpushed at close; repo-lint 0 fail / 1 legacy warn. Root CA + vault init recorded no-secrets in `docs/audit/stage5-dc0-redeploy-asexec-20260804.txt`. +- **NEXT:** triage the ceph-mon/2 + ceph-radosgw/0 apt-wedge (appendix-A) -> ceph/storage-backend convergence -> continue Stage 5 (phase-03 core verify). Sweep: `docs/audit/queued-findings-20260805-vault-init-close.txt` (3 items). Status ONLY in CURRENT-STATE.md.