Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / session-ledger.md
@JANeumatrix JANeumatrix 6 hours ago 78 KB DOCFIX-161: final consolidation pass -- one more gap found, stale refs fixed
Raw Blame History

Session ledger -- in-flight work continuity record

Purpose. Long ops sessions on this cloud routinely exceed a single context window and get COMPACTED (sometimes several times). Anything living only in the chat scrollback is lost at compaction. This ledger is the durable, committed record of what is in flight, so any session -- after a compaction, or a fresh one, or the parallel Claude Code stream -- can resume without losing pending work.

How to use it (standing practice).

  1. At session start: read this ledger AND run bash scripts/ledger-scan.sh. Reconcile.
  2. ledger-scan.sh is the DRIFT CHECK -- it derives the open-work it reliably can (PROPOSED/OPEN decisions, OPEN security rows, next-free numbers) straight from the repo. This narrative must not claim CLOSED anything the scan shows OPEN, nor omit what it surfaces.
  3. Update this ledger at every deliverable/commit -- it is a standing deliverable like the changelog. The changelog says what CHANGED; this ledger says what is still OPEN.
  4. Numbers and the machine-derived section below are seeded from a scan; re-run the scan and re-seed rather than editing those by hand.

Machine-derived (re-seed from scripts/ledger-scan.sh; do not hand-edit)

As of 2026-07-09 scan (post-DOCFIX-161, final consolidation pass); re-run the scan to refresh:

  • PROPOSED / OPEN decisions: D-068 (Vault substrate hardening, Roosevelt -- 1.16 ruled out per amendment; off-EOL path OPEN), D-071 (routine update cadence + controller patch policy -- operator to rule; gated on the pre-DC-DC HA/backup session). D-100..D-110 are ALL ADOPTED as of 2026-07-09 (D-102 MERGED into D-101; D-109 confirmed standalone) -- Stage 0 of docs/dc-dc-deployment-workflow.md is CLEARED; see docs/dc-dc-buildout-design.md Section 10 for the ruling record. (D-050 RESOLVED/CLOSED 2026-07-06 via D-051; D-073 APPLIED live 2026-07-06 -- not scan-tracked as open. D-076..D-099 are a RESERVED source-project band, never assigned in this repo, per D-110.)
  • OPEN security rows: SEC-001 (rotate at v1 close, re-ruled 2026-07-06), SEC-003 (custodians + second-person unseal -- DEFERRED; keeps d011-06 MANUAL), SEC-004 (flip repo to private at v1 close -- deferral reaffirmed).
  • Next-free numbers: D = 111, DOCFIX = 162, BUNDLEFIX = 012. (Seeded from the 2026-07-09 scan, post DOCFIX-161.)

Active build (this session)

  • validate.sh D-011 runner -- modular. Foundation committed: scripts/lib-validate.sh (exit contract 0/1/2/3/4, emit, vr_json stderr-safe, run, env hygiene, disruptive gate) + scripts/validate.sh orchestrator (profiles, verdict, --stop-on-fail, --include-disruptive).
    • Batch 1 committed: checks/d011-01-charms, checks/d011-06-vault-unseal (MANUAL until the SEC-003 rehearsal closes).
    • Batch 2 committed: checks/d011-02-vip-jumphost (catalog-derive public origins, TLS+HTTP), checks/d011-03-vip-tenant (ephemeral agnhost pod egress -> keystone VIP, D-035 proof). post-restart profile now fully populated.
    • Batch 3 DRAFTED (mock-tested only, NEEDS LIVE VALIDATION): d011-04-octavia-lb (RR+member additive; amphora failover --disruptive, N+1 headroom-guarded) + d011-05-magnum-e2e (wraps tenant-acceptance + timing). full-d011 profile now COMPLETE.
    • Next: live-validate batch 3 (see Verify-live queue), then item-3/#5-#8 backlog.
    • D-011 AMENDED bar: 1 charms; 2 VIP jumphost; 3 VIP tenant; 4 octavia RR/failover/recovery; 5 magnum e2e + OCCM; 6 second-person manual unseal (attestation, D-069); 7 DROPPED (D-070 supersedes D-012 snapshots); 8 DROPPED (D-019, no Designate).

Script backlog (changelog-tracked)

  • DONE: tenant-offboard.sh, vault-kv-health.sh, cloud-assert.sh (item 3 largely), validate foundation + batch 1.
  • item 3 remainder: fold vault-kv-health.sh into cloud-assert as a section; use the combined gate to regenerate the stale restart-procedure runbook.
  • item 5: scripts/tenant-cluster-create.sh -- generalize the t2-02 stage6 + watch + D-066-evidence wrapper. Not started.
  • item 6: scripts/keystone-policy-drift.sh -- D-051 base_* alignment (LIVE-READ PENDING). Not started.
  • item 7: scripts/cloud-snapshot.sh -- juju-export baseline capture (D-070 retired KVM snapshots but KEPT the export/inventory baseline). Not started.
  • item 8: tests/tenant-acceptance/ harness -- pending (tenant-onboard harness landed).

Register / operator-gated (see docs/handoff-20260703-open-items.md)

  • R-1 / SEC-003: Vault unseal-key custodian assignment (OPEN). Gates V-2 and closes d011-06-vault-unseal (currently MANUAL). Operator input, recorded off-repo.
  • V-2: second-person unseal rehearsal (blocked on R-1).
  • R-2 (D-043 caveat): capi-mgmt-v2 auto-resume exclusion -- rule or accept.
  • R-3: was D-063 -- now CLOSED. The handoff doc still lists R-3 OPEN; update it.
  • D-1: Pattern-A full redeploy (VR0 DC0).

Verify-live queue (batch 3 drafts -- confirm on real cloud before trusting)

  • d011-04 amphora_headroom() field parsing (server show .flavor; hypervisor list --long vcpu/ram) -- safe default HOLDs on any parse miss, but confirm the OK path live.
  • d011-04 OCCM LB-name-contains-service-name assumption; agnhost /hostname round-robin behavior.
  • d011-05 needs a FOIL tenant (2nd tenant) for P3 isolation -- onboard one or set VR_FOIL_APPCRED (acme, the former foil, was offboarded).
  • Sequence: run d011-04 non-disruptively first (RR+member), then --include-disruptive once headroom confirmed.

Logged-not-actioned (small; would vanish at compaction)

  • offboard v2 --sweep-magnum-orphans mode (orphan per-cluster trustee in the magnum domain).
  • offboard stage-3 app-cred idempotency re-run guard.
  • host_href=None barbican observation (exercised fine in stage 6; probably closeable).
  • DOCFIX candidate: sweep repo for other -f json ... 2>&1 stderr-merge hazards (DOCFIX-085 class).
  • D-050 (keystone policyd override) and the D-063-adjacent identity:list_trusts="" hardening remain PROPOSED/OPEN, not actioned.

Jumphost stream -- ops-update-20260705: WINDOW CLOSED 2026-07-05 (addenda 13-16)

First execution of runbooks/ops-update-procedure.md (DOCFIX-086; addendum 13). Logged session ops-update-20260705; checkpointed here at each milestone for cross-stream sync. State as of the Section 2 entry checkpoint:

  • DONE Section 1 (pre-flight): client 3.6.25 / controller 3.6.24 / all 91 agents uniformly 3.6.24 -> target 3.6.25. Worklist = 17 apps. Quiesce clean. Pre-change BOM asbuilt/20260705-102951 committed on a TRUE cloud-assert PASS (required the DOCFIX-087 A7 false-negative fix, addendum 14 -- window was HELD on the operator's Option B ruling until it landed).
  • DONE Section 2.1: controller state backup via juju create-backup -m admin/controller (902MB, checksummed, ~/openstack-baseline/, jumphost-only). NOTE: controller model is admin/controller, NOT /controller.
  • DONE Section 2.2/2.3: controller upgraded 3.6.24 -> 3.6.25 (blind window ~1 min, as expected); post-controller cloud-assert PASS.
  • DONE Section 3: all 91 openstack-model agents + controller machine at 3.6.25, states clean (63 idle / 28 started), nothing stuck.
  • DONE Section 4 G0: keystone 778->817; token probe OK; cloud-assert PASS.
  • DONE Section 4 G1 (11 apps, standing group approval, serial+gated): placement 125->154, nova-cloud-controller 795->823, neutron-api 650->710, neutron-api-plugin-ovn 178->215, glance 642->681, glance-simplestreams-sync 124->152, octavia-diskimage-retrofit 196->232, cinder 733->820, cinder-ceph 533->568, barbican 209->265, barbican-vault 75->99. All probes OK; boundary cloud-assert PASS.
  • DONE Section 4 G2: octavia 441->542; LBs 2/2 ACTIVE/ONLINE; cloud-assert PASS.
  • DONE Section 4 G3: dashboards 728->750 / 59->122 / 120->168; D-044 override INTACT; HTTP+login healthy. RCA: VIP HTTPS dead SINCE DEPLOY (haproxy 443 backend targets vhost-less internal addr; L4 check masks; phase-03 3.3 check fails open) -- NOT a G3 regression; G3 stands (operator ruling; addendum 15). Upstream bug + 2 DOCFIX candidates queued post-window.
  • DONE (coordinated): vault bundle revert 1.16->1.8/stable (BUNDLEFIX-010) + D-068 AMENDMENT recorded; 1.16 ruled out (certs V0/V1, Raft-only, Ceph, BUSL); "off EOL 1.8" remains OPEN. Evidence: docs/D-068-vault-1.8-vs-1.16-analysis.md.
  • DONE Section 4 G4: nova-compute 827->894; hypervisors up; guests byte-identical pre/post.
  • DONE Sections 5-6: post cloud-assert PASS; post BOM asbuilt/20260705-194617 committed; version coherence exact (91 agents @ 3.6.25); BOM diff = 17 expected rev pairs + 1 explained metadata delta (cinder secrets-storage endpoint); appendix-B B.1 fully re-baselined; runbook as-executed corrections DOCFIX-088; D-071 amended (backup exists). WINDOW COMPLETE.
  • DONE follow-on (addendum 17): magnum 70->96 + vault 372->714 refreshed after download-and-diff analysis (mass-rebuild campaign upstream; vault stayed reactive/V0, handlers byte-identical, never sealed). Fleet FULLY CURRENT (zero can-upgrade-to); appendix-B re-baselined from asbuilt/20260705-201118.
  • DONE (addendum 18): D-072 ADOPTED+EXECUTED -- dashboard VIP TLS repaired (cluster binding -> metal-admin; BUNDLEFIX-011 durable; https 200 first time on this build). DOCFIX-089 phase-03 3.3 fail-closed + appendix-A entry. Upstream bug DRAFTED: docs/upstream-bug-draft-dashboard-tls.md.
  • POST-SESSION QUEUE (operator): submit the upstream bug + record LP number in D-072; rule on D-071 (still PROPOSED); Vault off-EOL path (Roosevelt-scale, 1.16 ruled out per D-068 amendment; in-channel 714 applied).
  • Window findings logged for close-out (do not action mid-window):
    1. magnum can-upgrade-to anomaly REPRODUCED (points at magnum-dashboard-122; evidence ~/openstack-baseline/magnum-can-upgrade-anomaly-20260705.json) AND Charmhub magnum 2024.1/stable latest release (r101, 2026-07-01) is s390x-ONLY -- no valid amd64 target; magnum EXCLUDED; report upstream to OpenStack Charmers.
    2. vault NEWLY offers in-channel 372->714 on 1.8/stable -- NOT refreshed (D-068); logged only.
    3. juju create-backup/download-backup EXIST on juju 3.6 -- authoring assumption ("removed in 3.0, expected absent") is WRONG; correct ops-update-procedure 2.1 and the D-071 risk section at window close (DOCFIX candidate).
    4. juju status --format=line omits workload messages on 3.6 (root cause of the A7 false negative; fixed by DOCFIX-087).

Jumphost stream -- session 2026-07-05 (post-window resume)

  • Orientation + reconcile DONE: machine-derived block re-seeded from today's scan (DOCFIX next-free 090 -> 091 was stale); shared-section stale-facts correction appended per the handoff reconcile item. Fences validate 4/0/0.
  • FINDING (logged, not actioned): repo_lint.py L5 "next-free identifiers" info line counts ANY textual mention of D-/DOCFIX-/BUNDLEFIX-NNN across all files, so it is inflated by the deliberate high-numbered decoys in the tests/ledger-scan fixture ("must NOT inflate") and by next-free pointer lines themselves. ledger-scan (heading-based, pointer-excluding) is the numbering authority and is correct. DOCFIX candidate: make L5 reuse ledger-scan's exclusion rules or drop the line. No number consumed. NOTE (2026-07-06 correction): the first version of THIS bullet quoted the decoy tokens literally and thereby inflated ledger-scan's own DOCFIX/BUNDLEFIX next-free (docs/ prose is scan-scanned; only "next-free" lines are excluded). Tokens de-fanged; standing rule: never write identifier-shaped tokens above the real high-water mark in docs/ or runbooks/ prose.
  • Standing by for batch-3 live validation support (verify-live queue).

Jumphost stream -- session 2026-07-06 (operator ruling sweep; addendum 20)

  • RECORDED: D-050 RESOLVED/CLOSED via D-051 (operator ruling); D-073 ADOPTED (identity:list_trusts tightened to modern keystone default -- IMPLEMENTATION + TESTING PENDING, nothing live yet; proposed sequencing: before batch-3 d011-05); D-072 amendment (upstream dashboard-TLS bug WITHDRAWN -- operator judges site misconfig, draft retained); SEC-001 -> rotate at v1 close; SEC-003/004 deferred.
  • PINNED (operator): D-071 ratification and D-068 off-EOL path choice held until A1/A2 (batch-3 entry) complete. Magnum can-upgrade-to upstream report pinned.
  • A1 foil-tenant onboarding APPROVED in principle; tenant-structure model under discussion first (SCS Domain Manager persona confirmation + operator/tenant responsibility split). A2 (d011-04 disruptive gating) held behind A1.
  • IN DISCUSSION (B2): OpenBao evaluation; D-068 item 2 (vault_url cleartext http); D-068 item 3 (AppRole TTL audit + proactive probe) scoping.
  • DELIVERED (addendum 21): DOCFIX-091 docs/tenant-onboarding-contract.md per A1 rulings 1-4 (boundary, roles, intake, dangers, hardening register H1-H5 -- proposals logged, none implemented). A1.4 interpretation pending operator confirm: "SSH access into their domain" read as canary access-proof (H3). Verify-live additions: Horizon manager-role GUI identity probe (contract 5.3).
  • RULED (2026-07-06 cont.): A1.3 H1-H5 ADOPTED as pre-v1-close tasks; A1.4 canary interpretation CONFIRMED + adopted as pre-deployment task. Full onboarding completion package (scripts + internal docs + client-facing drafts + live end-to-end validation) added to handoff section 6 v1-close checklist.
  • RULED (2026-07-06 cont.): clientdocs/ TOP-LEVEL for client-facing drafts; build INTERLEAVES with batch-3 (foil onboarding = live workflow validation); A2 disruptive half PRE-APPROVED conditional on live N+1 headroom PASS.
  • DELIVERED (addendum 22): H1 scripts/tenant-assert.sh + 7-case harness (green); clientdocs/ package DOCFIX-092 (README/intake/welcome/self-service drafts). Remaining in the interleave before foil onboarding: H2 idempotency guard, H3 canary stage, D-073 list_trusts zip implementation (then gated live apply).
  • DELIVERED (operator-away batch, addenda 23-25): H2 re-run guards + H3 canary stage7 (harness 12/12); D-073 implementation STAGED (39-rule zip, drift check PASS -- live apply GATED, block in the D-073 amendment); DOCFIX-093 runbooks/d011-batch3-window-DRAFT.md (the one-window do-doc: D-073 apply + foil stages 0-4 + tenant-assert + canary + d011-04/05 + close-out); H5 embedded; docs/D-068-openbao-assessment-DRAFT.md (B2 research input -- no OpenBao charm exists, MySQL backend removed upstream, V1 goes to Sunbeam not legacy charms; research rec: deadline-bound risk acceptance). Stderr-merge sweep DONE: 10-site inventory logged in addendum 25 (fixes = DOCFIX candidate, not actioned).
  • REPO-SIDE QUEUE now EMPTY except: H4 policy-drift script (main-chat backlog item 6 -- coordinate before building) and the client-docs sweep-on-change rule. EVERYTHING ELSE waits on the operator-present live window (the do-doc).
  • DELIVERED (operator-away batch part 2, addendum 26): DOCFIX-094 stderr-separation fixes (cloud-assert A5 FAIL-OPEN closed + harness proof; onboard resolvers; offboard blind-sweep guard exit 20; 3 runbook paste blocks; acceptance:43 rides its future harness). Vault measurements: charm channel + ssl-* options verified; snap has NO 1.13/1.14 track -- max MPL payload in-charm = 1.12.11; OpenBao memo updated. Orphan-trustee sweep draft NOT started (queued).
  • BLOCKER: juju controller macaroon EXPIRED ~17:10 2026-07-06 -- all juju cmds prompt for password. OPERATOR: juju login -u jessea123 at next presence. Blocked reads queued: keystone policy.json PENDING-LIVE-READ trio; barbican host_href look. (Vault reads above completed BEFORE expiry.) CLEARED later 2026-07-06: auth verified working; both queued reads DONE (see addendum 27 closures).

Jumphost stream -- session 2026-07-06 (cont.): d011-batch3 WINDOW EXECUTED (addendum 27)

  • WINDOW COMPLETE, all gates green (logged d011-batch3, per-command wrap -- operator-ruled Claude Code lane, now in the convention doc as DOCFIX-097): D-073 APPLIED live (verify pair + H4-live PASS proof); foil1 onboarded 10.20.29.0/24 (stages 0-4 first-try, tenant-assert PASS 15/15 after fix, H2 double-SKIP, H3 canary PASS FIP 10.12.7.177/ubuntu); d011-04 PASS both halves (A2 conditional honored; headroom OK on all 3 hypervisors); d011-05 PASS (P1/P2/P3 with foil1); baseline + close-out cloud-assert PASS.
  • In-window checker fixes (each harness-proven then re-run live): DOCFIX-095 tenant-assert owner-context app-cred probe (admin-side list is always-403 by base policy admin_or_owner -- measured, NOT a D-073 effect); DOCFIX-096 d011-04 amphora_headroom --long + display-name hypervisor keys.
  • D-011: batch-3 CLOSED; D-011 CLOSED-except-item-6 (SEC-003 manual unseal rehearsal). v1-close checklist (handoff section 6) now QUEUED.
  • Queued-read closures: keystone trio CONFIRMED (D-064 values explicit in live base policy; overlay comment tags dischargeable -- DOCFIX candidate); barbican host_href CLOSED benign-with-corrected-mechanism (charm renders literal https://None:9312, suppressing the upstream request-derived fallback; catalog unaffected; charm-level fix = future item needing D-NNN).
  • Operator rulings this session: devteam pinned as next tenant, onboarding HELD until a cloud snapshot exists (item-7 cloud-snapshot.sh draft launched); orphan-sweep audit stays rc 0 report-only; clientdocs L7 receipt keeps all 14 files; H4 design defaults accepted as delivered.
  • Worktree drafts DELIVERED (uncommitted, integration queued): orphan-trustee sweep (14/14), clientdocs L7 (17/17), tenant-acceptance harness (16 checks; closes the last stderr-inventory site), H4 keystone-policy-drift (10/10 + first live PASS), clientdocs handover pack (lint clean), cloud-snapshot.sh (in flight). Integration order: after this window commit; tenant-acceptance
    • L7 receipt last (receipt must hash post-integration file states).
  • Open (small): Horizon manual checks on foil1 (operator, browser, non-blocking); capi-test-1 CREATE_FAILED leftover (orphan-sweep candidate); beta-cluster lbtest leftover; repo-lint worktree-exclusion DOCFIX candidate.
  • SCRIPT-QUALITY BATCH LANDED (addendum 33, 2026-07-07): repo-wide sweep (110 scripts) -> operator-ruled fix scope shipped as DOCFIX-112..117: stage2 anti-escalation guard fail-closed (SECURITY -- was fail-open both ways); tenant-offboard hardening cluster (stderr-as-argv, Phase B counter CLOSED after 2 days logged, LB-wait fails loud, cred-file auth_url); validate.sh nonstd-exit HOLD; juju-spaces-check macaroon-vs-absent; pipefail sweep + vault-kv-health rc-test; guard-hook blocklist extension. Gauntlet 38/38. Deferred S-class + minor R items in docs/script-quality-findings-20260707.md (operator has NOT ruled S1-S5/S7/S8).

Jumphost stream -- session 2026-07-08: devteam cluster-template incident (addendum 39)

  • DIAGNOSED + DELIVERED: devteam "Unable to create cluster template" = Public/Hidden flag -> clustertemplate:publish (admin_api) refusal, BY DESIGN; read-only diagnosis, no cloud change. DOCFIX-121 (clientdocs template-creation guidance + Public/Hidden warning) + DOCFIX-122 (appendix-A symptom entry); L7 receipt re-recorded; ~/devteam-package re-rendered (kubernetes.md, self-service-guide.md) for tonight's push.
  • OPEN (operator option, unruled): stage5 for devteam -- devteam-k8s was never delivered (window ran stages 0-4); one gated command (bash scripts/tenant-onboard.sh devteam 5) closes the docs' delivered- template promise; else devteam self-creates per new docs.
  • OPEN (operator, small): role-less GUI test user in the devteam domain (grant member or delete); Role-dropdown question on the dashboard-override test (why load-balancer_admin was submitted) -- evidence folded into the D-076 draft (worktree, uncommitted).
  • TestCluster NoValidHost + decommission audit (addendum 40): devteam 1+3 cluster stalled on RAM capacity (NOT quota); operator deleted it, devteam re-created 1+1 (TestCluster2). QUEUED RULINGS from the audit: (a) beta-cluster teardown (~17GB reclaim) -- cross-stream, main-chat owns the resize-acceptance role; (b) lbtest LB leftover reclaim via kubectl (beta kubeconfig, main-chat asset); (c) foil1 offboard (RECOMMENDED -- dormant shell); (d) quota-vs-capacity expectation (trim testcloud quotas or document in clientdocs) -- unruled.

Jumphost stream -- session 2026-07-08 (cont.): Jenkins+K8s guide overnight (addendum 41)

  • DELIVERED (DOCFIX-123): new clientdocs/jenkins-kubernetes-guide.md -- the single end-to-end Jenkins-to-Kubernetes workflow HUB (Pattern A deploy- into-cluster + Pattern B build-agents; self-create cluster path; kubeconfig- as-Jenkins-credential; NoValidHost sizing; flannel/Public-Hidden/keypair traps; troubleshooting; glossary; cross-links). Flannel hole also closed in tenant-skill kubernetes.md. Guide added to LEAK_FILES (harness change), README updated, L7 receipt 30 files, gauntlet 38/38. Instantiated to ~/devteam-package (devteam values). Operator pushes tonight.
  • OPERATOR RULINGS (on return, 2026-07-08 PM) + status:
    1. devteam cluster: DELETE + DELIVER TEMPLATE ruled. DONE (addendum 42): TestCluster2 confirmed gone + devteam-k8s calico template delivered. 2b. Handover package RE-consolidation (operator re-ruling, addenda 46-47): glossary folded into handover-pack + glossary.md removed + account-table hard de-dup (DOCFIX-128); tenant AI skill aligned to the consolidated package -- 3 missing failure signatures added, capacity caveat, Jenkins- guide integration (DOCFIX-129). Top-level docs 9->8. Package re-instantiated (glossary appendix in handover-pack, skill refreshed). DONE.
    2. Consolidation review M1-M7: ALL APPROVED. DONE (addendum 43, DOCFIX-124): M5 error-holes in self-service-guide; M3 entry point + goal-map in welcome; M1/M2 single-source-of-truth + pointers (softer than hard-delete -- flagged, protects the AI skill + P2 test); M6 Jenkinsfile path/stage fixes; M4 new glossary.md. Package re-instantiation for the changed set = next step. FLAGGED: package handover-pack.md has unfilled {{CUSTODIAN_1/2}} (operator value; pre-existing, devteam placeholder-custodian test).
    3. Gated script fixes: BOTH DONE (addendum 45). DOCFIX-126 offboard E0 always-403 no longer FATALs before Phase F (harness case added, 21/21); DOCFIX-127 onboard cred files now carry dashboard_url (env-overridable, as-built https://10.12.4.58/horizon). LIVE VERIFY STILL GATED: next real onboard/offboard confirms end-to-end (onboard cred-write stages 1/2 have no unit coverage -- pre-existing; logged future harness item).
    4. DOCFIX list CLOSED (operator); handover-pack.md fine as-is (already carries {{DASHBOARD_URL}}/{{AUTH_URL}}).
    5. HANDOVER READY (operator confirmed 2026-07-08). Two flags ACCEPTED for the devteam FIRST INTERNAL TEST: (a) package handover-pack.md keeps literal {{CUSTODIAN_1/2}} -- fine for the test, fill with real names for a real client handover; (b) existing ~/tenant-devteam cred files predate the DOCFIX-127 dashboard_url field (written 2026-07-07) -- fine for today, the dashboard URL is in handover-pack section 3 which the team receives. Repo fully pushed (HEAD db284fc). REMAINING = operator-manual delivery only: copy ~/devteam-package + ~/tenant-devteam to the two custodians in person (guard blocks agent access to cred material).
  • Offboard E0 defect (for the fix): admin-side identity:list_application_credentials is always-403 (base admin_or_owner; measured), so offboard v2 --apply counts 3 false failures and FATALs at exit 23 BEFORE Phase F (cred-dir retirement). foil1 offboard hit this live; cred dir retired by hand. Fix: owner-context-or-skip E0 inventory, do not count known-403 as failure; + harness case.

Jumphost stream -- session 2026-07-08 (cont.): devteam self-service cluster live (post-handover)

  • devteam self-created their first cluster (read-only audit, no cloud change). TestCluster (uuid 92b92eed-afa3-4aaf-ab33-13d3ddeea869) CREATE_COMPLETE + HEALTHY (cluster / infrastructure / controlplane / nodegroup all Ready), created 2026-07-08 17:18 UTC in devteam-prod. 1 master + 1 worker, keypair devteam-key, api https://10.12.7.130:6443. calico + small-first + correct keypair + no Public/Hidden flag -- the DOCFIX-121 dashboard symptom resolved IN PRACTICE on their first real self-service create. (Note: this fresh TestCluster reuses the name of the earlier deleted TestCluster/TestCluster2, addenda 40/42 -- cosmetic.)
  • Built via HORIZON, not the CLI. They created their own template TestTemplate (calico, public/hidden False, keypair baked in = devteam-key) in the dashboard and launched TestCluster from it, rather than the delivered CLI template devteam-k8s (keypair_id None; measured, correct-by-design, still unused). Both are valid Magnum objects.
  • What the Horizon path changes (LOGGED, not actioned; DOCFIX candidate, no number consumed):
    1. Nothing about the running cluster -- same Magnum API object either way.
    2. Explains the keypair-on-template confusion this session: Horizon's Create Cluster Template dialog exposes an OPTIONAL Keypair field, so their GUI-built TestTemplate shows a key while the delivered devteam-k8s (CLI stage5 omits it by design) shows none. Comparing the two in-GUI drove the "no keypair -> must be required" read. It is not; the key is supplied at cluster-create time and their live cluster already carries it.
    3. Doc-gap: jenkins-kubernetes-guide.md is CLI-centric. Their next guide step (Section 3, kubeconfig fetch via openstack coe cluster config) has NO Horizon path -- the supported retrieval is the CLI. (Earlier draft of this bullet claimed magnum-ui "offers a cluster-config download" -- UNVERIFIED, retracted; do not assert it.) The Jenkins integration requires the CLI/app-cred path regardless.
  • ADDRESSED: DOCFIX-130 (addendum 48, COMMITTED f0f11ae; ~/devteam-package guide RE-INSTANTIATED 2026-07-08, 34937 bytes, tokens filled from live-measured AUTH_URL/REGION). Grew from 2 gaps to a full guide pass on operator instruction ("fold and add 1-7... make GUI/CLI paths better defined"). clientdocs/jenkins-kubernetes-guide.md, additive only, no token touched:
    • STRUCTURE: new Section 0 dashboard-vs-CLI selector + On the command line / In the dashboard markers; dashboard equivalents for create (2.3), resize (2.5 new), teardown (8). Rule: lifecycle = either path; everything Kubernetes (kubeconfig onward) = CLI only.
    • GAPS 1-7 closed: registry/imagePullSecret (5.0; VERIFIED no registry service in catalog), ingress-nginx how-to + no-DNS (5.4 new), cluster resize (2.5 new; VERIFIED syntax), persistent storage self-verify (5.5 new), cert expiry read one-liner (3), tooling install + kubectl-skew (1); + ImagePullBackOff and PVC-Pending troubleshooting rows (9).
    • Leak harness CAUGHT two self-inflicted client-doc leaks ("Horizon", "magnum") on first pass -- reworded, now PASS 8/0; gauntlet 38/38.
    • ~/devteam-package NOT re-instantiated (refresh at next package sweep). See the changelog DOCFIX-130 entry. Numbers: DOCFIX next-free now 131.
  • STORAGE OPERATOR FOLLOW-UP (logged, not actioned): delivered cluster template has volume_driver unset + no docker volume (MEASURED). Whether the built cluster gets a default StorageClass from the cinder-CSI default is UNVERIFIED (verifying = kubectl into devteam's live cluster; deferred for tenant isolation). If none, persistent volumes are unavailable out of the box -- candidate template improvement (would need a D-NNN). 5.5 is written self-verifying so the doc is correct either way.
  • CLEANUP QUEUE RECONCILED against live (2026-07-08 read-only audit) -- LIVE IS AHEAD OF LEDGER, several "open" items already resolved:
    • foil1 offboard (was RECOMMENDED, addendum-40 c): DONE -- no foil project/user, no orphan trustee. (Closes the dormant-shell item; also means the gated DOCFIX-126/127 offboard live-verify was NOT exercised by us -- still unverified end-to-end, next real offboard confirms.)
    • capi-test-1 CREATE_FAILED delete (REMAINING-OPEN item 3, was operator-executes): DONE -- no cluster record, trustee reaped.
    • beta-cluster teardown + lbtest LB (addendum-40 a/b, cross-stream/main-chat): DONE -- no beta cluster, no stray LB. (Main-chat to log its side.)
    • Audit state: only devteam TestCluster (CREATE_COMPLETE) + its kubeapi LB + its protected trustee remain; orphan sweep ORPHAN=0 PROTECTED=2 UNPROVEN=0; zero unattached floating IPs.
    • STILL OPEN from the addendum-40 audit: only (d) quota-vs-capacity expectation -- trim testcloud quotas or document the capacity ceiling in clientdocs (unruled, operator). Everything else in that audit is closed.

Jumphost stream -- session 2026-07-08 (cont.): S-class script-quality batch (operator-greenlit)

Operator greenlit the recommended sequence: (1) Phase A stderr fix, (2) S7+S4, (3) S8; HOLD S1/S2/S3/S5 (lib-validate adoption on tenant-onboard/offboard) until the DOCFIX-126/127 live-verify lands. Source: docs/script-quality-findings-20260707.md.

  • DONE -- DOCFIX-131 (addendum 49): tenant-offboard.sh Phase A stdout-only. New tinv() (tenant-scoped inv() twin) for the 3 Phase A coe cluster list captures; wait loop retries on list-error, final check fails closed. Latent bug: merged stderr could false-trip exit 21 on an already-clean tenant. Harness un- excludes coe from warnstderr + new coe-stderr-not-argv assertion (22/22). Gauntlet 38/38, lint 0 fail, L7 re-recorded. Committed + pushed.
  • DONE -- S7 (DOCFIX-132) + S4 (DOCFIX-133), addendum 50. S7: removed dead CF (d011-04) + subsumed vr_is_ipv4 guard (d011-03). S4: three inline trust filters -> one fixture-tested scripts/trust_filter.py (+tests/trust_filter/ 9/9); SDIR added to offboard + acceptance. Gauntlet 39 ALL GREEN. Committed + pushed.
  • S8 RE-SCOPED (bigger than its "S" estimate; operator decision needed before I wire it): the KEYSTONE_VIP literal (10.12.4.50) lives in 6 files, and 3 of them (tenant-onboard/offboard/acceptance) do NOT source lib-net.sh -- so centralizing means adding . lib-net.sh sourcing into client-critical scripts I just modified (DOCFIX-131 + S4), which stacks risk. Breakdown:
    • S8a (FIP pool FIP_START/END -> lib-net.sh): DONE (DOCFIX-134). FIP_POOL_START/END added to lib-net.sh; phase-04 create/verify reference them. Gauntlet 39/39.
    • S8b (KEYSTONE_VIP -> lib-net.sh): DEFERRED -- needs sourcing plumbing into 6 scripts incl. the 3 tenant scripts (onboard/offboard/acceptance) that do NOT source lib-net. Operator decision needed before wiring client-critical scripts.
    • S8c (provider-bundle-check.py restates plane CIDRs): DEFERRED -- design change (python can't source bash -- needs a generated/parsed form).
  • DONE -- DOCFIX-135 (addendum 51): tenant AI skill registry gap. devteam asked (via the assistant) if there's a registry setup step; the DOCFIX-130 registry content was not mirrored into the skill (grep found zero). Added "Images: bring your own registry" to references/kubernetes.md + ImagePullBackOff signature to references/troubleshooting.md; skill harness 8/8, gauntlet 39/39; package skill files re-instantiated. Committed + pushed. (Skill-lag class; watch for it on future guide changes.)
  • DONE -- DOCFIX-136 (addendum 52): tenant AI skill FULL SWEEP vs DOCFIX-130. Operator asked for a full sweep after the registry gap. Grep-audit found 6 more guide topics absent; all added concise (pointer to guide for detail): resize command, kubeconfig cert-expiry (+openssl read), dashboard-vs-CLI split, no-DNS, NEW Persistent storage section, ServiceAccount pointer; + PVC-Pending and cert-expired troubleshooting signatures. Skill now covers the full DOCFIX-130 set (ingress stays recommendation+pointer by design). Harness 8/8, gauntlet 39/39, package re-instantiated. Committed + pushed. STANDING LESSON: guide changes must sweep the skill in the same change -- skill-lag caught twice (DOCFIX-135/136).
  • DONE -- DOCFIX-137 (addendum 53): the standing lesson is now ENFORCEABLE. repo-lint L8 guide<->skill coupling guard: clientdocs/guide-skill-coupling.txt pins the guide's sha256; any guide change FAILS lint (naming the skill files to review) until re-recorded with bash scripts/repo-lint.sh --record-guide-skill-coupling. Harness +5 (28/28), gauntlet 39/39. So the next guide edit CANNOT land without a conscious skill review. Skill-lag closed at the source.
  • S-CLASS BATCH STATUS (operator ruled 2026-07-08): DONE = S7, S4, S8a (DOCFIX-132/133/134). S8b APPROVED -- DO IT (next actionable): centralize KEYSTONE_VIP into lib-net.sh. DONE (DOCFIX-138): KEYSTONE_VIP_DEFAULT in lib-net.sh; all 6 sites reference it (3 forms: KEYSTONE_VIP, KEYSTONE_HOSTPORT, acceptance inline); lib-net source added where absent. Gauntlet 39/39, lint 0 fail. S8 FULLY DONE (S8a+S8b). S8c RULED WON'T-DO (operator: marginal value for as-built constants vs bash<->python emit/parse machinery). S1/S2/S3/S5 (lib-validate) STILL HELD -- operator keeps DOCFIX-126/127 live-verify open for an upcoming opportunity (devteam actively using resources; not now). S6 no-action.
  • quota-vs-capacity (addendum-40 d): CLOSED -- operator ACCEPTS as-documented (DOCFIX-130 sec 2.3/5.5 + skill cover NoValidHost/capacity thoroughly; no quota trim).
  • StorageClass follow-up (DOCFIX-130 sec 5.5): RESOLVED 2026-07-08. Operator authorized full read of devteam's cluster (fully internal domain); scoped via ~/devteam-cluster-openrc (snap needs $HOME for --dir; kubeconfig fetched under $HOME, used, DELETED). devteam TestCluster HAS a default StorageClass: default (default) provisioner cinder.csi.openstack.org, Retain, WaitForFirstConsumer, expansion=true. So the delivered template (volume_driver unset) DOES yield PV support via the cinder-CSI magnum default -- persistent storage works out of the box; no template change needed. sec 5.5 doc is correct and self-verifying; OPTIONAL enhancement: state the expected default cinder class (would trigger L8 skill sweep). Cleanup: kubeconfig removed, no residue.
  • D-076 assessment (operator #6: "tested working w/ devteam domain-admin? check for additional work"): NOT fully working -- optimistic recollection. Revision 1 FAILED (full-page 403s + forced logout), reverted. Revision 2 PARTIALLY passed: identity panels render for the manager persona + session intact, BUT the create-user GUI flow submits a NON-managed role (load-balancer_admin) whose grant 403s, stranding a role-less user (flow not atomic). ADDITIONAL WORK before it can advance: (1) root-cause why the Role dropdown submitted load-balancer_admin (form default-role preselection vs mis-select); (2) add a per-view SEMANTIC harness test asserting create-user grants exactly member (invisible to the existing 50 policy pass/fail assertions); (3) clean up the orphaned test user 9a62f88e331f41adb5c9e41225ec8698 (name test1, CONFIRMED still present in devteam, ZERO roles -- delete or grant member); (4) operator RULING: adopt rev2 (a) vs close GUI/CLI-only (b) + accept/reject the charmhelpers blacklist-hole reliance; (5) another operator-present gated window to re-test after rework. NOTE: live override state UNVERIFIED -- juju macaroon expired (needs juju login); confirm use-policyd-override is reverted to baseline. Draft in worktree agent-a73ec798 (uncommitted). Clients unaffected: CLI identity path is the documented behavior.
  • D-076 PINNED (operator 2026-07-08): do NOT work D-076 while devteam are running their own tests in the env. Deferred until devteam testing completes; no further D-076 investigation/window until then. Draft stays in worktree.
  • test1 orphan SCHEDULED FOR REMOVAL (operator 2026-07-08): approved to delete; staged, NOT run now. Exact gated command (re-verify role-less at execution): openstack user delete 9a62f88e331f41adb5c9e41225ec8698 (name test1, devteam domain, 0 roles measured 2026-07-08). Run at the next convenient window; it is a role-less shell with no resources, so removal does not affect devteam's tests.

REMAINING OPEN WORK (single list for session resume; 2026-07-07, updated post-devteam-window)

  1. Devteam delivery -- last two steps are OPERATOR-MANUAL (everything else DONE, addenda 34-35): intake collected 2026-07-07 (internal test; placeholder custodians operator-accepted; filed jumphost-local ~/devteam-intake-record.md, NEVER commit); quota applied+verified (10/20/51200, runbook standard); pack instantiated to ~/devteam-package (17 files, tokens verified zero, banners stripped; templates in clientdocs/ untouched -- L7 receipt intact). REMAINING (operator): (a) copy the CA into the package: cp ~/vault-init/vault-ca-root.pem ~/devteam-package/omega-cloud-ca.pem (guard correctly blocks agent access to ~/vault-init); (b) deliver ~/devteam-package + the ~/tenant-devteam credential files in-person to the two named custodians.
  2. Horizon contract checks: CLOSED for devteam (2026-07-07 full three-account walkthrough, addendum 38). 5.1 VERIFIED (login + Identity-only view + expected popups); 5.3 verdict: GUI identity probe FAILS as the contract anticipated -- CLOSED via the CLI-path clause (DOCFIX-120 contract amendment); D-075 TLS-leg redirect CONFIRMED (operator browser). Isolation, keypair asymmetry, per-user app-cred visibility, and OBJECT STORAGE (container create/delete -- first live proof, closes the acceptance-checklist caveat) all verified in-GUI. foil1's own 5.1/5.3 rows: surface now proven on devteam; operator may run or waive foil1's pass (non-blocking either way). 2c. Dashboard policy override (GUI identity for manager persona): first live apply FAILED acceptance + REVERTED 2026-07-07; rework IN FLIGHT (background agent, worktree). Zip mechanics + PO: activation worked; panels rendered but page-level checks refused (Domains/ Projects/Users/Groups "not authorized"); Roles panel popped two errors then FORCED LOGOUT. Reverted to baseline in one config cycle (operator-confirmed). Decision draft stays PROPOSED (uncommitted, agent worktree; next-free D claimed at commit). Next: agent rework (per-view rule/target mapping, logout root cause, scope-down to Users+Projects, offline per-view semantic tests), then a second gated apply in a later window. Clients are unaffected: CLI identity path is the documented behavior. 2b. D-075 RULED (option c) + EXECUTED 2026-07-07 (was: root-autoindex finding): redirect / -> /horizon + Options -Indexes via conf-enabled/99-omega-root.conf on the dashboard unit. First attempt (RedirectMatch) caught leaking the backend port in verify and replaced with per-dir mod_rewrite. Delivered as D-075 entry + DOCFIX-118 (phase-03 PER-REBUILD block + appendix-A symptom entry). PER-REBUILD like the D-044 override.
  3. capi-test-1 CREATE_FAILED record delete: RULED YES (2026-07-07), execution handed to operator -- the Claude Code permission classifier (correctly) refuses agent-side deletes of resources it did not create; exact command staged in-session (uuid fae7ec70-712c-4e58-88dd-da0d7b8c89b2, measured this window). After it runs: next tenant-offboard.sh --sweep-magnum-orphans --audit reaps its trustee.
  4. S-class consolidation + minor R items: proposals in docs/script-quality-findings-20260707.md -- present options to operator before building (S1-S5/S7/S8 unruled; S6 ruled no-action).
  5. Pinned decisions (do NOT re-ask): D-071 ratification (pre-DC-DC HA/ backup session); D-068 off-EOL path (Roosevelt-scale); SEC-001/003/004 at v1 close; d011-06 stays MANUAL until SEC-003 rehearsal (gates D-011 FULL close; v1-close checklist QUEUED per handoff section 6).

  6. Cross-stream: main-chat to reconcile its ledger section against addenda 27-33 (its backlog items 6/7/8 delivered here; its Verify-live queue fully cleared; D-050/D-073/D-074 all resolved).

  7. Horizon three-account walkthrough + docs alignment (2026-07-07 cont., addendum 38): operator live-tested all three devteam accounts in the GUI; every observation contract-correct (details in item 2 above). DOCFIX-119: clientdocs aligned to observed GUI reality (5 files by background docs agent -- first-login expectations, CLI user-creation commands, popup signatures, object-storage row de-hedged + acceptance script aligned; harnesses 8/8 + 47/47; L7 receipt re-recorded) + README instantiation guidance (DASHBOARD_URL includes webroot). DOCFIX-120: contract section 5 amended with live verdicts (5.1 verified; 5.3 closed-by-CLI-clause; stage-7 canary marked IMPLEMENTED). Package RE-INSTANTIATED to ~/devteam-package from updated templates (incl. /horizon URL); OPERATOR: re-copy to local + deliver. Dashboard override attempt/revert logged in item 2c.

  8. devteam intake + quota + pack instantiation (2026-07-07 cont., addendum 35): intake walkthrough with operator (7 sections; CIDR + short name pre-settled by rulings); quota set devteam-prod 10/20/51200 (values equal nova class defaults -- set explicitly per runbook "documents the record"); first-ever clientdocs pack instantiation: 17 files to ~/devteam-package via scratchpad script (9 tokens: endpoints MEASURED in-session -- AUTH_URL https://10.12.4.50:5000/v3, RegionOne; dashboard https://10.12.4.58 carried from D-072 as-executed verification). Guard/classifier blocks honored: no ~/vault-init access (CA copy handed to operator), no curl -k probe. LOGGED-NOT-ACTIONED (DOCFIX candidate): tenant-skill/references/day2-operations.md line ~7 explains the {{THIS}} notation in DELIVERED prose, so the literal token survives instantiation; reword at next clientdocs sweep (L7 receipt re-record required).
  9. devteam ONBOARDED (2026-07-07 window ops-devteam-onboard, addendum 34): stages 0-4 + tenant-assert (15/15 PASS) + stage7 canary (SSH via FIP 10.12.7.43, teardown clean) ALL FIRST-TRY on TENANT_CIDR=10.100.0.0/24 (D-074 ruled fallback -- first live use of the fallback path and first client-facing workflow execution; ZERO as-executed corrections needed). Stage-2 anti-escalation guard (DOCFIX-112 batch) first live client run: admin grant denied as designed. Baseline reused per ruling (asbuilt/20260706-224851, not stale). Open cloud-assert PASS (A5-A7 held on first run -- one-shot shell lacked admin scope, known jumphost trait, re-run sourced) and close cloud-assert PASS. Handover creds staged in ~/tenant-devteam (0600, jumphost-only). Pack instantiation HELD on intake answers (item 1 above). capi-test-1 delete handed to operator (item 3).
  10. CLIENT PACKAGE RELEASED (addendum 31, 2026-07-07): operator-requested polish sweep integrated (DOCFIX-110) + banner ruling applied (DRAFT removed; the three per-client templates keep TEMPLATE; README says RELEASED-for-handover). Package is handover-ready pending only per-client instantiation. In flight: repo-wide script quality sweep (report-only agent; findings to operator triage next).
  11. CLIENT PACKAGE COMPLETE (addendum 30): starter kit (DOCFIX-108, 6 deliverables incl. tenancy-audit from the skill delivery; pipeline-smoke + leftover-check deduped away) + tenant AI skill omega-cloud-tenant (DOCFIX-109, leakage-gated) integrated; L7 coverage recursive; receipt 29 files; gauntlet 37/37. REMAINING before handover: operator-requested polish pass over the final clientdocs set (in flight), then devteam onboarding next session instantiates the pack.
  12. D-074 EXECUTED (addendum 29): operator ADOPTED the main-chat CIDR correction PLAN same-day (overlap-allowed tenant CIDRs; client-choice + 10.100.0.0/24 fallback; capi-mgmt stays reserved; L5 print removed). Phase 0 gates honored: allow_overlapping_ips=True measured; overlap+Magnum proof via foil1 (EXACT 10.20.28.0/24 overlap with beta; cluster ~14.5 min; 2 Ready nodes via API-LB while beta served the same CIDR); teardown clean; cloud-assert PASS. Guard rewritten (DOCFIX-106, net_overlap.py, harness 12->17), docs aligned (contract/runbook/intake), D-016 amended, repo_lint L5 next-free removed (DOCFIX-107). FINDING-doc BUNDLEFIX token de-fanged (counter was inflated to 052; restored 012). Phase 3 (devteam onboarding, fallback CIDR) remains NEXT SESSION. Corrections fed back to main-chat via addendum 29: northwind example fictional; runbook filename wrong in PLAN.
  13. INTEGRATED (addendum 28): all six worktree drafts -> main as DOCFIX-098 (H4 drift detector), 099/100 (offboard orphan sweep + Phase E0 guard), 101 (clientdocs handover pack), 102 (cloud-snapshot.sh -- unblocks the devteam snapshot gate), 103 (tenant-acceptance harness + the last stderr-inventory fix), 104 (lint L7 sweep-on-change ENFORCED; receipt live, 17 files). Gauntlet 35/35 on the integrated tree. Main-chat backlog items 6/7/8 are now DELIVERED (by this stream, operator-assigned) -- main chat to reconcile its section.
  14. devteam gate SATISFIED (2026-07-06): cloud-snapshot first live runs both green -- asbuilt/20260706-224851 committed (12 redactions verified; scrubber caught a real embedded CA private key) + 1.1GB controller backup jumphost-local (~/openstack-baseline/, NEVER commit). Operator ruled: devteam onboarding NOT this session -- next session runs stages 0-4 + tenant-assert
    • canary on 10.20.30.0/24 (recommended, unruled), then instantiates the handover pack with dev-team intake answers (7 questions in the DOCFIX-101 entry). Also still open: foil1 Horizon manual checks (contract 5.1/5.3).

Jumphost stream -- session 2026-07-09: new repo openstack-caracal-dc-dc first load (repo-rename sweep)

  • Environment note: this session's shell is Windows (win32), not the Linux jumphost (vopenstack-jesse) CLAUDE.md describes -- flagged to operator. run-tests-all.sh gauntlet on this box shows 23/39 pre-existing failures (missing jq, tty/exit-code quirks) unrelated to any change made this session; repo-lint (the only suite touched) is ALL PASS standalone and in the gauntlet. Confirm actual jumphost behavior at next Linux session.
  • DONE -- DOCFIX-139 + DOCFIX-140 (changelog docs/changelog-20260709-dc-dc-repo-rename-sweep.md): repo-rename sweep (stale openstack-caracal-ipv4 path/URL references repaired to openstack-caracal-dc-dc across the skill, 4 runbook RUN blocks, and the repo_lint.py docstring; clientdocs L7 receipt re-recorded after confirming no client-facing drift) + a real repo_lint.py bug fix (L1 Windows path-separator: the docs/design-decisions.md legacy-carve-out comparison used str(p.relative_to(R)), which never matches on Windows, so the documented WARN silently became a FAIL -- fixed to .as_posix() at all 3 affected sites; harness gained T3b covering the carve-out path directly, 29/29 green). NOT actioned (flagged, larger scope): README.md and environment.md's body content still describe VR0 DC0/v1 single-DC scope wholesale -- awaiting operator direction on whether/when to rewrite for VR1 DC-DC. Uncommitted -- operator has not yet asked to commit.
  • DONE -- DOCFIX-141 (changelog docs/changelog-20260709-repo-agnostic-sweep.md): operator asked to scope the "make scripts repo-agnostic" idea as its own sweep + a lint rule, following directly from the DOCFIX-139/140 experience. New repo-lint check L9 (self-reference guard) FAILS any script/RUN-block/ skill file that hardcodes the current repo's own directory name; added with a 5-case harness (34/34 green). Along the way, narrowed all_text()'s .claude exclusion from "skip all of .claude/" to "skip only .claude/worktrees/" -- the blanket exclusion meant checked-in .claude/skills/ and .claude/hooks/ content was invisible to lint entirely (why the SKILL.md hardcode from DOCFIX-139 wasn't caught). Established a $REPO session-variable convention (documented in runbooks/README.md Conventions) and converted the 8 remaining hardcoded clone-path references (SKILL.md + 4 runbooks) to require it explicitly (fail loud if unset, no inferred fallback, per hard rule 2). environment.md's repo URL is KEPT (documented anchor fact, same as its other facts) but tagged with the new repo-lint: allow-selfref opt-out. LOGGED, NOT ACTIONED: L3/L4/L6/L9 only recognize triple-backtick fences for "in code" detection; d011-batch3-window-DRAFT.md (indented pseudo-code) and tenant-onboarding-v2-DRAFT.md (BEGIN/END block comments) use different conventions and are invisible to these checks -- a pre-existing gap, not introduced here, DOCFIX candidate (teach the checks the other conventions, or standardize runbooks onto fences). Uncommitted -- operator has not yet asked to commit.
  • DONE -- workflow doc + gap register (2026-07-09): docs/dc-dc-deployment-workflow.md added (Stage 0-7 followable checklist + tooling gap register audited directly against the repo) + a companion visual Artifact (https://claude.ai/code/artifact/e752f6dd-23ff-4c59-9ce5-1dedb917e5ea). COMMITTED + PUSHED (374aef5).
  • DONE -- DOCFIX-142 (changelog docs/changelog-20260709-opentofu-scaffold.md): first OpenTofu content in the repo (gap register item 2). opentofu/ network + storage-pool scaffold: dc-planes (six per-DC planes), mesh-link (D-100 dark-fiber triangle), dc-storage-pool modules; DC1 + Office1 wired, DC2 held back pending its CIDR assignment (D-101). Provider schema (dmacvicar/libvirt v0.9.8) verified against its actual current docs this session, not memory -- differs materially from older common examples (network isolation is forward.mode, nested, not top-level mode; domain resource restructured to ~40 args mirroring raw libvirt XML). Node-VM/volume resources, OPNsense, DC2 planes, and tc netem application DELIBERATELY DEFERRED -- schema too unfamiliar to author safely without a real tofu providers schema pass; see opentofu/README.md. Added scripts/opentofu-validate.sh + tests/opentofu-validate/ (2/2 green -- only the guard-clause paths testable without a tofu binary, which is not available anywhere this repo has been worked in this session; the actual fmt/init/validate path is UNTESTED, logged not hidden). Extended repo_lint.py's L1 file-extension set to include .tf (previously uncovered since no Terraform/OpenTofu files existed). repo-lint 34/34 harness green, 0 fail/1 documented warn. Uncommitted -- operator has not yet asked to commit this piece.
  • NEXT ACTIONABLE STEP (logged): on a machine with the tofu binary + registry network access, run scripts/opentofu-validate.sh, then tofu providers schema -json against the initialized provider, and read the real libvirt_domain/libvirt_volume schema directly before authoring the node-VM module.
  • DONE -- OPNsense gap detail (2026-07-09): gap register item 4 split into its 4 separable sub-gaps (network substrate done; VM/image blocked on the domain module; OPNsense's own routing config not started, no mechanism chosen; tc netem a separate OS-level thing blocked on an unruled D-100 parameter). Mirrored into opentofu/README.md + the visual tracker. COMMITTED + PUSHED (0bb6cff).
  • DONE -- DOCFIX-143 (changelog docs/changelog-20260709-repo-lint-crlf-fix.md): fixed the CRLF write_text bug flagged during the DOCFIX-141 commit -- --record-clientdocs-sweep/--record-guide-skill-coupling now write_bytes() with an explicit ascii-encode instead of relying on .gitattributes eol=lf to clean up CRLF after the fact. Verified 0 CR bytes on a live re-record; harness 34/34 unchanged; repo-lint 0 fail/1 warn. COMMITTED + PUSHED (f93dd0d).
  • DONE -- DOCFIX-144 (changelog docs/changelog-20260709-opentofu-node-vm.md): moved into the domain module as requested. Built opentofu/modules/node-vm (blank-disk PXE-boot MAAS-managed node VM, D-103's pattern for Stage 3) -- this time from the provider's own real example .tf files, not doc-summarization alone, after last session deferred libvirt_domain for exactly that confidence gap. Found and fixed a real bug in the already-committed DOCFIX-142 delivery in the process: dc-planes/ mesh-link/dc-storage-pool used classic HCL block syntax (domain { ... }) where the provider's current schema actually needs attribute-style objects (domain = { ... }) -- confirmed once real provider examples surfaced the convention; doc-summarization alone had NOT caught this. All three corrected; opentofu/README.md's schema notes now state the attribute-object convention as a provider-wide assumption. UNVERIFIED, flagged: the per-device boot-order attribute's exact shape (an inference by pattern). Still deferred: the cloud-init/base-image VM pattern (Office1 services + OPNsense), DC2 planes, netem. COMMITTED + PUSHED (3324387).
  • DONE -- DOCFIX-145 (changelog docs/changelog-20260709-opentofu-cloudinit-vm.md): built opentofu/modules/base-image + modules/cloudinit-vm (the OTHER VM pattern, for Office1's own service VMs -- MAAS-region, NetBox, GitBucket). Same high-confidence source as DOCFIX-144 (examples/alpine_cloudinit.tf, fetched as real code). Threaded the base image's .path output across the module boundary as a real attribute reference rather than reconstructing a volume path from pool+name strings -- a cleaner design than the earlier draft, avoiding an unconfirmed-convention guess entirely. user_data/meta_data/network_config are required inputs with no default (Office1 VM configs aren't designed yet; a plausible DHCP default would also silently fail since these planes carry no libvirt DHCP). OPNsense applicability explicitly NOT confirmed -- FreeBSD-based, cloud-init/NoCloud support not the same guarantee as Linux images, flagged prominently rather than assumed. Neither module instantiated (no image source chosen for any service VM). Workflow doc + visual tracker updated. Uncommitted -- operator has not yet asked to commit this piece.
  • DONE -- OPNsense deployment research (changelog docs/changelog-20260709-opnsense-research.md, no DOCFIX -- research delivery, matches the design-doc precedent): operator asked to check OPNsense's actual documentation. Confirmed cloud-init is genuinely unreliable on OPNsense (FreeBSD; OPNsense's own forum consensus), closing the DOCFIX-145 open question. Found the real mechanism instead: the Configuration Importer (official docs) scans an attached volume for /conf/config.xml in a 2-3s boot window; ISO9660 support was added to it specifically for VM/cloud automation (a closed, milestone-targeted, core-dev-assigned GitHub issue) -- mechanically identical to cloudinit-vm's cdrom-attach shape, different payload. Use the nano image for KVM (pre-installed, serial-ready), not the installer images. Fully sourced in opentofu/README.md. NOT built yet: a decompress/convert preprocessing script for the nano image, an ISO9660 config.xml-seed builder, and the real config.xml content -- itemized as the next actionable step. COMMITTED + PUSHED (75ca823).
  • DONE -- DOCFIX-146 (changelog docs/changelog-20260709-opnsense-edge-module.md): operator asked to continue with OPNsense while the research was fresh. Built all three itemized next-steps: scripts/opnsense-prep-image.sh (nano image decompress+convert, no hardcoded mirror -- OPNSENSE_MIRROR_BASE required), scripts/opnsense-build-config-iso.sh (ISO9660 config.xml seed), and opentofu/modules/opnsense-edge wiring both together -- mechanically identical to cloudinit-vm's cdrom-attach shape, no libvirt_cloudinit_disk involved. Explicit lan_network_name/ wan_network_name variables (not an ordered list) to prevent a silent LAN/WAN swap. Both harnesses green (4/4, 3/3) exercising REAL missing-tool guards (qemu-img/genisoimage/xorriso are all genuinely absent this session). Two things flagged, not presented as fact: whether the Configuration Importer's ISO9660 behavior holds on a real boot, and whether devices.interfaces list order reliably maps to LAN/WAN NIC enumeration for this guest. config_iso_path has no default -- real config.xml content per site is still undesigned. Not instantiated in root main.tf. opentofu/README.md + workflow doc + visual tracker updated to reflect BUILT (not just researched). COMMITTED + PUSHED (a3ff07f).
  • DONE -- DOCFIX-147 (changelog docs/changelog-20260709-opentofu-maas-netem.md): operator asked to pull docs and validate before drafting the next modules -- the two remaining unbuilt mechanisms from the OpenTofu scope (MAAS VM-host registration, tc netem application). Researched a real official MAAS provider (canonical/maas v2.7.2) and OpenTofu's own local-exec/terraform_data docs BEFORE writing any code, per instruction. Caught a real design mistake before it happened: read both maas_vm_host (register) and maas_vm_host_machine (compose -- takes cores/memory/disks as INPUTS, MAAS's "Compose machine" pod feature) schemas, cross-checked against D-103's explicit "MAAS does NOT compose new ones" ruling, and built modules/maas-vm-host using ONLY maas_vm_host -- the other resource would have had MAAS and node-vm fighting over VM creation. Also confirmed local-exec runs on the machine invoking tofu apply (Office1 per D-103), not the vcloud host where bridges live -- modules/netem-link wraps its tc qdisc command in an explicit SSH hop because of this, using terraform_data (OpenTofu's own recommended replacement for null_resource) with a destroy-time provisioner to clean up. Neither module instantiated (MAAS zone/pool, vcloud SSH target, and real netem parameters all still pending). opentofu/README.md gained a new sourced research section; workflow doc + visual tracker updated. COMMITTED + PUSHED (ea276c5).
  • DONE -- DOCFIX-148 (changelog docs/changelog-20260709-opentofu-audit.md): operator asked to check back over all current modules, pull docs for anything drafted without it, and review for errors/assumptions. Grepped every "UNVERIFIED"/"inferred"/"assumed" marker across all 9 modules and researched each further rather than re-stating hedges unchanged. Found and fixed a genuine documentation error, not just a hedge: opnsense-edge's comments claimed interface list-order directly sets OPNsense's LAN/WAN role -- researching OPNsense's own interface-assignment model showed role assignment is a SEPARATE, explicit config.xml mapping, not automatic from device order ("vtnet0=WAN" is a convention some guides use, not an enforced rule). Comments + both LAN/WAN variable descriptions corrected; no config.xml has been written yet so this fixes documentation accuracy for whoever writes it next, not current runtime behavior. Confidence UPGRADED (not fully resolved) on three items: node-vm's boot-order shape (confirmed the provider 1:1-mirrors libvirt's own XML, and libvirt's own docs confirm <boot order='N'/>'s single-attribute shape), create.content.url accepting local paths (second independent source), and the genisoimage flags (confirmed standard, cleanly separated from the still-open "does OPNsense's Importer actually read it" question). Confirmed safe: maas_vm_host's zone/pool computed-if-unset behavior. opentofu/README.md gained a new "Audit pass" section. Deliberately did NOT re-research already-well-sourced findings (Importer mechanism, vm_host-vs-vm_host_machine, the syntax-bug fix) -- targeted only real gaps. Also added (same delivery): .claude/skills/openstack-cloud-ops/references/ opentofu-provider-docs.md -- indexes every provider/doc source used building opentofu/ with exact URLs, confirmed versions, default branches, AND the fetch methodology that actually worked this session (Registry doc pages are JS-rendered, use GitHub instead; branch names vary, check default_branch via the API; real example .tf files beat doc-summarized prose for syntax questions). New SKILL.md routing entry; cross-referenced from opentofu/README.md in both directions, no content duplicated (README.md stays the single source of truth for per-module findings). COMMITTED + PUSHED (96ecbad).
  • DONE -- DOCFIX-149 (changelog docs/changelog-20260709-opnsense-config-design.md): operator asked to move into designing the real config.xml content. Fetched OPNsense's own real, currently-shipped config.xml.sample (opnsense/core repo) and its Route.xml static-routes model before drafting. Built opentofu/templates/opnsense-config.xml.tmpl ({{TOKEN}}-parameterized, reusing this repo's existing clientdocs convention) + scripts/opnsense-render-config.sh. The real sample fetch DIRECTLY CONFIRMED the DOCFIX-148 audit finding with an actual example: ships literal placeholder device names inside each interface's own <if> element, proving LAN/WAN role assignment really is the explicit per-block mapping DOCFIX-148 concluded. The renderer is tested END-TO-END, not just guard clauses (8/8 -- it needs no external tool, a first among the opnsense-* scripts) -- and the harness caught a REAL bug before it shipped: the token HOSTNAME collides with bash's own built-in $HOSTNAME variable (unset doesn't actually clear it), silently passing a test that should have failed. Renamed to OPNSENSE_HOSTNAME throughout. Full token legend in opentofu/templates/README.md, marking exactly which values are real (NTP pool, D-106 naming) vs. still pending Stage 0 ratification (D-100/D-101/D-107) vs. only measurable on a real boot (vtnetN assignment) vs. a security requirement (root password hash must be freshly generated, never the stock sample's own shipped default). opentofu/README.md + workflow doc + visual tracker updated. Uncommitted -- operator has not yet asked to commit this piece.
  • DONE -- Stage 0 (decision ratification) RATIFIED 2026-07-09 (changelog docs/changelog-20260709-stage0-ratification.md, DOCFIX-150): operator walked through the 6 buildout-design redline items live (AskUserQuestion, 2 rounds): Ceph size=3-by-default (size=2 only as an explicit Phase-0 fallback); netem same-metro lean; D-102 folds into D-101, D-109 stays standalone; metal-admin GAINS a ULA leg (reverses the stated lean -- a real deviation, not a rubber-stamp); COS per-DC-only confirmed; mirror sync topology independent-per-DC confirmed. All 11 of D-100..D-110 individually flipped PROPOSED -> ADOPTED in docs/design-decisions.md (D-102's content preserved under a MERGED-INTO-D-101 status line, per the append-only discipline -- not deleted). docs/dc-dc-buildout-design.md Section 10 rewritten from "open items" to a ruling record; Section 3's Ceph gate and the stray "Section D-102" cross-reference corrected to D-101. Verified via ledger-scan.sh: D-100..D-110 no longer appear under PROPOSED/OPEN -- only D-068/D-071 remain (separate, pre-existing, unrelated to this gate). repo-lint 0 fail/1 documented warn, harness 34/34. Stage 0 of docs/dc-dc-deployment-workflow.md is CLEARED.
  • Operator context, 2026-07-09 evening: operator stepped away from the workstation for the night, granted blanket git permission for the session, and authorized autonomous subagent use to make maximum progress toward an end-to-end-ready project by morning. Operator clarified this session runs from the WORKSTATION, not the vopenstack-jesse jumphost -- tonight's scope is PREP ONLY (docs, scripts, tests, hardened runbooks); no live infra (juju/MAAS/OpenStack APIs) is reachable or in scope tonight. Operator will switch to the vcloud host in the morning to run actual installs/deployments. This explains (and validates) why run-tests-all.sh shows 23 pre-existing failures in this environment -- all either "jq required" (not installed on the workstation) or live-cloud-dependent suites (cloud-assert, preflight, tenant-*, etc.) -- confirmed via git stash A/B to be identical with and without this session's own changes, i.e. an environment gap, not a regression.
  • DONE -- Tooling gap register #1 CLOSED 2026-07-09 ($DC selector convention, changelog docs/changelog-20260709-dc-selector-convention.md, DOCFIX-151): added lib_net_select_dc() to scripts/lib-net.sh (dc0/dc1 no-op per D-101's inherited-layout ruling; dc2 fails loud -- gap #3 still open) and lib_hosts_select_dc() to scripts/lib-hosts.sh (dc0 no-op; dc1 AND dc2 BOTH fail loud -- no real per-DC host enrollment exists for either yet, an intentional asymmetry vs. the net selector, documented in-line in both files). Backward compatible by construction -- no existing caller of either library is affected. New harness tests/dc-selector/run-tests.sh: 21/21 PASS (caught and fixed a real pipefail-vs-nonzero-exit-code bug in the harness itself while writing it). repo-lint 0 fail/1 documented warn (unchanged); full gauntlet confirmed via git stash A/B to introduce zero new failures. Stage 5's runbook (not yet authored) still needs to actually CALL lib_net_select_dc "$DC" / lib_hosts_select_dc "$DC" -- the mechanism exists, the call sites don't yet.
  • DONE -- Tooling gap register #3 mechanism CLOSED 2026-07-09 (NetBox DC-DC pipeline, changelog docs/changelog-20260709-netbox-dc-dc-pipeline.md, DOCFIX-152): netbox/dc-dc-prefixes-import.py extends the v1 single-site IPv4-only NetBox import to VR1's two-DC dual-stack model (D-101). DC1's v4 planes hardcoded (D-101: inherited unchanged, this is decision text, not an inference); DC2's v4 supernet, the org ULA /48, and the per-DC GUA carve are all REQUIRED env vars with no defaults -- fails loud rather than inventing any of them. The per-plane subdivision scheme for the ULA/GUA blocks is flagged in the script's own docstring as a proposed, unratified convention pending operator review. New harness tests/dc-dc-prefixes-import/ (a fake in-memory NetBox client): 37/37 PASS at the time (later corrected to 40/40 by DOCFIX-160 -- a dead-code bug in the harness itself had left 3 happy-path assertions silently uncounted; see that entry below). Caught and fixed two real authoring bugs at delivery time: (1) example GUA addresses in the first draft had host bits set beyond the /40 boundary, caught by Python's own ipaddress.ip_network(strict=True); (2) the test's own "count CREATED lines" assertion was inflated by the script's unrelated site-creation message also containing the word CREATED -- tightened to match prefix-specific lines. Flagged, not fixed (out of scope): the existing v1 ipv4-prefixes-import.py's role slugs predate the current six-plane model in scripts/lib-net.sh; this new script uses the CURRENT slugs instead. repo-lint 0 fail/1 documented warn (also caught and fixed a CRLF regression introduced by a Python find/replace during test-data cleanup). Full gauntlet 23/45 FAILED -- confirmed identical pre-existing environment-gap failures (jq missing / no live juju-cloud access on this workstation), zero regressions. UNVALIDATED against a real NetBox (none reachable this session). This closes the TOOLING half of gap #3 only -- the literals themselves (ULA /48, GUA carve, DC2 supernet) still need real assignment before this script can be run for real.
  • DONE -- Stage 1 runbook WRITTEN 2026-07-09 (changelog docs/changelog-20260709-dc-dc-phase0-runbook.md, DOCFIX-153): runbooks/dc-dc-phase0-vcloud-prep.md -- command-level Phase 0 (vcloud host prep) steps, matching the phase-00 runbook's style (sequence table, read-only CHECK before every mutation, individually gated mutations, an explicit gate section, delivery checklist). Surfaces a genuine open structural question rather than silently resolving it: opentofu/main.tf's unconditional provider "maas" block may force MAAS credentials to be set at tofu plan time even though this stage's plan touches only libvirt resources -- documented both possible outcomes and what to do for each. Also flags the VR0/DC0-pool-path-collision risk if the vcloud host is the same physical machine being repurposed (buildout design's own framing). NOT YET EXECUTED -- this is the runbook the operator will run tomorrow morning after switching to the vcloud host; nothing in this delivery claims a real run happened. repo-lint 0 fail/1 documented warn (unchanged, prose-only).
  • DONE -- Stages 2, 3, and 5 runbooks WRITTEN 2026-07-09, three of five remaining runbooks delegated to parallel subagents (Stages 2, 3, 4, 6, 7) while Stage 5 was authored directly (the ground-truth/copy-point stage):
    • Stage 2 (changelog docs/changelog-20260709-dc-dc-phase1-runbook.md, DOCFIX-154): runbooks/dc-dc-phase1-office1-standup.md. Every VM- creation step forks explicit Option A (OpenTofu, blocked pending cloud-init content design)/Option B (manual, logged debt). Found a real, previously-unnoticed gap: no Office1-local virtual network is modeled anywhere in opentofu/. Caught and fixed a repo-lint L2 false-flag during review (a correctly-sourced historical D-014 path reference, reworded onto one line so the rule's own EXEMPT pattern applies, not deleted).
    • Stage 3 (changelog docs/changelog-20260709-dc-dc-phase2-runbook.md, DOCFIX-155): runbooks/dc-dc-phase2-tofu-dc-substrate.md. DC1-first; DC2 hard-gated (not "later"). Five !!! callouts naming real constraints (Stage 2 dependency, DC2 gate, no-Roosevelt-analog shim, undecided node sizing, unruled netem params) plus OPNsense tool- presence check. GATE section concludes "CONDITIONALLY MET AT BEST," refusing to claim more than is actually true.
    • Stage 5 (changelog docs/changelog-20260709-dc-dc-phase4-runbook.md, DOCFIX-156): runbooks/dc-dc-phase4-juju-bundle-per-dc.md. Describes adapting (not duplicating) phase-01..08 to run twice, once per DC. DC1 needs zero VIP/CIDR edits (D-101 inheritance); DC2 blocked on the same NetBox literals as Stages 3/4. Mandates lib_net_select_dc "$DC"/ lib_hosts_select_dc "$DC" before every adapted command block touching those libraries (DOCFIX-151's first real consumer). Surfaces two genuine open design gaps rather than inventing around them: the D-101 IPv6 family-matrix overlay (exact charm config option names unconfirmed) and D-109's IPv6-SAN cert-issuance charm support (unverified assumption). All three: repo-lint 0 fail/1 documented warn; NOT YET EXECUTED (prep-only session, no live infrastructure reachable).
  • DONE -- Stages 4, 6, 7 runbooks WRITTEN 2026-07-09 -- ALL SEVEN Stage 1-7 runbooks now complete, tooling gap register item #9 CLOSED:
    • Stage 4 (changelog docs/changelog-20260709-dc-dc-phase3-runbook.md, DOCFIX-157): runbooks/dc-dc-phase3-maas-enlist-deploy.md. Mandates lib_net_select_dc "$DC" then HONESTLY documents lib_hosts_select_dc "$DC" as expected to fail for both DCs until this stage's own enrollment populates real host data -- a deliberate chicken-and-egg. Names 7 real gaps in its own section (no OpenTofu module stands up a per-DC MAAS rack controller VM; three scripts not $DC-parameterized; node sizing undecided; DC2 blocked at the selector layer; no owner for the per-DC mirror; VM-host-discovery MAAS status unconfirmed; DC0- specific tag/fabric literals).
    • Stage 6 (changelog docs/changelog-20260709-dc-dc-phase5-runbook.md, DOCFIX-158): runbooks/dc-dc-phase5-dr-failover-drill.md. Found three real bundle.yaml gaps by grepping it directly (no cinder-backup charm, no ceph-rbd-mirror charm, ceph-mon's rbd-mirror binding on the wrong plane) and makes fixing them the runbook's own first gated mutation. Includes a negative-control drill (partition-only, expected to correctly refuse promotion) before the real hard-down drill. Every Ceph object name is an explicit placeholder.
    • Stage 7 (changelog docs/changelog-20260709-dc-dc-phase6-runbook.md, DOCFIX-159): runbooks/dc-dc-phase6-designate-cos-magnum.md. Quotes the bundle's actual current "NO designate" text verbatim before describing the reversal; correctly identifies D-106's reactivation also reverses the bundle's separate B5 os-public-hostname-dropped posture (two coupled changes, not one); proposes per-DC hostname overlays to keep bundle.yaml DC-agnostic; quotes D-046's domain-setup trap verbatim twice; leaves COS's deployment mechanism as an explicit operator decision point rather than inventing one. All three: repo-lint 0 fail/1 documented warn; NOT YET EXECUTED. Four new tooling-gap-register items (#12-15) added for the cross-cutting findings this whole runbook-authoring pass surfaced (Office1-local network missing, IPv6 family-matrix overlay missing, D-108 bundle gaps, Stage-4 MAAS/multi-rack gaps) -- see docs/dc-dc-deployment-workflow.md.
  • DONE -- Adversarial review pass across the whole evening's work (changelog docs/changelog-20260709-adversarial-review-fixes.md, DOCFIX-160): ran a dedicated fresh-eyes review subagent (no authoring context) across lib-net.sh/lib-hosts.sh's new selectors, netbox/dc-dc-prefixes-import.py, and all seven Stage 1-7 runbooks, instructed to find real defects and cross-check every claim against ground truth directly (not trust the runbooks' own self-report) -- including running both new test harnesses live. Found and fixed 3 real issues: (1) a genuine Python control-flow bug in tests/dc-dc-prefixes- import/test_logic.py's expect_ok() -- a return inside try made the else clause unreachable dead code, silently leaving 3 happy-path assertions uncounted (37/37 was true for what ran, but overstated what was verified; now 40/40 with the fix, independently re-verified via a live python3 -c reproduction before trusting the agent's claim); (2) a self-contradictory pair of adjacent comments in dc-dc-prefixes- import.py about /19-vs-/22 arithmetic (no functional bug, just wrong prose); (3) stale "as of this writing" self-references in the Stage 4 runbook, written before Stage 3 existed and never reconciled once it did. Propagated the corrected 40/40 count into docs/dc-dc-deployment- workflow.md and this ledger's own earlier entry (left the original delivery changelog un-edited as historical record of what was believed at the time). The review explicitly found NOTHING wrong with: the $DC selector asymmetry description (consistent everywhere), the hand- verified ULA/GUA carve arithmetic, Stage 6's bundle.yaml claims (confirmed byte-for-byte), 3+ direct D-NNN quotes (verbatim-accurate), and DOCFIX/D-NNN numbering consistency across every file -- and found no inferred/invented literal value anywhere in the reviewed material. repo-lint 0 fail/1 documented warn; both affected test suites re-run clean (40/40 and 21/21) after the fixes.
  • DONE -- Final consolidation pass (changelog docs/changelog-20260709-final-consolidation.md, DOCFIX-161), closing out tonight's autonomous session: a last full read-through of docs/dc-dc-deployment-workflow.md for internal cross-stage consistency found ONE MORE real gap -- no stage explicitly owns creating Office1's own OPNsense edge VM, despite the topology being explicit that the design is per-site (three sites, three edges: DC1, DC2, AND Office1) -- logged as gap #16, not silently decided. Also fixed two stale cross-references left over from earlier in the same evening (the design-doc summary row's confusing "DONE, PROPOSED" phrasing; the lib-net/lib-hosts row's "Stage 5 runbook still needs to call it," now true of both Stage 4 and Stage 5). Companion Artifact redeployed twice more: all seven stage pills moved to a new "Runbook written" state (a 4th click-cycle state added, distinct from not-started/in-progress/ done), gap register items 1/3/9 marked closed and 12-16 added, and the 37/37-to-40/40 test-count correction propagated in. Final full gauntlet re-run: same 23 pre-existing environment-gap failures as every earlier check tonight (jq missing / no live juju-cloud access on this workstation) -- zero regressions across the entire evening's cumulative changes. repo-lint 0 fail/1 documented warn. Session summary for the operator's morning read: 11 commits (DOCFIX-151 through DOCFIX-161) delivered gap #1 (closed), gap #3's tooling half (closed, data half open), and all seven Stage 1-7 runbooks (gap #9 closed), plus an adversarial review pass and this consolidation. Nothing was executed against real infrastructure -- prep-only session, as the operator specified. The morning's first real action is Stage 1's runbook (runbooks/dc-dc-phase0-vcloud-prep.md) on the actual vcloud host. Genuinely open blockers, roughly in the order they'll be hit: gap #7 (MTU/Ceph disk-budget arithmetic), gap #16 (Office1 OPNsense edge ownership), gap #3's data half (NetBox literals), gap #2's UNVALIDATED status (no tofu binary has run yet), gaps #12-15 (Office1-local network, IPv6 family-matrix overlay, bundle.yaml DR gaps, MAAS/ multi-rack gaps).

State facts to remember

  • beta cluster left at node_count=2 (deliberate; bonus resize acceptance coverage).
  • repo is temporarily PUBLIC for Claude web_fetch (SEC-004) -- flip private at v1 close.
  • Jumphost stream: see "Active window" section above (ops-update-20260705 in flight). Vault stays 1.8/stable (D-068 unruled).
  • CORRECTION (2026-07-05, jumphost stream, per handoff reconcile item): the two bullets above are stale -- ops-update-20260705 is CLOSED (fleet fully current, juju 3.6.25); D-068 is RULED on the 1.16 question (1.16 ruled out; vault stays 1.8/stable rev 714; the off-EOL-1.8.8 path remains OPEN under D-068).

Project-completion (execute after D-011 passes)

  • Consolidate 10 per-phase do-documents into docs/v1-deploy-runbook.md.
  • Set repo visibility PRIVATE (SEC-004).
  • v2-deferred: SSH on GitBucket (port 29418), IPv6 dual-stack, NetBox import bundle.

Cross-session pins (append-only)

  • D-071 completion sweep (2026-07-05, DONE): the ops-update window closed; D-071 was AMENDED (create-backup premise corrected -- juju create-backup/download-backup DO exist on juju 3.6; a 902MB controller backup was taken). D-071 REMAINS PROPOSED -- not ratified to ADOPTED. Operator ratification is pending; the single-controller (no-HA) half of the risk is unresolved by backups alone. See D-071 + its 2026-07-05 amendment.
  • Pre-DC-DC controller HA/backup planning (PINNED): before the Roosevelt (DC-DC) phase, hold a dedicated planning session on Juju controller High-Availability / backup architecture. Backups now exist (per D-071 amendment); controller HA is the open design item. This resolves the single-controller risk recorded under D-071 for production.