diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index f650de4..84f8d78 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -69,6 +69,14 @@ > redeploy (specs in days); this teardown/redeploy is the test of the module-deployment project, > and it compresses the Roosevelt-review Group-4 horizon from far-future to weeks-away (re-run that > assessment against real hardware sheets when they land). +> **CONTAINER-ELIM PASS SCOPED (2026-08-09), execution DEFERRED to a NEW session.** A multi-agent +> READ-ONLY planning pass (phases 0-4; standard 3-5 sonnet workers + 1 fable administrator/phase; +> final fable advisor) is fully scoped as a cold-start handoff: +> `docs/audit/container-elim-pass/SCOPE-AND-EXECUTION-PLAN.md` (+ `phase-prompts.md`). Locked: +> phases 0-4, layered module system (IaC + procedure), Phase-0 proposes the target topology -> +> operator CONFIRMS before Phase 1 (hard gate), operator gets ONE report at the end. Goal: plan the +> dc0/dc1 container-layer elimination for the 10.13 redeploy AND turn the redeploy steps into a +> repeatable layered module workflow feeding the pre-Roosevelt bare-metal test. NOT YET RUN. > > **dc0 checkpoint scope (operator 2026-08-08): "activate + smoke-test"** -- networks + Octavia (1 test > LB) + Designate (1 test zone) + wrap gates (cloud-assert BOM, controller backup, verify-live diff --git a/docs/audit/container-elim-pass/SCOPE-AND-EXECUTION-PLAN.md b/docs/audit/container-elim-pass/SCOPE-AND-EXECUTION-PLAN.md new file mode 100644 index 0000000..9efd8af --- /dev/null +++ b/docs/audit/container-elim-pass/SCOPE-AND-EXECUTION-PLAN.md @@ -0,0 +1,217 @@ +# Container-layer-elimination pass -- SCOPE & EXECUTION PLAN (handoff for a NEW session) + +**STATUS: SCOPED, NOT YET RUN.** This folder is the self-contained execution package for a +multi-agent planning pass. It was scoped 2026-08-09 in a large session and deliberately +DEFERRED to a fresh session (context budget). A new session should be able to run the whole +pass from this file without re-deriving anything. **This is a READ-ONLY PLANNING pass** -- it +produces a plan / change-set / module design. It performs NO teardown and NO live mutation; +execution is the redeploy itself, gated separately. + +--- + +## 0. How the NEW session starts (bootstrap, then begin Phase 0) + +1. Standard bootstrap: `git pull`; `bash scripts/repo-lint.sh`; read `docs/CURRENT-STATE.md` + in full; `docs/session-ledger.md` + `bash scripts/ledger-scan.sh`. +2. Read THIS file end to end, then `phase-prompts.md` in this folder. +3. Confirm the advisor model with the operator (`/advisor`) -- the final review step and the + per-phase administrators assume the advisor/administrator tier the operator intends. As of + 2026-08-09 the `advisor()` tool is **Fable 5** (operator-configured). Name a model only when + confirmed (see the instrument-currency memory terminology caveat -- prior "fable advisor" + labels were actually Opus 5). +4. Begin **Phase 0**. Do NOT skip the Phase-0 operator gate (Section 4). + +--- + +## 1. Purpose & framing + +**Goal.** Plan the ELIMINATION of the dc0/dc1 container layer as part of the imminent +`10.13.0.0/16` re-IP redeploy (D-143), and structure the redeploy so its steps become a +**LAYERED, MODULE-BASED workflow** -- repeatable, less scattered, and directly reusable for +the pre-Roosevelt bare-metal test that follows. + +**Operator directives that frame this pass (verbatim, 2026-08-09):** +- Container layer: *"We will be eliminating the dc0/DC1 container layer. It will not provide + the testing environment I was assuming ... has only caused more trouble than beneficial + data. That will pull all deeper layers up one and simplify the wiring."* +- Follow-on: *"after the next rebuild we will be moving to a pre-roosevelt bare metal test on a + smaller set of hardware ... I will provide hardware specs within a few days ... a good test + of the module deployment project we are developing during the teardown and redeploy."* +- Module meaning (scoping answer): **"Both -- layered module system"** (IaC modules for the + substrate + procedure/runbook modules for the orchestration on top; this pass defines how + they layer into one repeatable workflow). + +**Why now / why a redeploy.** The 10.13 redeploy is a full teardown + rebuild that is already +going to re-carve the substrate. It is the cheapest possible moment to also flatten the +topology and to convert the (currently scattered) deploy steps into modules -- doing it +piecemeal later would repeat the work. + +--- + +## 2. Grounded current-state -- what the "container layer" IS (verify in Phase 0, do not trust this alone) + +Confirmed read-only 2026-08-09: +- **Model B (D-122/D-123):** the OUTER OpenTofu root on vcloud (`opentofu/main.tf`, `qemu:///system`) + creates a per-DC CONTAINMENT VM `vvr1-dc0` / `vvr1-dc1` that **holds the inner libvirt pool** + (`opentofu/variables.tf:153,191` -- "Model B: holds the inner libvirt pool"). The node VMs are + created INSIDE that containment VM by a SEPARATE inner OpenTofu root via the `qemu+ssh` inner + provider (`opentofu/main.tf:26-33`), run from the Office1 headend per D-128. +- **The "container layer" = the `vvr1-dcN` containment VMs + their inner libvirtd + the 6 inner + plane bridges + the transit leg.** Current nesting: `vcloud (outer libvirt) -> vvr1-dcN + (containment VM = inner libvirt) -> node VMs`. +- **"Eliminate + pull layers up one"** = collapse to `vcloud (libvirt) -> node VMs` directly. + The 6 inner plane bridges + transit + `br-ex` "wiring" simplify accordingly. The EXACT target + is Phase 0's job to define and the operator's to confirm. +- **Governing decisions that touch this layer** (Phase 0 maps the full set): D-122 (site shape), + D-123 (Model B containment), D-124 (region<->rack transit), D-125 (per-DC egress / SEC-010 + forwarding), D-126 (base-leg reach), D-127 (autostart), D-128 (operating model / execution + hosts), D-138 (client-in-DC), D-121/D-134 (node layout / octet bands), D-143 (the re-IP the + redeploy executes). SEC-010 (transit forwarding drop) is wiring-sensitive. + +--- + +## 3. Decisions LOCKED for this pass (from the 2026-08-09 scoping exchange -- do not re-ask) + +| Item | Decision | +|---|---| +| Phase structure | **Phases 0-4** (adopted as proposed; see Section 4) | +| Module meaning | **Layered module system** -- IaC (OpenTofu) modules + procedure/runbook modules; the pass defines the layering | +| Target topology | **Phase 0 PROPOSES 1-2 targets; operator CONFIRMS before Phase 1** (HARD GATE) | +| Fleet per phase | **Standard: 3-5 sonnet worker agents + 1 fable administrator** | +| Final review | **fable advisor** (`advisor()` tool) reviews the aggregate findings before the operator report | +| Mutation | **NONE** -- read-only planning pass; produces a plan, gated for later execution | +| Context discipline | agents WRITE durable per-phase docs in this folder and RETURN BOUNDED SUMMARIES (never dump full findings into the orchestrator's context) | +| Reporting | operator gets ONE report at the END (after all phases + the fable advisor); the orchestrator reviews between phases and feeds findings forward | + +--- + +## 4. Phase definitions (0-4) -- objective, workers, administrator, deliverable, gate + +Each phase: spawn the sonnet workers (each owns one dimension) in parallel; when all return, +spawn the fable administrator to synthesize their durable docs into the phase report; the +orchestrator reviews the phase report and feeds it into the next phase. Draft prompts are in +`phase-prompts.md`. + +### Phase 0 -- Baseline & target topology **(ends in an OPERATOR GATE)** +- **Objective:** map the container layer + full blast-radius EXACTLY; propose the target topology. +- **Sonnet workers (4):** + - W0.1 **Substrate/tofu map** -- outer `main.tf` + `modules/` + inner root; how `vvr1-dcN` and + the nested node VMs are created; disk/pool/cloud-init/seed dependencies. + - W0.2 **Network/wiring map** -- the 6 inner plane bridges, transit leg, `br-ex`, MTU + (jumbo / geneve-over-v6 budget), SEC-010 forwarding, D-124/125/126 paths; what "simplify the + wiring" touches. + - W0.3 **Consumer/dependency inventory** -- everything keyed to the containment layer: + `lib-hosts.sh` (VIRSH_POWER_ADDRESS, NIC maps, CARVE_AUX_HOSTS), `lib-net.sh`, + `maas-node-power.sh` (qemu+ssh power addresses), `site-headend-install.sh`, `dc-rack-net.sh`, + the carve scripts, and the D-128/D-138 execution-host assumptions. + - W0.4 **Target-topology options** -- propose 1-2 concrete "layers-up-one" targets (node VMs + on vcloud libvirt directly; where the 6 planes + transit land; capacity/FIT impact; what + breaks; what simplifies), with tradeoffs, for operator confirmation. +- **Fable administrator:** synthesize into (a) the as-is map, (b) the blast-radius inventory, + (c) the target-topology options + a recommendation. +- **GATE (do not skip):** orchestrator presents the target-topology options to the OPERATOR; + operator confirms the target BEFORE Phase 1. This is the one decision that governs everything + downstream. + +### Phase 1 -- Planning review +- **Objective:** what the confirmed target topology + module framing change in the PLANNING. +- **Sonnet workers (3-4):** W1.1 redeploy/teardown planning (D-143 execution, `dc-dc-teardown- + rollback.md`, the phase runbooks); W1.2 deployment-workflow doc + stage/gate structure vs the + module framing; W1.3 teardown->redeploy SEQUENCING with no container layer; W1.4 how planning + artifacts map onto the layered module model. +- **Fable administrator:** planning change-set + module-planning implications. + +### Phase 2 -- Tools review +- **Objective:** what changes in the tooling; what becomes reusable modules. +- **Sonnet workers (4-5):** W2.1 OpenTofu roots/modules (what collapses; what becomes reusable + IaC modules, D-140-aligned); W2.2 `lib-hosts`/`lib-net` containment-keyed values (power + addresses, NIC maps, plane CIDRs) for a flat topology + 10.13; W2.3 carve/power/network scripts + (`dc-node-carve`, `maas-node-power`, `dc-rack-net`, `site-headend-install`); W2.4 module + decomposition -- which tools become procedure-modules; the IaC/procedure layering. +- **Fable administrator:** tools change-set + module decomposition. + +### Phase 3 -- Tests review +- **Objective:** what changes in the harnesses/gates; what new tests the flat topology + modules need. +- **Sonnet workers (3-4):** W3.1 harnesses/gauntlet that assume the container layer; W3.2 + preflight / cloud-assert / stage gates for a flat topology; W3.3 new per-module test harness + requirements (each module ships its own tested harness -- repo discipline). +- **Fable administrator:** tests change-set + per-module test requirements. + +### Phase 4 -- Change synthesis & module-workflow design +- **Objective:** consolidate into the change-set + the layered module workflow; frame the owed decision. +- **Sonnet workers (3-4):** W4.1 consolidated master change-inventory (planning+tools+tests); + W4.2 the layered module-workflow DESIGN (IaC modules + procedure modules + how they compose + into the repeatable redeploy); W4.3 execution sequencing into the teardown->redeploy (module + invocation order), feeding the pre-Roosevelt bare-metal test; W4.4 the owed [ARCH] decision + framing (a D-123 amendment or a new D-number -- the container-elim + module workflow). +- **Fable administrator:** the final consolidated plan + module design + the decision package. +- **THEN:** the orchestrator reviews the whole; the **fable advisor** (`advisor()`) reviews the + aggregate findings; the orchestrator writes the operator report. + +--- + +## 5. Orchestration flow & context discipline + +1. Sequential phases 0->4; **feed-forward** (each phase's administrator report is an input to + the next phase's worker prompts -- earlier findings shape later dimensions). +2. **Every agent writes its output to a durable file in this folder** (`passN-wM-.md` for + workers; `passN-admin-report.md` for administrators) and returns to the orchestrator only a + **bounded summary** (<= ~15 lines: what it found, where the durable doc is, the top items). + This is the load-bearing context-discipline rule -- a multi-phase 25+ agent run must not dump + full findings into the orchestrator's window. +3. **Phase-0 gate is a real stop:** present target-topology options to the operator; do not + start Phase 1 until confirmed. +4. Between phases the orchestrator (main session) reviews the administrator report, reconciles it + against repo ground truth, and adjusts the next phase's prompts if a finding warrants it. +5. **Findings are LOGGED, not executed** (hard rule 1). Any live-state question an agent needs + is READ-ONLY (the operator has authorized read-only live discovery in prior sessions; confirm + it still stands at run time). No teardown, no mutation. +6. Final: fable advisor review -> operator report. The report states the confirmed target + topology, the master change-set, the layered module design, the sequencing into the redeploy, + and the owed [ARCH] decision. + +--- + +## 6. Deliverables & file layout (what the pass produces in this folder) + + docs/audit/container-elim-pass/ + SCOPE-AND-EXECUTION-PLAN.md <- this file (the handoff) + phase-prompts.md <- draft worker + administrator prompts, per phase + pass0-w1-substrate-map.md <- (produced when run) ...per worker + pass0-admin-report.md <- ...per phase administrator + pass1-*.md ... pass4-*.md + FINAL-PLAN.md <- Phase 4 consolidated plan + module design + FINAL-advisor-review.md <- the fable advisor's aggregate review + operator-report.md <- the single report to the operator + +--- + +## 7. Governing constraints the pass must respect + +- **Hard rules 1-4** (CLAUDE.md): current-step-only, no inferred values, verify-before-mutate, + USE-THE-TESTED-ARTIFACT (`docs/tool-index.md` first). This pass writes a plan; it does not run + raw substrate commands. +- **The re-IP (D-143)** is ruled and is the deployment the container-elim rides. Octet-preserving + 10.12->10.13; apex fork B2; lib-net shape (i). The container-elim is ADDITIVE to D-143's + address change -- keep the two changes distinguishable in the change-set. +- **The container-elim itself is an OWED [ARCH] decision** (flagged in `roosevelt-held-decisions- + review-20260809.md` Section 0.1 + CURRENT-STATE): a D-123 amendment or a new D-number. Phase 4 + FRAMES it (options, reconciliation, Roosevelt-delta); it is ruled by the OPERATOR (GA-R5), not + by the pass. +- **Module discipline:** every script/module change ships with its tested harness + a changelog + entry with a revert; repo-lint clean. The pass PLANS this; delivery is the execution session. +- **Roosevelt-delta lens:** the layered module workflow is a Roosevelt deliverable -- design it to + transfer to the pre-Roosevelt bare-metal test and beyond (minimize-delta-to-Roosevelt). + +--- + +## 8. Open inputs still owed from the operator (gather at run time, not blocking the scope) + +- **Target topology detail** -- resolved via the Phase-0 gate (operator confirms Phase-0's + proposed targets). +- **Pre-Roosevelt bare-metal hardware specs** -- "within a few days"; NOT required for this + planning pass (the pass is topology-flattening + module-structuring, hardware-agnostic), but + W4.3's sequencing should note where the specs plug in. +- **Existing "module deployment project" artifacts** -- if any already exist outside this repo or + in flight, point the Phase-2/Phase-4 workers at them so they build ON rather than invent. (At + scope time none was identified in-repo; confirm at run time.) diff --git a/docs/audit/container-elim-pass/phase-prompts.md b/docs/audit/container-elim-pass/phase-prompts.md new file mode 100644 index 0000000..7120324 --- /dev/null +++ b/docs/audit/container-elim-pass/phase-prompts.md @@ -0,0 +1,98 @@ +# Container-elim pass -- draft agent prompts (ready to adapt & spawn) + +Spawn workers with the **Agent tool, `model: "sonnet"`**; administrators with **`model: "fable"`**. +Run each phase's workers in ONE message (parallel). When all return, spawn the administrator. +Adapt paths/line-refs against live repo HEAD at run time. Every prompt inherits the COMMON +PREAMBLE below. + +--- + +## COMMON PREAMBLE (prepend to every worker prompt) + +> You are a READ-ONLY planning agent for the container-layer-elimination pass. Repo: +> `/home/jessea123/openstack-caracal-dc-dc`. Read `docs/audit/container-elim-pass/SCOPE-AND- +> EXECUTION-PLAN.md` first for full context (Model-B container layer; the 10.13 redeploy; +> "eliminate the container layer + pull layers up one"; the LAYERED module goal = IaC modules + +> procedure modules). RULES: (1) READ-ONLY -- no mutation, no teardown, no raw substrate +> commands; findings are logged, not executed. (2) Consult `docs/tool-index.md` before citing +> any operation. (3) Never assert an inferred value -- cite path:line or measured output; if you +> can't verify, say so. (4) WRITE your full findings to the durable file named below; RETURN to +> me only a BOUNDED SUMMARY (<= 15 lines: key findings + the durable-doc path + top risks). +> Do not dump full findings into your reply. + +## COMMON ADMINISTRATOR PREAMBLE (prepend to every administrator prompt) + +> You are the FABLE ADMINISTRATOR for Phase N of the container-elim pass. Read +> `docs/audit/container-elim-pass/SCOPE-AND-EXECUTION-PLAN.md` and the phase's worker docs +> (`passN-w*.md`) in full. Your job: SYNTHESIZE the workers' findings into one coherent phase +> report -- reconcile contradictions, flag gaps/unverified claims, and produce the phase's +> change-set. Adversarially check the workers (a plausible finding is not a verified one). WRITE +> `passN-admin-report.md`; RETURN a bounded summary (<= 20 lines) + the top decisions/risks the +> orchestrator must weigh before the next phase. + +--- + +## PHASE 0 -- Baseline & target topology (workers write pass0-w*.md) + +- **W0.1 substrate/tofu map** (`pass0-w1-substrate-map.md`): Map exactly how the container layer + is built -- `opentofu/main.tf` + `modules/` + `variables.tf` (outer root creating `vvr1-dcN`), + and the INNER root that creates node VMs via `qemu+ssh` inside it. Document disk/pool/cloud-init/ + seed-volume dependencies and the outer<->inner apply ordering. Deliver an as-is substrate diagram + (text) + every file/module involved. +- **W0.2 network/wiring map** (`pass0-w2-network-map.md`): Map the 6 inner plane bridges, the + transit leg, `br-ex`, the MTU/jumbo/geneve-over-v6 budget, SEC-010 forwarding, and the + D-124/125/126 paths that ride the containment layer. Identify precisely what "simplify the + wiring" touches if the containment VM is removed. +- **W0.3 consumer/dependency inventory** (`pass0-w3-consumers.md`): Enumerate EVERY consumer keyed + to the container layer -- `lib-hosts.sh` (VIRSH_POWER_ADDRESS, NIC maps, CARVE_AUX_HOSTS), + `lib-net.sh`, `maas-node-power.sh` (qemu+ssh power addresses), `site-headend-install.sh`, + `dc-rack-net.sh`, carve scripts, and the D-128/D-138 execution-host assumptions. For each: what + breaks / changes when nodes run directly on vcloud libvirt. +- **W0.4 target-topology options** (`pass0-w4-targets.md`): Propose 1-2 concrete "layers-up-one" + targets (node VMs on vcloud libvirt directly; where the 6 planes + transit land; the + capacity/FIT impact; what breaks; what simplifies). Give tradeoffs + a recommendation. This + feeds the operator gate. +- **Administrator** (`pass0-admin-report.md`): as-is map + blast-radius inventory + target-topology + options with a recommendation. Flag the OPERATOR GATE explicitly. + +## PHASE 1 -- Planning review (feed in the CONFIRMED target topology) + +- **W1.1 redeploy/teardown planning**: D-143 execution, `runbooks/dc-dc-teardown-rollback.md`, the + `dc-dc-phase*` runbooks -- what the confirmed target changes. +- **W1.2 workflow/gates**: `docs/dc-dc-deployment-workflow.md` + the stage/gate structure vs the + layered module framing. +- **W1.3 sequencing**: teardown->redeploy ordering with no container layer (what steps vanish/ + reorder). +- **W1.4 module-planning**: how the planning artifacts map onto the layered (IaC + procedure) + module model. +- **Administrator**: planning change-set + module-planning implications. + +## PHASE 2 -- Tools review + +- **W2.1 tofu roots/modules**: what collapses; what becomes reusable IaC modules (D-140-aligned). +- **W2.2 lib-hosts/lib-net**: containment-keyed values (power addresses, NIC maps, plane CIDRs) for + a flat topology + 10.13. +- **W2.3 scripts**: `dc-node-carve`, `maas-node-power`, `dc-rack-net`, `site-headend-install`, carve + tooling -- what changes. +- **W2.4 module decomposition**: which tools become procedure-modules; the IaC/procedure layering. +- **Administrator**: tools change-set + module decomposition. + +## PHASE 3 -- Tests review + +- **W3.1 harnesses/gauntlet**: which harnesses assume the container layer; what changes. +- **W3.2 gates**: preflight / cloud-assert / stage gates for a flat topology. +- **W3.3 new tests**: per-module test-harness requirements (each module ships its tested harness). +- **Administrator**: tests change-set + per-module test requirements. + +## PHASE 4 -- Change synthesis & module-workflow design + +- **W4.1 master change-inventory**: consolidate planning+tools+tests into one change-set. +- **W4.2 module-workflow design**: the layered IaC + procedure module design + how they compose + into a repeatable redeploy. +- **W4.3 execution sequencing**: module invocation order across teardown->redeploy; note where the + pre-Roosevelt bare-metal hardware specs plug in. +- **W4.4 decision framing**: the owed [ARCH] decision (D-123 amendment or new D-number) for the + container-elim + module workflow -- options + reconciliation + Roosevelt-delta (operator rules it). +- **Administrator** (`FINAL-PLAN.md`): consolidated plan + module design + decision package. +- **THEN**: orchestrator review -> `advisor()` (fable) aggregate review (`FINAL-advisor-review.md`) + -> `operator-report.md`.