Author: the Phase-1 administrator (multi-agent pass, SCOPE-AND-EXECUTION-PLAN.md Section 4). Date: 2026-08-09. Inputs: pass1-w1-redeploy-teardown.md, pass1-w2-workflow-gates.md, pass1-w3-sequencing.md, pass1-w4-module-planning.md -- read in full, adversarially cross-checked against repo ground truth (checks logged in Section 1). Baseline consumed: pass0-admin-report.md incl. Section 7a gate outcome -- Option 1 CONFIRMED (flat node VMs on vcloud libvirt + small non-hypervisor vr1-dcN-client VM per DC), cross-DC handling (a) CONFIRMED (new vcloud-level host isolation control + --check gate + SEC-NNN row), MAAS region stays on vr1-dcN-maas-01, rack-controller-remainder placement OPEN, all riding D-143. READ-ONLY synthesis; no mutation; findings are logged, not executed.
W1.2's Stage-2-vs-Stage-3 correction: CONFIRMED, load-bearing. Read directly: docs/dc-dc-deployment-workflow.md:60-88 -- Stage 2 is the Office1 SITE standup, its Owns line is D-114 (site containment VM voffice1 + MAAS-composed LXD VMs), not D-123. docs/dc-dc-deployment-workflow.md:149-158 -- Stage 3 is the per-DC substrate, Build line verbatim: "D-123 Model B (TWO OpenTofu roots + a bootstrap gate between)" (:154). Stage 3 is the stage container-elim restructures; Stage 2 is untouched. D-114's containment pattern (voffice1) is a SEPARATE, KEPT decision -- the project carries two different "containment VM" patterns going forward (D-123's retired, D-114's kept) and the Phase-4 change-set must name that explicitly so name-similarity does not sweep Stage 2 in. Any earlier prompt text saying "Stage 2 restructures" is wrong; every table below keys off Stage 3.
Root-topology fork: NOT a contradiction, but W1.3's headline over-asserts -- reconciled. W1.1 Sec 5 item 1 leaves "one merged root vs shared-outer + per-DC roots" OPEN (its destroy commands are marked illustrative for exactly this reason). W1.3's B.4 step 8 headline says "ONE FLAT tofu root, ONE apply cycle" -- but W1.3's own risk #5 explicitly leaves one-state-per-DC vs one-state-total open ("this document leaves that split as a Phase-4 decision"). Ruling of this synthesis: the sequence holds under EITHER root shape. The invariant B.4 actually establishes is: one apply CYCLE per DC, run on vcloud against local qemu:///system, no bootstrap-gate seam, no inner/outer ordering, no qemu+ssh dial. What the fork changes is only (i) how destroy SCOPING is achieved (-target set vs "pick the root"), (ii) per-state blast radius (~38 domains x 2 DCs under one state vs split), and (iii) the exact wording of the teardown primitive. Read W1.3's "ONE FLAT root" as "one flat apply scope," not a resolved root design. Pointer discrepancy also reconciled: W1.1 says Phase 2/W2.1 decides; W1.3 risk 5 says Phase 4 -- resolution: W2.1 (tofu module design) DESIGNS the root split; Phase 4 RATIFIES it in the module-workflow synthesis. Carried to Phase 2 as explicitly open (Section 7). Interaction with the (a) control: under per-DC roots the second DC's apply is a discrete event the control can precede; under one merged root the FIRST full apply may create both DCs' planes at once -- so the only fork-robust ordering is "control installed + verified before ANY flat substrate apply" (Section 3).
Two SEC-010 successors: CONFIRMED distinct, BOTH OPEN, neither dropped. All three sources agree (pass0 Section 5 "two controls"; W1.1 phase-2 table step-B row "related but distinct"; W1.3 B.3 vs B.5 step 17 "do not read B.3 as closing SEC-010's full scope"):
vvr1-dcN's transit NIC (+ voffice1 peer); the qemu+ssh purpose dissolves but operator ssh -J and Office1-originated flows (e.g. MAAS rack enrollment traffic) still ride the transit -- WHICH ends get the drop (client VM + voffice1?) is OPEN (pass0 Section 6 item 5). Sequenced at W1.3 B.5 step 17, deliberately separate from B.3. Neither exists yet; both are owed artifacts (Section 6).The (a) control consolidated into ONE design requirement -- Section 3.
D-143 orthogonality: holds as "every diff attributable," NOT as "no intersections." W1.1's framing (D-143 = value/data substitution; container-elim = shape change) is correct at the structural level and the attribution discipline is sound. But four line-items are genuinely [both] and must carry dual labels (W1.3 Part C + W1.2 Stage-4 row):
D-128 amendment: REAL, not a doc-currency nit -- verified. docs/design-decisions.md: 5354-5364 read directly: Plane 2's DEFINITION includes "the INNER tofu root (opentofu/vr1-dc0-substrate/, qemu+ssh FROM Office1 into vvr1-dc0, R-5)" executing on voffice1. Under Option 1 that object ceases to exist -- the entire substrate build becomes Plane 1 (vcloud-local), and Plane 2 shrinks to MAAS/NetBox (with juju/openstack already moved to the DC client by D-138). That is a substantive scope change to a RULED decision's own definition, not stale prose. Flagged for the Phase-4 [ARCH] decision framing: the D-128 amendment rides alongside the D-123 amendment/new-D, the D-125 retirement, and the D-138 concrete-host change.
Worker-claim verification + contradictions:
runbooks/ dc-dc-phase6-designate-cos-magnum.md:437-444 says "Run them from the Plane-2 host (voffice1) per D-128" / "CHECK ... from the Office1 headend (voffice1)" before a juju status command. Its U9 measurement (2026-07-27) predates D-138 (2026-07-30); phase4's own RUN-LOCATION table records the correction. A pre-existing stale-D-138 defect the rewrite sweep must also fix -- not a new container-elim delta.vm-host"; the workflow doc's Stage-3 State note (:164-166) records that maas-vm-host registration was REFUTED for DCs and replaced by per-machine virsh power (D-103/D-123 amendments 2026-07-20; W1.1's Step-D row + DOCFIX-179 agree). Correct reading: the mechanism is per-machine power_type=virsh; only the power-address VALUE re-derives from "dial the containment VM's libvirtd" to "dial vcloud's own libvirtd" (lib-hosts.sh VIRSH_POWER_ADDRESS*, pass0 rows 4-5). The sequence position is right; the noun is not.vm-secret-locations, SEC-026/-028/-029 pointers) re-point to the client-VM host class; the credential MATERIAL is revoked with the old host (Part A step 3) and freshly minted on the new one. No secret material moves between hosts.dc-substrate-flat apply; W1.4 classifies it L1 (a cloudinit-vm instance, same module type as voffice1/edges). Layer CLASSIFICATION vs apply GROUPING -- both can hold; Phase 2's module design picks the call site.variables.tf:143-150, already flagged at pass0 check 5a). W1.2's Gap-#20 verdict ("probably unchanged") is properly hedged as needing re-measurement post-build -- carried as owed, not as fact.opentofu/main.tf) resolved to the cited lines.| Runbook | Verdict | Change (all against confirmed Option 1) |
|---|---|---|
dc-dc-teardown-rollback.md |
REWRITE (same weight as its own 2026-07-16 "MODEL B RESHAPE" banner) | Two-clones/two-hosts preamble DELETED (one clone, vcloud); Path A root-choice becomes shared-outer vs per-DC-flat (per root-fork resolution); Step-4 verify collapses to ONE vcloud-local virsh block (the "vcloud grep passes against an intact DC" trap goes moot); Path B "everything" = N+1 roots on ONE host, qemu+ssh guard branch dropped; rollback-tree Question 0 redrawn (fewer roots); item 5's virsh destroy vvr1-dcN lever needs a REPLACEMENT artifact (Section 6 #2); mesh-leg caveat REWORDED (no qemu+ssh path; still carries client-VM reach + ssh -J); Paths M and C are UNCHANGED as procedures (above the substrate); footer needs a fresh re-measurement pass against the new module tree at delivery |
dc-dc-phase0-vcloud-prep.md |
NO CHANGE | Zero containment hits (W1.1-verified) |
dc-dc-phase1-office1-standup.md |
NO CHANGE | Its containment VM is voffice1 under D-114 -- out of scope (check 1) |
dc-dc-phase2-tofu-dc-substrate.md |
HEAVIEST REWRITE | Step B (bootstrap gate) ELIMINATED WHOLESALE; Step C (inner apply) MERGES into Step A (one apply, one root-scope, one host, one state); the DC-substrate USAGE of expose_nested_virt drops (vvr1-dcN's true-setting; the module VARIABLE stays -- voffice1 consumes it per the Stage-2 Build line, and the client VM sets it false); D-125 bridge-in rows DELETED (modules/wan-bridge dead for VR1; edge WAN -> direct vr1_dcN_uplink NAT, same /24); Step D loses the "inner virsh" framing (DOCFIX-179 deferral stands); netem caveat reworded only; step list re-grouped fresh-linear (no A-E lettering needed); definition-of-done re-homed off --host-nodes; Step-13 backup = ONE state file on vcloud |
dc-dc-phase3-maas-enlist-deploy.md |
LOW DELTA | Two SSH-jump-target lines (:424,430): far end becomes the ruled placement host (client VM OR maas-01 -- OPEN, do not pre-pick); mechanics otherwise topology-agnostic (per-machine virsh power; only the power-address VALUE changes) |
dc-dc-phase4-juju-bundle-per-dc.md |
MODERATE | RUN-LOCATION table gets its THIRD correction: row 1 juju/openstack CLI -> vr1-dcN-client ("the rack" is retired vocabulary); row 2 (voffice1: maas/NetBox/tofu) and row 3 ("never vcloud") unchanged -- tofu's Plane-2 half shrinks per the D-128 amendment; staged-scripts caveat re-points its noun (client VM still has no repo clone); below-juju-destroy caution re-points at the rewritten teardown runbook |
dc-dc-phase5-dr-failover-drill.md |
NO DIRECT CHANGE | Inherits phase4's table |
dc-dc-phase6-designate-cos-magnum.md |
RIDE-ALONG FIX | :437-444 pre-existing stale-D-138 defect (verified, check 7) -- fix in the same sweep as phase4's table; not a container-elim delta |
dc-dc-office1-service-reip.md |
OUT OF SCOPE | D-114 territory |
docs/dc-dc-deployment-workflow.md)| Stage | Change |
|---|---|
| Stage 1 | Gate content UNCHANGED (nested-KVM still needed for Stage 2's voffice1). Relationship note: the six per-DC planes RETURN to vcloud level -- Stage 3 reconverges onto Stage 1's own execution shape. Recommended new owner of the (a) control (Section 3) |
| Stage 2 | NONE (D-114; check 1). Phase-4 change-set must state the two-containment-patterns distinction explicitly |
| Stage 3 | THE restructured stage. Build line: Model-B two-root text -> single flat apply (planes + edge + 12 node VMs + client VM); also fix stale "~416 GiB" en route to deleting the sizing. Gate line: depth-4 boot gate -> depth-2; bridge-in isolation test -> direct-NAT egress assertion; add the (a) --check. Owns line: D-123/D-125 phrasing VOID pending the Phase-4 ruling; D-124 survives only for the client VM's transit leg. Reuse-vs-new: "NEW, no precedent" should be revisited -- the flat shape is MORE reusable, closer to Stage 1's |
| Stage 4 | Placement, not mechanism: gate-line prose + G17 literal need the ruled artifact-source host + 10.13 address (dual-cause, check 5) |
| Stage 5 | No gate-content change; every literal naming the containment VM's transit IP re-points to the client VM (e.g. docs/CURRENT-STATE.md:7829 "openstackclient ... ON THE dc0 RACK (172.31.0.2)"). D-140 keeps Stage 5 a PROCEDURE module -- settled constraint, not open |
| Stages 6-7 | No container-layer dependency found (verified by W1.2, not assumed) |
| Gap register | #2 reshapes (first Stage-3 exercise becomes a single flat apply; wan-bridge deleted; inner roots retire); #17 closing-mechanism note becomes HISTORICAL (doc-currency addendum owed at ruling time); #19b unaffected; #20 verdict RE-VERIFY post-build (its own expiry clause triggers); #21, #22 unaffected (checked, excluded); NEW register entry owed for the (a) control (no entry exists today) |
docs/CURRENT-STATE.md section 6)| Gate | Status |
|---|---|
| G9/G10 | CLOSED/historical; their SUCCESSOR for the 10.13 rebuild is a SINGLE apply-and-verify gate (no outer/inner pair): substrate apply + (a) --check + depth-2 boot proof + direct-NAT egress test. The retired --host-nodes --check sub-item is replaced by the (a) control's check -- which is host-scoped, hence the Stage-1 home (Section 3) |
| G12 | CLOSED/historical -- read as "the shape being replaced," never a template |
| G17 | OPEN; the one dual-cause edit (D-143 address + container-elim placement) -- record as two line-items landing in one edit |
| G14 | Indirect: residency re-points + >=1 new SEC row are COUNT-affecting; flag for the next ledger-scan.sh reader (instrument-currency lesson #25) |
| G18, G1-G8, G11, G13, G15, G16 | No container-layer dependency (verified per-gate by W1.2) |
Consolidating Phase-0 7a (confirmed deliverable: --check gate + SEC-NNN row), W1.2 (no stage home, no register entry, risk of ad-hoc under-gated build), and W1.3 (must precede co-residency; E.1):
Requirement. A vcloud-level host isolation artifact (nftables, SEC-010's proven pattern one layer up) asserting no inter-plane / inter-DC forwarding on vcloud's own kernel, shipping with: a mechanical --check gate, its own tests/<name>/run-tests.sh harness, a new SEC-NNN ledger row, a NEW workflow-doc gap-register entry, and a gate row in the rebuild's G-series successor. Parameterized by site token (W1.4 principle 1), never DC-hardcoded.
Ordering invariant (fork-robust). The control must be installed and --check-verified BEFORE the first flat substrate apply that can make any two DCs' planes co-resident -- i.e. before ANY per-DC flat apply, since under a merged single root the first apply may create both DCs' planes at once (check 2). "Before the second DC's apply" (W1.3's phrasing) is the minimum; "before any flat apply" is the only ordering that survives the open root fork.
Stage placement recommendation: Stage 1 (vcloud host prep), as a host-level control, re-verified (i) at each per-DC substrate apply's close and (ii) at Stage-5 verify-live (W1.3 B.7 step 24 -- the first point real traffic tests the claim). Rationale: it is host-scoped, not per-DC-apply-scoped (W1.2's G10 analysis); Stage 1 gives it a named stage owner so it cannot be orphaned; and the placement is indifferent to the root-topology fork.
Artifact kind (resolves W1.2-vs-W1.4): SEC-010's actual pattern -- a script-installed nftables control + --check + harness, i.e. a procedure/L5 verify artifact, NOT an OpenTofu module. W1.2's sketch slot "[3]" expressed invocation ORDER (before any DC apply), not artifact kind. Phase 2 builds ONE artifact.
Distinct from the re-authored transit-leg FORWARD-drop (check 3) -- two controls, two SEC rows, both owed.
Two axes tagged throughout: [D-143] (value substitution) / [CE] (container-elim, shape) / [both] (dual-labeled per check 5).
already describes this shape correctly and needs no rewrite to tear it down)
[D-143] Back up state -- BOTH roots, BOTH hosts (outer on vcloud; each inner on voffice1).[D-143] MAAS machine census FIRST, from voffice1 (Step 2, both lenses).[both] R7 credential-revocation checklist (OWED BUILD -- Section 6 #5), run BEFORE any substrate destroy (revoking after the hosts are gone degrades to "assume it's moot"): SEC-026 client credential, SEC-028 service credential, SEC-029 rack-local PKI copy (shred; headend canonical unaffected), D-126 per-env qemu+ssh keys (clean RETIREMENT -- no successor exists), rack enroll-secret residue. Enumerate from EVERY vm-secret-locations row keyed to the rack host class, not just the named examples; mark rows RETIRED (append-only). Each revocation individually confirmed with captured output.[D-143] MAAS machine-record release/delete (OWED sequenced step -- Section 6 #6): guaranteed non-zero LENS-2 on a live checkpoint; release/delete per record (operator-gated), re-run both lenses to zero; ALSO clean the region-side residue -- the vvr1-dcN rack-controller's own enrollment record + the region's primary_rack/DHCP reference.[D-143] Plan destroy, INNER first (from voffice1), capturing the pre-destroy virsh baseline while the containment VM is still reachable.[D-143] Plan destroy, OUTER second (vcloud): vvr1_dcN + uplink + storage modules only. Do NOT target mesh/netem legs (working assumption: mesh triangle SURVIVES the pivot -- confirm at Phase 2, do not destroy speculatively).[D-143] Apply destroys, INNER then OUTER; verify each half from the host that can see it.[D-143] Untargeted tofu plan drift gate: zero unexpected drift.[D-143] NetBox DCIM decommission of the vvr1-dcN device records.[both] vcloud capacity re-measurement + FIT-calculator extension (the "~176 GiB freed" stays directional until then); [D-143] NetBox B2 apex re-carve (10.13.0.0/16, octet-preserving); [D-143] lib-net.sh 10.13 literal blocks; [both] D-124 transit re-point -- octet math is D-143, the bearer host changes rack -> client VM (CE).[unchanged] vcloud host prep; Office1 headend (if not already up). voffice1 remains MAAS-region + Plane-2 host for what still needs it.[CE] Install + verify the (a) isolation control -- BEFORE ANY flat substrate apply (Section 3's fork-robust invariant; supersedes W1.3's "before the second DC's apply" as the minimum reading).[CE] One flat apply CYCLE per DC (root shape per the OPEN fork, Section 7), vcloud-local qemu:///system: mesh legs + uplink NAT [unchanged]; per-DC storage pool collapses to one; six planes re-homed (same CIDRs/families/MTU -- D-139/D-143 own the values); edge re-homed, WAN -> direct NAT; 12 node VMs re-homed (MAC re-pinning is a likely force-replace -- re-measure every MAC after apply, before B.6 trusts one); [new] the vr1-dcN-client VM. VANISHES: vvr1_dcN modules + sizing/addressing/pubkey vars, the two inner roots AS ROOTS, the qemu+ssh dial + D-126 keys (no successor), wan-bridge + netplan bridge, the bootstrap gate's --host-nodes duty.site-headend-install.sh --role rack, WITHOUT --host-nodes) onto the ruled host; D-131 forwarder (dc-rack-net.sh) -- must land SOMEWHERE or the SERVFAIL bug returns; artifact service (.4); the re-authored transit-leg FORWARD-drop (control (ii), Section 3 -- distinct from B.3, ends still open).[unchanged*] MAAS discovery/commission/carve -- mechanism is per-machine power_type=virsh (check 7 wording correction); *only the power-address VALUE re-derives to vcloud's own libvirtd (lib-hosts.sh re-derivation + every invocation-site example).[unchanged] Juju bootstrap + bundle from the vr1-dcN-client VM ("never the vcloud jumphost" doctrine intact); SEC-026/028/029 freshly MINTED here (register rows re-point; no material migrates -- check 7 reconciliation); verify-live gates: Ceph-over-v6 (new literals per D-143), geneve-encap-assert.sh (MTU budget analytically unchanged -- live assert still OWED), plus the (a) control's --check re-run now that both DCs' planes are actually co-resident.[CE] NetBox DCIM registration of the client VM + flat roster; [deferred] the container-elim [ARCH] ruling (Phase 4 frames, operator rules) -- the redeploy is NOT "done" while that ruling is owed.L0 host/inter-site substrate (IaC: mesh, pools, office1-network -- Stage 1) -> L1 site/edge nodes (IaC: voffice1, DC edges, + the client VM as a new instance of the SAME cloudinit-vm module type) -> L2 DC substrate (IaC: planes + node VMs; the two-root split collapses INTO L2 -- the one place the layer-boundary rule was violated (inner root dialing a cross-host provider) and Option 1 removes that violation structurally) -> L3 enlist/commission (procedure; rack-remainder placement OPEN) -> L4 Juju/OpenStack deploy (procedure; D-140 PINS it as procedure for this redeploy -- settled, do not fold into IaC) -> L5 verify/gate (cross-cutting; the (a) control is a NEW L5 artifact per Section 3).
Design principles carried to Phase 4: site-token parameterization (never DC-hardcoded); every module ships its harness; idempotence at every layer; no layer reaches past the one below; findings logged at their true layer; D-140 is a distinct axis (its later adoption would make L4 IaC WITHOUT reshaping L0-L3); Roosevelt-transfer judged per layer (L0's node-VM shim does NOT transfer; L1's client-VM pattern + L3/L4 procedures ARE the pre-Roosevelt deliverable). D-143 is a PARAMETER change threaded through every layer, not a layer -- the model keeps the two axes structurally distinguishable.
The five core artifacts:
tofu group-destroy procedure (+ harness) re-earning D-122's one-command site-down for the flat shape (final form depends on the root fork; Phase 2 designs, Phase 4 ratifies).--check gate + harness + new SEC-NNN row + new gap-register entry + gate row (full spec Section 3).vm-secret-locations rows for the rack host class; gates Part A step 3.primary_rack/DHCP-reference cleanup.Additional owed items surfaced (kept separate so the core count stays legible):
virsh destroy loop over the DC's domain set, roster-derived from lib-hosts.sh (rollback-tree item-5 replacement; W1.1 Sec 1(b)) -- distinct from #1 (emergency vs gated path).vvr1-dcN records; register client VM + roster.geneve-encap-assert.sh) on the vcloud-level planes; gap-#20 site-baseleg verdict re-verification (its own expiry clause triggered).SETTLED (do not re-open): Option-1 target; handling (a) as a required deliverable; MAAS region stays on vr1-dcN-maas-01; Stage 2/D-114 out of scope; Paths M + C unchanged; D-140 pins Stage 5/L4 as a procedure module for this redeploy; mesh triangle persists as the working assumption (Phase-2 confirmation, not speculation-destroy); the two-axis attribution discipline (with the four named [both] items); the (a)-control ordering invariant (before ANY flat apply) and its recommended Stage-1 home; the teardown ORDER of the current checkpoint (Part A -- unchanged by container-elim).
OPEN -- carried to Phase 2 (design):
.4) placement (client VM vs maas-01 vs retire-with-evidence) -- blocks B.5, the phase3 SSH-target edits, and G17's host half. THE highest-leverage open item: three sequence steps and two runbook edits key off it.vvr1-dcN).site-headend-install.sh refactor scope (rack-role remainder vs dead --host-nodes).OPEN -- carried to Phase 4 (decision framing; operator rules, GA-R5):
OPEN -- operator inputs (SCOPE Section 8): pre-Roosevelt hardware specs (plug into B.1.1 capacity + B.4 sizing only -- and B.5's placement should be RE-DECIDED for bare metal, not carried blindly); any external "module deployment project" artifacts.
Author = "the administrator" (no model name asserted, operator instruction). Direct reads this session: docs/dc-dc-deployment-workflow.md:36-200 (Stage 1/2/3 identity), docs/ design-decisions.md:5344-5373 (D-128), runbooks/dc-dc-phase6-designate-cos-magnum.md: 430-450 (the "ALREADY WRONG" claim), plus the six pass documents in full. Worker citations were spot-checked, not re-derived wholesale; every correction in Section 1 names its source lines. Findings are LOGGED only; nothing here was executed.