Newer
Older
openstack-caracal-dc-dc / docs / audit / container-elim-pass / pass4-w4-decision-framing.md

Pass 4 -- Worker W4.4: the owed [ARCH] decision framing (container-elim + layered module workflow)

Author: W4.4 (Phase 4, SCOPE-AND-EXECUTION-PLAN.md Section 4). Date: 2026-08-09. Inputs read in full: SCOPE-AND-EXECUTION-PLAN.md, pass0-admin-report.md, pass1-admin-report.md, pass2-admin-report.md, pass3-admin-report.md; docs/design-decisions.md entries D-122, D-123 (+ both amendments), D-124 (+ 3 amendments), D-125, D-126, D-127, D-128, D-131, D-132 (+ 2026-07-30 amendment + addendum), D-134 (+ 5 amendments), D-138, D-143; the GA-R3 admission test (SKILL.md:406-410). READ-ONLY. This document FRAMES a decision package for the operator (GA-R5); it rules nothing. Every recommendation below is graded "recommend," never "ruled."


0. What this pass confirmed is already SETTLED and is not re-opened here

Per pass0 Section 7a (operator-confirmed at the Phase-0 gate, a DIRECTIONAL PLANNING CONFIRMATION -- explicitly not a GA-R5 [ARCH] ruling): target topology = Option 1 (flat node VMs on vcloud libvirt + one small non-hypervisor vr1-dcN-client VM per DC); cross-DC adjacency handling = (a) (accept co-residency + a new vcloud-level host isolation control); MAAS region stays on vr1-dcN-maas-01. Phases 1-3 planned, tooled, and tested against this. The formal [ARCH] ruling this document frames is the one thing the Phase-0 gate explicitly deferred (SCOPE Section 7: "the container-elim itself is an OWED [ARCH] decision... Phase 4 FRAMES it... it is ruled by the OPERATOR").


1. THE CORE QUESTION: D-123 amendment, or a new D-number?

1.1 Apply the GA-R3 admission test verbatim (SKILL.md:406-410)

"a new D-number needs architectural consequence beyond the stage, a Roosevelt-delta (the A1 test: a Roosevelt build session would grep it before touching a built surface), or supersession. Everything else is OPERATIONAL... doubt resolves DOWN to OPS."

All three admission triggers fire, independently:

  • Architectural consequence beyond the stage. Confirmed across every phase: the change restructures Stage 3 wholesale (pass1 Section 2.2), amends D-128's own definition (pass1 check 6), voids/re-causes a D-124 clause (pass0 row 6, Section 6 item 8), terminates D-125 (Section 3 below), reopens D-131's standing-pattern status (pass2 4.2(ii)), touches D-132's addendum premise (pass2 4.2(i)), forces a D-134 map addition (pass2 4.3), and changes D-138's concrete host (pass0 check 3). No single existing entry's scope contains all of that.
  • Roosevelt-delta (A1 test). Explicit and operator-stated: SCOPE Section 1 quotes the operator directly -- the redeploy is "a good test of the module deployment project we are developing during the teardown and redeploy," and SCOPE Section 7 names the layered module workflow a "Roosevelt deliverable... design it to transfer to the pre-Roosevelt bare-metal test." A future Roosevelt-adjacent session would grep this decision before laying out any future site's substrate shape or module structure -- textbook A1.
  • Supersession. D-123's core adopted ruling -- Model B, 4-level nesting, node VMs live INSIDE vvr1-dcN, single-virsh-destroy site-down (design-decisions.md:4929-4939) -- is directly reversed. Option 1 does the opposite of what D-123 rules: nodes return to being flat vcloud-level siblings (D-123's own rejected "Model A" shape, now readopted with 2026-07-30+ rulings layered on).

1.2 The load-bearing precedent already in this ledger: D-143

D-143 is the closest analog and it is a directly on-point precedent, not an analogy: D-143 "AMENDS D-115 premise" and "TERMINATES D-101... clause" (design-decisions.md:8083) -- exactly the shape of relationship the container-elim has with D-123/D-124/D-125/D-128 -- and it was minted as its OWN new D-number, not appended as a D-115 or D-101 amendment. D-143's Reconciliation section (:8124-8147) is the template this document's Section 3 follows: a new entry states its verb (SUPERSEDES / AMENDS / TERMINATES / CONSISTENT-WITH / PRESERVES) against each affected decision, rather than the affected decision being edited to absorb the reversal.

1.3 Why "D-123 amendment" under-fits

D-123 already carries two prior AMENDMENT entries: the 2026-07-16 ruling that ADOPTED Model B over the recommended Model A, and the 2026-07-20 correction that Model B's virsh-pod MECHANISM was refuted (per-machine power replaces it) while Model B's SHAPE stood. A third amendment that reverses D-123's own central ruling -- the shape itself, the thing the decision exists to answer -- would have D-123 amend itself out of existence: a future reader grepping "D-123" for "what shape is a DC site" would need to read three superseding layers to reach a NO that contradicts the entry's own header. GA-R1's append-only discipline is better served by leaving D-123's history intact (Model A recommended -> Model B ruled -> mechanism refuted) and recording the reversal as a fresh, separately-dated decision that names D-123 as SUPERSEDED -- exactly D-143's pattern against D-101/D-115.

1.4 RECOMMENDATION

Mint a new D-number (next-free = D-144, per ledger-scan.sh run this session inside pass3 -- pass3-admin-report.md Section 1 item 6: "D next-free=144" -- re-verify at mint time per repo numbering discipline, do not assign here). Title shape (for the operator's edit, not a pre-write): "D-144: VR1 container-layer elimination -- flat per-DC substrate + per-DC client VM, layered module workflow (SUPERSEDES D-123 Model B)". This is a recommendation, presented alongside the D-123-amendment alternative below for the operator to weigh; GA-R5 forbids the pass from picking.

Alternative presented for completeness (not recommended): amend D-123 in place. Would keep one canonical "DC site shape" entry instead of a reader needing D-123 -> D-144 to reach current truth. Weighed against: it fights the D-143 precedent, and D-123's entry would need heavy internal surgery (its own "Model A" section would need to become "Model A, re-adopted under D-144" -- essentially rewriting the entry to point at a new one anyway, which is the same reader cost with none of the whole-history discoverability of a fresh entry).


2. RIDE-ALONG DECISIONS -- each framed ([ARCH]/[OPS], reconciliation, Roosevelt-delta)

Seven items, per the assignment; doubt resolves DOWN to OPS (GA-R3).

(1) D-128 amendment -- the Plane-1/Plane-2 execution-host split

[ARCH]. D-128 is itself tagged [OPS] in the ledger, but this specific edit meets the GA-R3 bar on its own: it changes a RULED decision's own DEFINITION (pass1 check 6, verified direct read design-decisions.md:5354-5364) -- Plane 2 currently includes "the INNER tofu root (opentofu/vr1-dc0-substrate/, qemu+ssh FROM Office1 into vvr1-dc0)"; under Option 1 that object ceases to exist, so the substrate build becomes wholly Plane 1 (vcloud-local) and Plane 2 shrinks to MAAS/NetBox (juju/openstack already moved off Plane 2 by D-138). Reconciliation: AMENDS D-128 -- narrow, single-entry scope change, not a reopening of the two-plane MODEL itself (Claude-stays-on-jumphost, workstation-is-human-path all stand unchanged). Roosevelt-delta: indirect -- D-128 states explicitly it has "No Roosevelt analog" for Plane 1 (throwaway simulation scaffolding); the amendment doesn't change that, it only shrinks what Plane 2 covers. Recommendation: own dated "D-128 -- AMENDMENT" entry, riding the same operator exchange as the core ruling (it is a direct, mechanical consequence of Option 1, not an independent question) but recorded against D-128, not folded into D-144's body.

(2) D-125 bridge-in retirement -- within the core ruling, or its own note?

[ARCH] (D-125 is itself [ARCH]) but with no independent architectural surface once separated from the core question. D-125 exists SOLELY to solve OBS-3 -- the egress-dead-end created by Model B's WAN NAT living inside a nested containment VM with no direct route out (design-decisions.md:5091-5096). Remove the nesting and OBS-3's precondition disappears; there is nothing left for D-125 to fix. This is not a parallel decision riding alongside the core ruling -- it is a direct, total consequence OF it (pass2 Section 3.1: wan-bridge "collapses," edge WAN reattaches to the vcloud-level NAT directly -- literally D-122's ORIGINAL pre-D-125 intent, restored). Roosevelt-delta: none of its own -- D-125's bridge-in mechanism was always VR1-only rehearsal scaffolding for a nesting pattern that itself has no Roosevelt analog. Recommendation: do NOT mint or independently amend D-125 as a ride-along item; record its TERMINATION as a named consequence inside the core new entry's reconciliation ledger (Section 3 below), the same way D-143 named D-101's clause TERMINATED inside D-143's own body rather than as a separate D-101 amendment entry.

(3) The D-134 .8 octet addition for vr1-dcN-client

[ARCH]-adjacent by D-134's own established pattern, but narrow. D-134 is tagged [ARCH] and has FIVE prior dated amendments, three of which are single-slot octet-map extensions with the identical shape to this one (.5 juju 2026-07-29, .6 MAAS region via the D-132 addendum, .7 tailscale 2026-08-07 -- design-decisions.md:5947-6007). Each was minted as its own dated "D-134 -- AMENDMENT" entry, never folded into the decision that motivated the new host class (D-104 for juju, D-132 for MAAS, D-129(iii) for tailscale). The client VM is .8, next free slot (pass2 Section 4.3, W2.2 verified against the standing table which enumerates through .7). Reconciliation: AMENDS D-134 (adds one row; does not touch the CIDR/band structure itself). Roosevelt-delta: real and explicit -- D-134's own amendment text states "A future DC standup carves .N for its [service] by this standard" (:6006); a Roosevelt build session greps D-134 for the utility-band map before assigning any new per-DC service an address, exactly the A1 test. Recommendation: own dated "D-134 -- AMENDMENT" entry, following the established pattern exactly (motivated by/cited from the core ruling, minted separately) -- consistent, not a new D-number, not folded into D-144's body.

(4) Root-topology (B) + root naming

[OPS]. Shared-outer + per-DC-flat tofu roots vs a single merged root is a tofu-state organization choice: both alternatives deploy the identical physical/network topology (pass2 check 3, verified: "root-shape (B) mitigates NONE of" the three isolation concerns -- it is about state blast radius, destroy scoping, and apply ordering, not what gets built). It does not change what exists on the wire or in the hypervisor; it changes how the tofu STATE that describes it is partitioned. Root NAMING (-flat vs reserving -substrate) is purely a repo convention. Reconciliation: none against an existing D-number -- no prior decision rules tofu root topology at this granularity. Roosevelt-delta: weak/indirect -- Roosevelt has no "tofu roots" concept for physical hardware it doesn't create; the delta that DOES transfer (module composition, dc-site as a reusable unit) is captured by the module-workflow design itself, not by the root split. Recommendation: ratify as part of the Phase-4 module-workflow design / delivery change-set (an OPS-graded record: changelog entry + docs/dc-dc-deployment-workflow.md Stage-3 Build-line text), not a D-number.

(5) The three isolation controls' SEC rows -- power-key mitigation especially

Three distinct controls (pass2 Section 2, verified this session's cross-reads); each graded separately:

  • (i) cross-DC plane-bridge network adjacency (the "(a)" control) and (ii) the SEC-010 transit-leg successor: [OPS]. Both are new/re-authored SECURITY MITIGATIONS -- nftables artifacts with --check gates and SEC-NNN ledger rows -- of the same kind SEC-010 itself was, and SEC-010 was never a D-number (D-125 only cross-references it). GA-R3's OPS bucket is precisely "runbook edit / session-changelog line / as-built row" for this class of work; a SEC row plus a harness is the established mechanical pattern (pass3 confirms next-free is SEC-034, computed by direct ledger grep since ledger-scan.sh does not compute SEC numbers). Recommendation: SEC-ledger rows, not D-numbers; the REQUIREMENT that they exist and their ordering invariant (installed before any flat apply) is recorded as an owed artifact inside the core new entry's execution notes, mirroring how D-125 recorded SEC-010's constraint without itself being a SEC entry.

  • (iii) the MAAS power-key blast radius -- the hardest of the three, argued in detail. This is the item the prompt specifically asks whether it "also warrants a D-number given it makes SEC-012/016 per-DC separation vacuous" (pass2 Section 2.3, verified this session's citations: maas-region-power-key.sh installs a per-DC key SEC-012/SEC-016 assume stays scoped; maas-node-power.sh:28-30 confirms the REGION dials power; opentofu/main.tf:175 confirms voffice1 is a sibling domain on the SAME outer libvirtd that will also hold every flattened DC node -- so under Option 1, each DC's region-resident power key opens a qemu:///system connection with virsh control over EVERY domain vcloud manages, not just its own DC). Grading reasoning: the mechanism CHOICE (restricted key / wrapper / polkit ACL) is squarely OPS -- a SEC-NNN mitigation like (i)/(ii). But the FINDING itself -- that flattening (the core ruling's own structural consequence) silently VOIDS a previously-relied-upon per-DC credential-isolation guarantee that SEC-012/SEC-016 were written to provide -- is architectural framing content: it is a tradeoff the operator is accepting BY ruling the core question, not an independent decision with its own Roosevelt-delta or supersession target. It has no existence apart from the core ruling (unlike D-125, above, it doesn't supersede or terminate any EXISTING decision by name -- SEC-012/016 aren't D-numbers to supersede -- it just makes them functionally moot). Net classification: [ARCH]-adjacent finding, [OPS] mitigation. Recommendation: do NOT mint a separate D-number for this. Record the finding and the accepted tradeoff explicitly INSIDE the core new entry's body (so a future reader sees "flattening was known to weaken per-DC power-key scoping, mitigated by SEC-034[+1]" in the same place they see the topology ruling) -- this is the one item where recommending "fold into the core entry's text, not its own line-item" matters most, because burying it only in a SEC row (which nobody greps before touching architecture) would repeat exactly the failure class GA-R3's A1 test exists to prevent.

(6) The rack-controller retirement + D-131 retire-with-evidence

Split, two different triggers, two different answers -- do not conflate them.

  • Rack-controller retirement itself: [OPS]. Both DCs' region+rack MAAS controllers already measurably run all rackd duty (pass2 check 2, direct reads: changelog-20260807-dc1-region- sequence.md:80-89 dc1 transcript-grade; changelog-20260730-dc0-region-migration.md:~532-549 dc0 process-measured, pgrep -c dhcpd=0 on the rack). Retiring vvr1-dcN's vestigial Office1 rack registration is a decommission step (a maas rack-controller delete + runbook note) -- it does not change any RULED architecture, it cleans up a registration nobody is using. This is the MAAS machine-record release/delete class of work already scoped as owed artifact #5.
  • D-131 retire-with-evidence: [ARCH]-touching, but AMENDS D-131, does not need a new D and is NOT itself created by the container-elim. D-131 sub-decision 1 explicitly RULED the forwarder as "the STANDING per-DC pattern... applied to dc0 and part of every future DC standup's definition-of-done" (design-decisions.md:5658-5661) -- retiring that changes an ARCH decision's forward applicability, which is a real edit, not a nit. But the TRIGGER is D-132's already-ruled per-DC-region architecture (2026-07-30) removing D-131's own stated precondition ("rack-only controller, remote region") -- container-elim did not create this fact, it merely removes the vestigial host that made the asymmetry easy to miss (dc0 already proves the end state; dc1's forwarder is still load-bearing today, pass2 check 4 -- a real, unequal-grade asymmetry, not assumed equal). Reconciliation: AMENDS D-131 sub-decision 1 (own dated entry, D-131's existing amendment-free structure notwithstanding -- this would be its first). Roosevelt- delta: real -- D-131 sub-decision 4 is explicitly "PINNED... to be executed at next-deployment design time," i.e. this retirement IS that next-deployment design point arriving early. Recommendation: own dated "D-131 -- AMENDMENT" entry, gated on the owed live re-measure (current-day primary_rack state, both DCs) landing BEFORE the ruling is asked for, not a D-144 ride-along and not folded into D-144's body (its trigger is independent of container-elim).

(7) Artifact-service re-homing/sizing

[OPS]. A capacity/placement decision resolved by measurement (the FIT-calculator extension, owed artifact #7/#13 amendment) against two concrete VM sizings that are both too small for dc0's full mirror as authored (pass2 Section 3.3, verified sizing cites: vr1-dc0-substrate/main.tf: 176-181 maas-01's 150 GiB earmarked "no spare"; client VM ~80 GiB). No architectural principle is at stake -- it is "which host gets a bigger disk," decided with numbers, not a ruling about shape. Reconciliation: none against an existing D-number. Roosevelt-delta: none identified -- mirror/cache sizing is a per-deployment capacity fact (D-135 already owns the dc0-full-mirror / dc1-cache-proxy split as an ARCH decision; this ride-along is purely which HOST realizes it here). Recommendation: resolved via the FIT-calculator numbers at delivery, recorded as a changelog entry, no D-number.

Ride-along count / [ARCH]-[OPS] split

7 ride-along items. Classification: 2 carry [ARCH] weight (item 1, D-128 amendment; item 3, D-134 amendment) that get their OWN dated amendment entries against existing D-numbers; 1 is [ARCH]-in-substance but has no independent existence apart from the core ruling (item 2, D-125 -- folds into D-144's reconciliation ledger, not its own entry) plus a second such item (item 5-iii, the power-key finding -- folds into D-144's BODY as an accepted tradeoff, its mitigation MECHANISM graded OPS/SEC-row); 1 splits into an [OPS] cleanup + a separate [ARCH] amendment (item 6: rack retirement OPS, D-131 amendment ARCH but independently triggered); 2 are cleanly [OPS] (item 4 root-topology, item 7 artifact-service sizing); item 5-i/5-ii (the two network isolation controls) are OPS/SEC-row work referenced from D-144's text. Net: 0 new D-numbers beyond the core D-144; 3 existing-decision amendments (D-128, D-134, D-131) filed separately; 2 findings folded into D-144's own body; the remainder is SEC-ledger/changelog/runbook work.


3. THE RECONCILIATION LEDGER -- what the container-elim (D-144, proposed) does to each existing decision

Verb vocabulary matches D-143's own precedent (design-decisions.md:8124-8147): SUPERSEDES (replaces the cited ruling outright), AMENDS (a factual premise or scoped clause changes, the rest of the ruling holds), TERMINATES (a clause/mechanism is retired with no successor), PRESERVES (the ruling's principle stands unchanged; at most its concrete realization updates), CONSISTENT-WITH (untouched, cited for completeness).

D-number Verb What changes / what does not
D-122 (site shape) AMENDS Intent preserved (each site still has a distinct containment/entry-point identity -- now the client VM); the "single virsh destroy <site-vm> = site-down" LITERAL claim regresses to a root/module-scoped group-destroy (pass0 row 9) -- a real capability loss D-144 must state honestly, not silently drop. Dark-fiber/per-site-ISP/edge-shape clauses: untouched.
D-123 (Model B containment) SUPERSEDED The core ruling -- nodes nested inside vvr1-dcN, depth-4, single-VM destroy -- is reversed. D-123's own history (Model A recommended -> Model B ruled 2026-07-16 -> mechanism refuted 2026-07-20) stays intact, append-only; D-144 records the supersession rather than editing D-123.
D-124 (region<->rack transit addressing) AMENDS (a clause reverses again) The Scheme-A transit addressing (172.31.0.0/24, region/rack /30s) SURVIVES -- the client VM keeps the transit leg. The 2026-07-16 "sizing VOID under Model B" amendment (rack must hold a full node fleet, ~416/480 GiB) itself gets RE-CAUSED: under Option 1 the client VM is NOT a node-fleet host, so sizing returns toward something near D-124's small original proposal (4 vCPU/8192 MiB/80 GiB) -- exact figure OWED, not re-derived here (pass0 Section 6 item 8, "D-124 sizing-void re-cause").
D-125 (bridge-in egress) TERMINATES Its sole reason to exist (OBS-3, nested-WAN-NAT-with-no-egress) disappears with the nesting. modules/wan-bridge retires; edge WAN reattaches directly to the vcloud-level per-DC uplink NAT -- D-122's ORIGINAL pre-D-125 realization, restored (see item (2) above; folded into D-144's body, not filed separately).
D-125's wan-bridge fallback note (double-NAT) moot alongside D-125 Never adopted; moot with the primary mechanism gone.
D-126 (rootless SSH access convention) PRESERVES The general decision (Option A, systemd --user local-forward, per-env key convention) is a reusable ACCESS PATTERN used by multiple site-service VMs, not solely the qemu+ssh dial into vvr1-dcN. Only ONE consumer of the pattern retires: the qemu+ssh dial's per-env key, which has "no successor" (pass1 Section 4, Part A step 3) because its object (the inner root's cross-host provider) is gone. D-126 itself is unchanged; note the retired consumer as a changelog line, not a D-126 edit.
D-127 (VM autostart policy) AMENDS The explicit classification row "vvr1-dc0 (and future vr1-dc1 containment) -> MANUAL" (:5307-5311, itself cross-referencing D-123) has no successor object. The client VM needs its OWN classification against D-127's stated rule of thumb (foundational/no-fragile-boot = autostart; deliberate/gated/resource-heavy = manual) -- pass3 confirms this is an OWED ruling, not inferable (A1's new test case cannot be written until it lands). Recommend a short D-127 amendment adding the client-VM row once ruled; the POLICY framework itself is unchanged.
D-128 (two-plane operating model) AMENDS Real, substantive (item (1) above): Plane 2's own definition shrinks (loses the inner-root qemu+ssh object); the two-plane MODEL and Claude's-jumphost-residency are unaffected.
D-131 (node-facing DNS strategy) AMENDS (independently triggered, rides alongside) Sub-decision 1's "standing per-DC pattern" status is reopened by D-132's already-ruled per-DC regions removing D-131's own stated precondition; container-elim only removes the vestigial host, it doesn't create the trigger. Filed as its own D-131 amendment (item (6) above), gated on the owed live re-measure.
D-132 addendum (region in own VM at .6) PRESERVES; premise-note only No change -- MAAS region stays on vr1-dcN-maas-01 (operator-confirmed at the Phase-0 gate). Flag only: the addendum's original "hypervisor-fate" rationale (why the region got its own VM rather than co-locating) becomes MOOT under Option 1 (nothing is co-locating with a hypervisor anymore) -- a premise-currency note inside D-144's text, not a reversal (pass2 4.2(i)).
D-134 (octet bands / standing cross-DC map) AMENDS (new row, existing pattern) Existing bands/CIDRs unchanged; adds .8 = client VM, following the identical amendment shape as .5/.6/.7 (item (3) above). Filed separately, per D-134's own established convention.
D-138 (client lives IN the DC) PRESERVES; concrete host updates The PRINCIPLE -- cloud-facing tools (Juju, openstack CLI) dial the cloud from inside the DC, not from voffice1 -- is UNCHANGED and is exactly what Option 1's client VM realizes. Only the CONCRETE HOST changes: from vvr1-dcN (a DC-node-fleet-containing hypervisor) to the new small non-hypervisor vr1-dcN-client. Credential-residency consequences (SEC-026/028/029 migration, pass0 row 8) are a direct, named consequence to record in D-144's reconciliation, not a D-138 rewrite.

Not superseded/amended by this change (verified, named to prevent scope creep): D-114 (Office1's OWN containment VM, voffice1 -- a structurally distinct, KEPT pattern; pass1 check 1 is explicit that Stage 2 is untouched and the two "containment VM" patterns must be named as DIFFERENT going forward so name-similarity doesn't sweep D-114 in); D-133 (flat per-NIC plane carve); D-139 (IPv6-only east-west); D-140 (Juju-as-tofu, pinned for a LATER redeploy, explicitly not this one); D-143 (the re-IP this pass rides -- additive, kept distinguishable per axis throughout Phases 1-3, [D-143]/[CE]/[both] tagging).


4. Summary for the operator (what D-144, if ruled, would need to say)

If the operator adopts the "new D-number" recommendation, D-144's body would need, at minimum: (a) the core supersession of D-123's Model B; (b) the D-125 termination and D-124 sizing re-cause, stated as direct consequences, not separate rulings; (c) the accepted power-key blast-radius tradeoff (Section 2 item 5-iii) named explicitly, with its SEC-row mitigation as the mitigating control, not the whole answer; (d) pointers to the three amendments filed alongside it (D-128, D-134, D-131) so a reader lands on the complete picture from one grep. This document does not draft that text -- GA-R5 reserves the ruling, and the ruling's exact wording, to the operator.