Fork: 0
OpenStack / openstack-caracal-dc-dc
Transfer to URL with SHA Find file
Newer
Older
openstack-caracal-dc-dc / docs / audit / container-elim-pass / operator-report.md
@JANeumatrix JANeumatrix 5 hours ago 9 KB container-elim planning pass RAN (phases 0-4 + advisor): FINAL-PLAN + operator-report + D-144 framing (READ-ONLY)
Raw Blame History

Container-layer elimination -- OPERATOR REPORT

Date: 2026-08-09. Pass: multi-agent, READ-ONLY planning (phases 0-4 + advisor review). Nothing was executed against the cloud; every owed artifact is LOGGED, not built. Full plan: FINAL-PLAN.md. Decision framing: pass4-w4-decision-framing.md. Advisor review: FINAL-advisor-review.md. All 24 pass documents are in this folder.

1. What you confirmed, and what's still yours to rule

At the Phase-0 gate you confirmed (directional -- NOT the formal ruling): Option 1 (flat node VMs on vcloud libvirt + one small per-DC vr1-dcN-client VM for the D-138 client role), and handling (a) for the cross-DC gap (a new vcloud host-level isolation control). The pass then planned the whole change against that target. The formal [ARCH] ruling is still owed -- it is the single most important decision below.

2. The one decision everything hangs off: DEC-01

Mint a new D-144 superseding D-123 (Model B), OR record it as a D-123 amendment. Recommendation: new D-144. The GA-R3 test fires on all three prongs (architectural consequence beyond the stage, the operator-stated Roosevelt-delta, direct supersession of D-123's core ruling), and the on-point precedent is D-143 itself -- it AMENDED D-115 and TERMINATED a D-101 clause yet was minted as its own number rather than editing either. A third D-123 amendment reversing D-123's own central ruling would amend the entry out of existence; append-only discipline prefers a fresh entry naming D-123 SUPERSEDED. Per SCOPE, the redeploy is not "done" while DEC-01 is owed -- and W4.2 names the ruling a PRECONDITION for the first flat Stage-3 apply.

D-144's body should carry, explicitly: the D-123 supersession; the D-125 termination + the D-124 sizing re-cause; and -- important -- the accepted power-key blast-radius tradeoff named in the body, not buried in a SEC row (below).

3. The Tier-1 package to rule WITH DEC-01 (before Part B's Stage 3 runs)

  • DEC-02 / DEC-13 / DEC-09 -- three amendments filed alongside: D-128 (Plane-2 shrinks -- the substrate build becomes wholly Plane 1), D-134 (adds .8 = the client-VM octet), D-131 (retire-with-evidence -- gated on a fresh live re-measure first). D-127's autostart amendment is DEFERRED until DEC-22 rules the value.
  • DEC-15 -- the power-key mitigation mechanism (restricted SSH key / per-DC virsh wrapper / polkit ACL) + its SEC row. The sharpest execution blocker: six test/tool edits are frozen until it rules, and the interim RED on dc-selector/maas-region-power-key once lib-hosts changes is the wanted fail-loud state -- do not green it early. The advisor sharpened this: the mechanism must also decide REACH (how an in-plane maas-01 reaches vcloud's own libvirtd without a host address on a plane bridge -- which the (a) control forbids). Reach + authorization ruled together.
  • DEC-14 / DEC-16 -- the (a) control's concrete nftables mechanism + SEC row; the SEC-010-successor disposition (new row vs SEC-010 amendment) + endpoints (client VM + voffice1, recommended).
  • DEC-11 / DEC-12 / DEC-23 -- ratify root topology (B) (shared-outer + per-DC-flat roots; recommended -- but note it's tofu state isolation only, it mitigates NONE of the three concerns), root naming, and the state-blast-radius weighing. Unblocks the largest cluster of rows.
  • DEC-08 -- rack-controller retirement (recommended; gated on the same fresh live re-measure).
  • DEC-24 (NEW, from the advisor review) -- the dc0<->dc1 mesh-leg consumer + cross-DC Ceph replication path. virbr5 (the netem'd leg) is the D-100/D-108 replication carrier; the pass reassigned the Office1-transit legs to the client VM cleanly but left this leg's post-flatten attachment undefined. Partly pre-existing (dc1 HELD -> never built), so the flat design must DEFINE it. Pairs with DEC-14; owed a design + gate, not a package blocker.

Tier-2 delivery-grade rulings (needed before their specific rows, not the package): artifact-service sizing (DEC-10, with the FIT numbers), D-127 autostart value (DEC-22), NetBox migration (DEC-21), the A11 gate home (DEC-20), wan-bridge dir delete-vs-leave (DEC-17), and two pre-existing items adjacent to the pass (DEC-18 maas-vm-host, DEC-19 maas-fabric-prune harness gap).

4. What the flattening buys -- and what it costs, stated honestly

Buys: two tofu roots + a bash bootstrap gate + a cross-host qemu+ssh provider dial collapse to one apply per DC, one state axis, no gate; nesting depth 4 -> 2 (the VR0-proven shape). The deepest structural win (found independently by two workers): the container layer is the one violation of the repo's IaC->procedure boundary rule -- the inner root's qemu+ssh provider is IaC reaching across a live-dial boundary -- and Option 1 removes it structurally. Module impact is small: 6 IaC modules re-home with zero body changes, wan-bridge collapses, one new dc-site module composes the per-DC stack.

Costs (each carried, none papered over):

  • D-122's one-command site-down (virsh destroy vvr1-dcN) is lost -- re-earned via a root-scoped teardown primitive + an emergency virsh-loop lever (both owed, don't exist yet).
  • Three NEW isolation exposures, each with its own owed control:
    1. cross-DC plane-bridge co-residency -- both DCs' bridges on vcloud's one kernel -> the (a) host-level control (Stage-1 gate + cloud-assert A11a; must REFUSE on partial resolution);
    2. the SEC-010 transit-drop successor on the new endpoints (client VM + voffice1; preflight P10);
    3. the MAAS power-key blast radius -- and this is the pass's most consequential finding: re-deriving node power to vcloud's own libvirtd would give each DC's region key virsh control over BOTH DCs' fleets + voffice1 + vcloud (verified: voffice1 is a vcloud-libvirtd domain, main.tf:175; no libvirt access-scoping exists in-repo). Flattening silently makes SEC-012/016's per-DC key separation vacuous unless the DEC-15 mitigation ships. Its gate is a negative test (the key CANNOT reach the other DC), not existence-only.

5. The layered module workflow (the pass's headline deliverable)

The redeploy becomes an L0-L5 layered module system -- IaC modules (OpenTofu) + procedure modules (runbooks/scripts) -- built from the EXISTING 12 modules + 8 $DC-parameterized stage runbooks, not invented. Composition = a (once) prefix (mesh/pools + install-and-verify the (a) control) then a per-$SITE loop (dc-site apply -> installs -> MAAS enlist -> juju/bundle -> verify). Roosevelt transfer (what the operator wants tested): the L1 client-VM + L3 MAAS + L4 juju/bundle procedures transfer to bare metal unchanged -- that is the "module deployment project" payload; L0/L2's virtualization IaC and the two co-residency-specific controls (the (a) control, the power-key mitigation) are the parts that do NOT carry over (bare metal uses separate hosts + IPMI/Redfish). The pre-Roosevelt bare-metal hardware specs plug in at named points (FIT/roster/MAC/sizing) -- mapped, so when the specs land the plug-in points are known.

6. Scale, owed work, and the two-axis discipline

  • Master change-set: 104 rows (23 decisions, 13 tofu-module, 3 lib, 19 script, 6 doc, 5 runbook, 10 gate, 21 harness, 4 SEC). Full table: pass4-w1-master-change-inventory.md.
  • 13 owed artifacts (all logged, none built), each shipping its own failable harness. 7 new-build harnesses / 9 change / 5 retire (10 of 103 existing harnesses touched; ~90 grep-confirmed clean).
  • 9 owed live measurements -- 2 needed BEFORE ratifying the recommendation-grade items (current-day primary_rack/rackd state both DCs; vcloud's live polkit/libvirt config), 7 at build time (fresh capacity + exact FIT -- until then "~176 GiB freed" is DIRECTIONAL only; geneve/jumbo live assert; MAC re-measure; client-VM transit NIC name; D-131 dig-evidence; gap-#20 re-verify).
  • Two-axis separation holds end to end: every change is attributable to [D-143] (the 10.13 value substitution, already ruled) or [CE] (this shape change); four dual-cause items carry [both]. lib-net.sh carries ZERO container-elim edits (D-143 axis only, grep-verified).

7. Sequencing note (do not miss)

The teardown (Part A) presumes the dc0 checkpoint wrap-gates close first -- cloud-assert BOM, controller backup, verify-live Ceph -- which are still owed per the session ledger. The redeploy (Part B) order honors three hard invariants: the (a) control before ANY flat apply; the power-key mitigation before the lib-hosts power-address re-derivation; R7 revocation + MAAS record-release before any substrate destroy. Full ordered step tables: pass4-w3-execution-sequencing.md.

Rule the Tier-1 package (DEC-01 D-144 + its amendments + DEC-15/14/16/11/08/24) in GA-R5 exchanges -- ONE decision per exchange, each with its exact utterance. That ratification is what turns this plan into buildable, gated work and unblocks the redeploy design. Everything below the package is delivery-grade and can follow. The pre-Roosevelt hardware specs, when they land, slot into the mapped plug-in points without reopening the topology.