# Changelog -- 2026-07-23 -- Stage 4 open: discovery + D-133 carve ruling

Session scope: open Stage 4 (Phase 3, MAAS enlist/commission/deploy per DC) on
branch `dc-dc-stage4-phase3-maas-deploy` off post-merge `main`. Bootstrap ran
clean (pull up-to-date at bff71d4; repo-lint 0 fail / 1 legacy warn; ledger-scan
reconciled -- 12 open SEC rows, D next-free 133). No cloud mutation this session
yet; everything below is read-only measurement + repo records.

## Item 1 -- Stage 4 OPENED: precondition + discovery (read-only), captured

- Branch `dc-dc-stage4-phase3-maas-deploy` created off `main` (bff71d4).
- Runbook Step 0 precondition PASS: both DC rack controllers enrolled
  (vvr1-dc0 `7chphy`, vvr1-dc1 `nmpcq4`).
- Step 2 discovery complete for BOTH DCs, all read-only, captured to
  `docs/audit/stage4-discovery-20260723.txt`: 9 nodes per DC, ALL Ready,
  power=virsh; dc0 = boot fabric-4 / MACs 52:54:00:*, dc1 = fabric-142 /
  pinned 52:54:01:d1:*; zone/pool `default` everywhere (no zone/pool
  convention exists -- fabric + MAC prefix is the grouping signal).
  Also captured: one dc0 node's full physical NIC list (6 flat per-plane
  NICs), `default_distro_series=noble`, and the synced boot-resources list
  (noble ONLY -- no jammy images on the region).
- Runbook-vs-reality deltas recorded in CURRENT-STATE (the runbook predates
  Stage 3's execution): Step 3 commissioning already satisfied; carve comes
  BEFORE deploy per the runbook's own Ready-state sequencing note; gap #6
  (pod-discovery landing status) is moot -- the pod path was refuted 2026-07-20.
- **Revert:** `git branch -D dc-dc-stage4-phase3-maas-deploy` (after moving off
  it); delete `docs/audit/stage4-discovery-20260723.txt`; revert the
  CURRENT-STATE Stage-4 bullet. No live state was touched.

## Item 2 -- D-133 ADOPTED: flat per-NIC carve (VR1); hardware-faithful NIC topology next deployment

- New entry `docs/design-decisions.md` D-133 [ARCH], Status ADOPTED 2026-07-23,
  operator utterance quoted verbatim per GA-R5 ("Fine, keep it flat but on the
  next deployment we need to build the VMs with exact hardware duplicated
  networking so we can work out any bonding/trunking kinks.").
- Part 1: VR1 DC nodes carve every plane FLAT on its dedicated NIC; no
  VLAN-103 stack in VR1 (supersedes the D-052-era Pattern A shape for VR1 DC
  nodes only; VR0 untouched).
- Part 2 (pinned to next deployment): node VMs duplicate target hardware's
  exact NIC topology (bond members, trunks) -- composes with D-132.
- Affected-surfaces list recorded IN the entry (buildout-design VID-103 text,
  workflow Stage 4 row, phase3 runbook Step 5 tree, lib-net vr1-dc0 arm's
  inherited METAL_INTERNAL_VID) -- queued as this stage's DOCFIX/delivery work,
  not silently edited.
- CURRENT-STATE section 1 updated in the same commit (GA-R1 C1): Stage 4 OPEN
  bullet with discovery facts + the open decision points (deploy OS series,
  octet plan, mirror gap #5, tag shape).
- **Revert:** remove the D-133 block from design-decisions.md and the Stage-4
  bullet from CURRENT-STATE.md (single commit; `git revert <sha>`).

## Item 3 -- D-134 ADOPTED: per-role octet bands for VR1 DC node statics

- New entry `docs/design-decisions.md` D-134 [ARCH], ADOPTED 2026-07-23.
  Sequence per GA-R5: the flat .40-.48 proposal was presented and REJECTED
  ("Different band"); operator directed "Per role bands with headroom. Give me
  a recommendation"; the recommended band table was presented with measured
  collision checks (bundle VIPs .50-.60 grep, 10.12.12.4, FIP pool, D-120
  dynamic range) and adopted ("Adopt as recommended").
- Bands: .1-.9 infra (.4-.9 utility-reserved), .10-.19 control, .20-.49
  compute, .50-.99 formal VIP band, .100-.200 dynamic (D-120), .201-.254
  storage. One octet per node across all six planes, both DCs; VR1
  assignments control .10-.12 / compute .20-.21 / storage .201-.204 keyed by
  tofu node name via pinned boot MAC.
- CURRENT-STATE Stage-4 bullet updated in the same commit (octet plan RULED;
  carve now unblocked on D-133 + D-134).
- **Revert:** remove the D-134 block + the CURRENT-STATE bullet edit
  (`git revert <sha>`).

## Item 4 -- D-134 AMENDED: contiguous band table (Option B swap)

- Operator rejected the original table's interleaving (node roles split by the
  VIP + dynamic blocks); movability analysis presented (pinned: .1/.2/.3 live,
  .50-.60 bundle VIPs, FIP rows 1-3 on provider; movable: the D-120-inherited
  dynamic range). Option B ruled: nodes get the contiguous .100-.200 run
  (control .100-.119 / compute .120-.149 / storage .150-.200), dynamic moves
  to .201-.254, utility .4-.49, VIP .50-.99.
- VR1 assignments now: control .100-.102, compute .120-.121, storage
  .150-.153 (identity table unchanged -- tofu name via pinned boot MAC).
- Consequence queued: the dc0/dc1 metal-admin dynamic-range MOVE is the
  carve's first gated live mutation per DC (before static assignment; nodes
  powered off). Office1's own D-120 ranges untouched.
- CURRENT-STATE Stage-4 bullet updated same-commit.
- **Revert:** remove the amendment block + CURRENT-STATE edit
  (`git revert <sha>`); no live state touched yet.

## Item 5 -- Approvals workflow: session-log-keyed permission rules (operator-directed)

- Operator directed session-length approvals mid-carve (manual mode was
  prompting per command; the earlier auto-mode classifier BLOCKED the first
  mutation outright -- the known dc1-session behavior, ledger 2026-07-23).
- `.claude/settings.local.json` (uncommitted, personal): ADDED allow
  `Bash(script -aqe ~/as-executed/2026-07-23-stage4-carve.log -c *)` (the
  logged-window wrapper -- inherently session-scoped, the rule names this
  window's log file) + ask rule keeping `maas admin machine *` shapes
  (commission/deploy) prompting inside that same wrapper. PRUNED seven dead
  rules keyed to retired log files (2026-07-21-ops-commissioning-diag,
  2026-07-22-dc1-deploy).
- FLAGGED for stage close, not acted on: local allow list carries a broad
  `Bash(ssh *)` from a prior session -- prune candidate.
- Review surface stays the as-executed log + this changelog (GA-R2
  blanket-approval precedent).
- **Revert:** remove the two added rules from settings.local.json (file is
  local-only, not committed).

## Item 6 -- Stage 4 carve EXECUTED, both DCs (logged window stage4-carve)

- All mutations in ~/as-executed/2026-07-23-stage4-carve.log (per-command
  wrap; helper scripts' full content logged verbatim before first use).
  Verification capture: `docs/audit/stage4-carve-verify-20260723.txt`.
- Set 0: dc0+dc1 metal-admin dynamic ranges MOVED to .201-.254 in place
  (iprange ids 2/3; office1 id 1 untouched) -- the D-134-amendment
  consequence, done before statics.
- Set 1: 10 named plane fabrics created (vr1-dc0-* ids 188-192 / vlans
  5189-5193; vr1-dc1-* ids 193-197 / vlans 5194-5198).
- Set 2: 90 NIC re-homes, one node per approval-unit, read-back each: every
  non-boot NIC moved from its private auto-fabric to its plane fabric;
  boot NICs stay fabric-4 (dc0) / fabric-142 (dc1). MACs matched
  config-of-record pins throughout.
- Set 3: provider subnets 10.12.4.0/22 + 10.12.64.0/22 MOVED off shared
  fabric-5 to their per-DC provider fabrics (+ ruled gateways .1); 8 plane
  subnets CREATED (dc0 12/16/32/36, dc1 72/76/80/84).
- Set 4: 6 spaces created (bundle binding names verified against
  bundle.yaml first); 12 untagged VLANs assigned (metal-admin = the two
  boot fabrics). Office1/transit subnets remain in `undefined` untouched.
- Set 5: 18 nodes' statics per D-134-amended assignments (control
  .100-.102, compute .120-.121, storage .150-.153, same octet all six
  planes) -- five raw-NIC statics + br-ex (OVS, Pattern A) carrying the
  provider static, provider NIC left raw (D-100). All nodes remain Ready.
- Residue NOT touched (flagged): fabric-5 still holds 192.168.1.0/24
  (factory-LAN residue, both edges re-addressed) and the now-empty
  ex-provider fabric-5 itself; the ~90 empty auto-created fabrics
  (fabric-98..187 class) also remain -- cleanup candidates, gated
  separately at stage close, zero functional impact.
- **Revert:** MAAS-record-level; reverse order: unlink statics + delete
  br-ex per node, delete the 8 created subnets, move the 2 provider subnets
  back to fabric-5, unassign spaces + delete the 6 spaces, re-home NICs to
  auto-fabrics (or delete the 10 named fabrics after), move dynamic ranges
  back to .100-.200. Nodes were powered off throughout; no OS-level state
  exists to revert.

## Item 7 -- Deploy-model ruling + DOCFIX-200 + jammy sync + placement tags

- **RULING (operator: "Ready + jammy images (Recommended)"):** Stage 4 ends
  with nodes READY + carved + tagged; MAAS-deploy is SKIPPED -- Juju
  provisions Ready machines at Stage 5 and requests the bundle's
  `default-base` (ubuntu@22.04). The phase3 runbook's Step 4 as written
  (`maas machine deploy` per node) contradicted phase-01's own precondition
  ("machines Ready/power=off") and would have made all 18 nodes
  un-allocatable to Juju. [OPS -- encoded in the proven phase-01 precedent;
  no D-number.]
- **DOCFIX-200:** `runbooks/dc-dc-phase3-maas-enlist-deploy.md` Step 4
  rewritten (tag + base-image sync, nodes STAY Ready; ruling quoted),
  sequence table line + GATE "Nodes deployed" bullet corrected against
  the buildout-design wording.
- **Jammy images:** boot-source selection id=2 (ubuntu/jammy amd64) created
  on source 1 + import triggered; `ubuntu/jammy` reads Synced (re-verify at
  close while sub-resources finish).
- **RULING (operator: "Adopt per-DC tags (Recommended)"):** deploy-placement
  tags are region-qualified per-DC -- `openstack-vr1-dc0` /
  `openstack-vr1-dc1` (D-119 naming), applied 9+9, add=9/remove=0 both.
  Stage 5's per-DC bundle parameterization constrains on them.
- **RULING (operator: "Build the mirror now"):** the D-107 per-DC artifact
  mirror gets BUILT in-stage (rehearsal exception REJECTED); Stage 4's
  mirror gate bullet stays OPEN until it exists and nodes' reachability is
  verified. Design presented as its own decision (D-135 candidate).
- Queued findings: `phase-00-maas-standup.sh`'s hardcoded PLANES table
  asserts the VR0 VLAN-103 shape and would false-DRIFT against the ruled
  D-133 flat carve (Step 8 satisfied instead by the spaces cross-check in
  the carve capture); Step 7's edge-NTP check text is superseded by the
  D-129(iv) ruling (MAAS hierarchy stays time authority; nodes measured
  NTP-ing to their rack during Stage 3 commissioning).
- **Revert:** runbook edits `git revert`; tags
  `maas admin tag update-nodes <tag> remove=...` x9 + tags delete;
  boot-source-selection delete id=2 (images age out).

## Item 9 -- INCIDENT (resolved): dc1 edge passed NO LAN traffic -- pf ruleset never regenerated

- Found at the dc1 mirror install: apt through the edge timed out. Measured
  chain: rack default route present, edge LAN gw ping 0%, internet 100% loss;
  `pfctl -s nat` on the dc1 edge showed ZERO outbound NAT rules while dc0's
  edge (identical config -- both carry the single WAN_GW gateway_item) showed
  the full automatic set. Root cause: the fresh dc1 edge's pf ruleset was
  NEVER REGENERATED after v4 addressing -- `opnsense-set-interface-v4
  --commit` runs `configctl interface reconfigure` but NOT a filter reload.
  dc0 was masked: its 07-20 plugin/qga work triggered filter reloads as a
  side effect. My dc0-only egress probe earlier this session was wrongly
  taken as covering dc1 (symmetry inference -- owned).
- Fix (minimal, logged): `configctl filter reload` on the dc1 edge -> NAT
  rules present, rack egress 0% loss / HTTP 301 -- identical to dc0.
- QUEUED: appendix-A entry (symptom: LAN clients of a freshly-addressed DC
  edge get 100% loss with reachable gateway; check `pfctl -s nat`); DC-standup
  DoD addition (filter reload + LAN-client egress probe after edge
  addressing); candidate opnsense-set-interface-v4 amendment (add the filter
  reload) -- a built-surface change, presented at stage close, not slipped in.
- **Revert:** n/a (the reload materializes the edge's own committed config).

## Item 10 -- dc-mirror installed BOTH racks; syncs RUNNING; HOME fix (harness 19)

- dc0: install pass-1 net layer up -> apt debmirror/nginx/ubuntu-cloud-keyring
  -> install PASS (15 checks; mirror answers http://10.12.8.4/ 200).
- dc1: same, after the item-9 incident fix (install PASS; http://10.12.68.4/
  200).
- First sync attempt FAILED both debmirror runs: "HOME not defined in
  environment!" under systemd's empty env (debmirror needs a GPG home).
  Fixed in gen_sync_unit (`Environment=HOME=/root`), harness case T19 added
  (19/19), REINSTALLED both racks (config-diff regeneration path exercised),
  syncs restarted: both RUNNING and downloading (dc0 400M / dc1 343M within
  the first minute). Daily timers enabled.
- Mirror gate remains OPEN until: initial syncs complete (last-sync.status
  OK) + mirror serves a real node (node-side probe at Stage 5 first boot or
  a gated rescue-boot check). Stage 4 close blocks on the mirror gate per
  the operator's build-now ruling.
- **Revert:** `systemctl stop/disable <site>-mirror-sync.timer + -net.service`,
  rm the generated files, `rm -rf /var/lib/dc-mirror`, apt remove the three
  packages; the edge filter reload stands (item 9).

## Item 11 -- Delivery batch: lib-hosts VR1 arms, D-133 guards, appendix-A, DoD

- **lib-hosts.sh VR1 arms POPULATED** (the runbook's standing delivery item):
  vr1-dc0/vr1-dc1 blocks with tofu-name-keyed HOSTS, D-134-amended octets,
  config-pinned boot MACs, per-DC power URIs (ruled step-D shape), per-DC
  tags; NEW `host_sysid_by_bootmac()` (the VR1 resolver -- MAAS hostnames are
  random and re-minted; MACs survive re-enlistment); one-selection-per-shell
  guard (lib-net precedent). dc-selector harness rewritten for the populated
  arms + identity/guard cases: 48 checks PASS.
- **D-133 guards:** `reenroll-hosts.sh` + `carve-host-interfaces.sh` now
  REFUSE vr1-* explicitly (VR0 flows: hostname-keyed re-enroll / Pattern-A
  VLAN-103 carve, both superseded for VR1) -- without the guard, populated
  VR1 arms would have let VR0 mechanics run against VR1 fleets. Harnesses
  updated to the refusal messages. Gauntlet **ALL GREEN (77)** after.
- **lib-net vr1-dc0 arm:** D-133 caveat comment -- inherited
  METAL_INTERNAL_VID/IFACE are VR0 carve artifacts; never consume for VR1.
- **appendix-A:** new entry for the edge-pf symptom (LAN 100% loss, gateway
  fine, `pfctl -s nat` empty; masking trap named). **DoD:** items 8 (filter
  reload + LAN egress probe) and 9 (dc-mirror install/sync) added; item 4's
  dynamic-range position corrected to the D-134-amended .201-.254.
- **Revert:** `git revert <sha>` (repo-only; no live state).

## Item 8 -- OWNED: pipe-masked L10 lint FAIL on the D-135 commit (abd6b92)

- The D-135 commit changed a design-decisions Status line WITHOUT
  CURRENT-STATE in the same commit (GA-R1 C1 / lint L10) and shipped anyway:
  my lint invocation piped through `tail`, masking the nonzero exit --
  the EXACT defect the 2026-07-23 queue-pass session owned same-hour
  yesterday. Repeated; owned.
- Repair: this commit carries the owed CURRENT-STATE update (post-carve
  rulings + D-135 + refreshed NEXT line). Practice fixed in-session: lint
  exit now checked via PIPESTATUS, never inferred from tail output.
- **Revert:** n/a (repair commit).
