# GUA carve completion -- proposal + missing-assignment scan (2026-08-09) **Status: DRAFT PROPOSAL. Nothing here is a ruling.** I propose; the operator rules (GA-R5). Not a status surface (GA-R1) -- stage/gate status lives in `docs/CURRENT-STATE.md` only. Origin: the geneve-over-v6 LB failure (`docs/audit/geneve-over-v6-rootcause-20260808.md`) -> a NetBox apex check -> operator direction: "assign the carves ... propose the missing gua carve, scan for any additional missing assignments, and provide a proposed matrix." **Rev 2** -- incorporates a fable-level adversarial review (findings folded in below; the review's evidence is cited where it changed a claim). **Evidence base.** Current-state figures are from a fresh READ-ONLY apex dump this session: `netbox/draft/vr1-office1-current-20260809.json` (office1-netbox 10.10.1.10, DOCFIX-195 VR1 working apex -- 152 prefixes / 27 ip-ranges / 194 ip-addresses; every number below independently re-derived in the fable review). Rulings cited inline. No mutation performed. --- ## 1. Governing frame (cited, not re-argued) - **D-139 ruling A** -- family matrix per DC: `provider-public` + `metal-admin` **dual-stack**; `metal-internal` / `data-tenant` / `storage` / `replication` IPv6-only (*target*); `lb-mgmt` IPv6-only (new plane). - **D-139 ruling B** -- **Full GUA on every plane**, from each DC's `/48` (`2602:f3e2:f02::/48` dc0, `2602:f3e2:f03::/48` dc1). **ULA `/48` `fd50:840e:74e2::/48` RETIRED for VR1.** (Deciding RFC-6724 reason struck 2026-08-01; ruling stands on Willamette/VR0 conformance + Roosevelt-delta. Ruling B is **orthogonal** to the v4-keep question -- a GUA dual-stack carve is achievable today.) - **D-139 "B plus C" narrowing** (changelog-20260731 Items 14/15/16): the v6-only conversion is a **bounded experiment on `storage` + `replication` TOGETHER** (coupled by `ceph-osd`'s global `ms_bind_ipv4=False`). **`metal-internal`, `data-tenant`, `lb-mgmt` stay dual-stack.** Item 14 (measured by charm/doc reading) records the blockers are **upstream charm defects**, not deployment faults: `metal-internal` (mysql URIs unbracketed-v6-invalid; corosync `ip_version: ipv4`), and -- the one that bears on data-tenant -- **"OVN documenting the encap column as *'The IPv4 address of the encapsulation tunnel endpoint'* with zero v6 encap values anywhere in OVN 24.03's test suite"** (flagged there as a sourced observation, *NOT re-verified*). See section 5 for how this is engaged. - **D-139 OOB amendments (2026-08-01 a/b)** -- `oob` dual-stack: v6 `f0X:f0::/60`+`/64`; v4 **dc0 `10.12.40.0/22`, dc1 `10.12.88.0/22`** (RULED, not built). VPN `:e0` deferred to Roosevelt. - **D-141** -- IPAM allocations authored **dual-stack, status-distinguished**: **v4 `active`**, **GUA v6 `reserved`** (until the consuming layer is capable; gate = `docs/charm-ip-family-compatibility.md`), **superseded ULA `deprecated`**; v6 host-numbering mirrors v4. - **D-134** -- per-plane `/22` v4 octet bands (utility `.4-.49`, VIP `.50-.99`, nodes `.100-.200`); its 2026-07-27 amendment defines the **v6 bands** -- do not invent v6 host values, read them there. - **D-139 build constraints (measured):** zero v6 `ip-range` rows are needed/possible (MAAS auto-reserves `::1`-`::ffff:ffff` per `/64`; node statics live at `::100...`; containers auto-get `:0:1::`). **A v6-only plane is expressible; dual-stack on a container plane is not** (juju takes one family via `addrs[0]`, non-deterministic -- LP #1723240; no knob). **Re-IP interaction:** the 10.12->10.13 re-IP pivot is **v4-only** (10.12 collides with the live cloud). **The GUA v6 carve is unaffected** (`f02`/`f03` unchanged) -> completing the v6 GUA apex is **durable, not throwaway**, and is the apex-drives-deploy direction D-141 requires. Only the **v4** OOB `/22`s move under the re-IP. --- ## 2. Current state -- measured (2026-08-09 apex dump; re-derived in review) **DC0 (`f02`) -- GUA carve structurally complete per convention.** - 9 planes carry GUA prefixes (`:10 :11 :20 :21 :30 :40 :50 :80 :f0`), status `active` (16 rows). - GUA VIP host-addrs: **39, all `reserved`** (13 each on `:11`/`:20`/`:21`) ok D-141. - ULA prefixes + ULA VIPs (26): all `deprecated` ok. - Node v6 statics: 0 apex records -- **correct by convention** (v4 side records no `.100-.200` nodes either). **DC1 (`f03`) -- GUA carve INCOMPLETE (headline gap).** - GUA prefixes: **provider-public only** (`:10 :11`). **MISSING 7 planes / 13 prefix rows** (`:20 /60+/64`, `:21 /64`, `:30 /60+/64`, `:40 /60+/64`, `:50 /60+/64`, `:80 /60+/64`, `:f0 /60+/64`). - GUA VIPs: **13 `reserved`** (provider-public only). **metal-admin/internal GUA VIPs MISSING (26).** - ULA prefixes still **`active`** (5 planes / 9 rows: `320/321/330/340/350`). ULA VIPs (26) still `reserved` on `:320`/`:321` -- DC1's metal-admin/internal VIPs live **only** on retired-family ULA. **Cross-cutting findings (some are ruled-vs-built divergences, not just gaps):** - **v6 utility host records = 0 on both DCs** -- juju `.5`/maas `.6`/tailscale `.7`/rack `.2` are v4-only. D-141 rule 1 (author dual-stack) -> v6 `reserved` twins owed. Low consequence (utility is v4-reached). - **lb-mgmt `:80` prefix `active` on dc0** -- but D-139's G18 annotation (2026-08-08) rules the apex GUA lb-mgmt **`reserved`** (no charm consumer). **Two rows** exist (`f02:80::/60` AND `::/64`); the G18 text names only the `/64` -- the `/60`'s status is unspecified and must be asked, not assumed. - **All 78 v4 VIP host-addrs are `reserved` on BOTH DCs** (not `active`). If dc0's API VIPs are live (dc0 is deployed, core services active), the apex is **under-promoted** vs D-141 rule 2 -- a divergence to confirm/correct. If they are not yet verified-live, `reserved` is correct. Not asserted either way here. - **Utility `.4` (apt-cacher proxy, D-135 + 2026-08-02(b) convergence ruling) has no apex host row** on any plane, either DC -- a candidate genuinely-missing v4 assignment (verify proxy build state before asserting). --- ## 3. Proposed GUA assignment matrix (both DCs) Prefix `active` = the plane object exists (D-139 carve convention). VIP-allocation `reserved` per D-141. `lb-mgmt` prefix `reserved` per G18. Host-numbering mirrors the v4 octet (D-134 + its v6-band amendment). | Plane | v6 `/60` parent | dc0 `/64` | dc1 `/64` | v4 (family) | Prefix status | VIP-alloc status | |---|---|---|---|---|---|---| | provider-public | `f0X:10::/60` | `f02:10::/64` | `f03:10::/64` | 10.12.4 / 64 (**dual**) | active | reserved | | metal-admin | `f0X:20::/60` | `f02:20::/64` | `f03:20::/64` | 10.12.8 / 68 (**dual**) | active | reserved | | metal-internal | (in `:20::/60`) | `f02:21::/64` | `f03:21::/64` | 10.12.12 / 72 (dual; v6-only *target*, blocked section 5) | active | reserved | | data-tenant | `f0X:30::/60` | `f02:30::/64` | `f03:30::/64` | 10.12.16 / 76 (dual; **v6-only decision section 5**) | active | -- | | storage | `f0X:40::/60` | `f02:40::/64` | `f03:40::/64` | 10.12.32 / 80 (v6-only *experiment*, gated) | active | -- | | replication | `f0X:50::/60` | `f02:50::/64` | `f03:50::/64` | 10.12.36 / 84 (v6-only *experiment*, gated) | active | -- | | lb-mgmt | `f0X:80::/60` | `f02:80::/64` | `f03:80::/64` | none (v6-only, new) | **reserved** (G18) | -- | | oob | `f0X:f0::/60` | `f02:f0::/64` | `f03:f0::/64` | **dc0 10.12.40 / dc1 10.12.88** (**dual**) | active | -- | *Prefix rows are authored now regardless of family conversion (both families coexist in the apex). "v6-only experiment" (storage/replication) is a **bounded experiment gated on the still-untaken `network-get` measurement**, not greenlit deployment.* --- ## 4. Missing-assignment scan -- enumerated | # | Missing / divergent assignment | Layer | Tool / step | Note | |---|---|---|---|---| | M1 | **DC1 GUA plane prefixes** (7 planes / 13 rows) | apex v6 | `netbox/d139-gua-carve.py --dc vr1-dc1` (CREATE-only; D-139 step 1) | executes ruled D-139 | | M2 | **DC1 metal-admin + metal-internal GUA VIPs** (26, `reserved`) | apex v6 | D-139 step 6 re-home | **needs its own GA-R5** (section 5.2) -- D-139 ORDERING RULING reserved step 6 | | M3 | **DC1 ULA retirement** -- 9 prefixes + 26 VIPs -> `deprecated` | apex v6 | D-139 step 6, **AFTER M2** (DEFECT 3) | same GA-R5 as M2 | | M4 | **OOB v4 `/22`** (dc0 10.12.40, dc1 10.12.88) | apex+MAAS v4 | RULED not built; carve tool is v6-only -> **tool gap** | **moves under re-IP** -> defer (section 5.2) | | M5 | **lb-mgmt status** -- dc0 `:80` `/64` (and `/60`?) `active`->`reserved`; author dc1 `:80` `reserved` | apex v6 | status edit(s) | **two rows**; `/60` status is an operator question (G18 named only `/64`) | | M6 | **v6 utility host records** on dual-stack planes | apex v6 | D-141 rule 1; **v6 values from D-134 amendment, not invented** | low consequence | | M7 | **DC1 v4 utility host records** (thinner than dc0) | apex v4 | at DC1 standup | expected while DC1 held | | M8 | **`.4` apt-cacher proxy host record** (both DCs) | apex v4 | verify proxy build first | candidate real gap (D-135 / 2026-08-02(b)) | | S2 | **MAAS GUA subnet carve per DC** (distinct from node statics) | **MAAS** | D-139 step 2 | out of apex-only scope; rack-side; listed so it is not lost | | S4 | **Octavia v6 IP-SAN reissue + `lib-net.sh` v6 arm** | PKI + repo | D-139 step 4 (`octavia-pki.sh reissue`) | out of apex-only scope | | S3 | **Node v6 statics in MAAS** (`::100...`/node) | **MAAS** | `dc-node-v6-carve.py` (D-139 step 3) | **NOT asserted absent** -- needs rack access; dc0 data-tenant confirmed carved (`::120/::121`), completeness UNMEASURED | | S5 | **lb-mgmt VLAN/space/subnet** | MAAS | D-139 step 5 | never carved either family | **Overlay/render coupling (from review -- must not be missed):** M2/M3 touch only the apex, but the D-136 **rendered per-DC VIP overlays** carry DC1's v6 VIPs in the ULA range. Retiring/ re-homing in the apex without regenerating the overlay consumers would leave a later DC1 deploy pointing at `deprecated` ULA VIPs (memory #16: enumerate a change's consumers). Enumerate the D-136 consumers as part of M2/M3. **Classification:** M1, M5, M6 execute existing D-139/D-141 rulings (gated mutations). **M2/M3 do NOT** -- see section 5.2. S2-S5 are MAAS/PKI/repo steps outside this apex-only proposal, listed for completeness. --- ## 5. Decisions ### 5.1 RESOLVED 2026-08-09 -- geneve-over-v6 CONFIRMED working; v6-only data-tenant stands (D-number owed) **Governing preference (operator, 2026-08-09; = the standing D-101/D-139 posture):** *"v6-only, when not possible dual-stack v4/v6, and only if that is not possible v4."* **The prerequisite this section used to gate on is now MEASURED, and it PASSED.** geneve-over-IPv6 works on this exact stack (OVS 3.3.0 / OVN 24.03.2 / kernel 5.15.0-186) -- LIVE-CONFIRMED on vr1-dc0: a real VM->VM cross-compute ping over a v6 geneve tunnel = **8/8, 0% loss** (evidence + tested sequence: `docs/audit/geneve-over-v6-rootcause-20260808.md`, LIVE-CONFIRMED section). **So v6-only `data-tenant` is viable and v4-forced is OFF the table** -- the posture's first rung holds. (A prior same-day test wrongly concluded "v6 broken"; it sourced pings from an OVN `localport` that never tunnels by design -- retracted.) **The v6-only fix is THREE parts, all now understood (it is NOT "just make the plane v6-only"):** 1. **Carve v6 on the containerized (octavia LXD) chassis' data-tenant leg** -- they auto-pick v4 (D-134); the original 2026-08-08 family split. Carve/assign, not auto-pick. 2. **Deliver `ovn-encap-ip` UNbracketed** -- `ovn-chassis` 24.03 emits it bracketed (`"[2602:...]"`), which OVS geneve rejects (tunnel `ofport -1`, `bad remote_ip`). Fixed charm revision OR a persistent post-deploy override. 3. **neutron `overlay_ip_version=6`** -- tenant MTU ~1422 (v6 geneve overhead), else a later large-packet bug. **dual-stack is still NOT a viable middle for this plane** (metal v6 + container v4 via `addrs[0]` = the split); the choice was only ever v6-only vs v4-only, and v6-only is now proven achievable. **Ordering hazard (D-139 DEFECT 2, unchanged):** `dc-node-v6-carve.py` needs v4 present (`if not v4: continue`; host part = v4 last octet). Carve node v6 statics WHILE v4 exists, THEN remove v4 -- or rewrite the script to pivot on the plane/space. **The gate is now BUILT:** `scripts/geneve-encap-assert.sh` (harness 16/16) asserts encap-family consistency AND every tunnel `ofport>=0` (the family-only check missed the bracket bug); wired into phase-04 Step 12.2. Run at Stage-5 close before any workload smoke. **DECISION for ruling (GA-R5):** ratify **v6-only `data-tenant`** (the fix is understood and proven) and settle the **D-number** -- the rootcause doc PROPOSED a new number; I lean **D-139 amendment** (parent owns the family matrix; avoids the mint-a-number trap). The finding is broader than one plane (any LXD-hosted ovn-chassis + the bracket bug) -- the Roosevelt-delta that argues either way. Operator's call. ### 5.2 Enumerated, NOT presented for ruling this exchange (recorded so they are not lost) 1. **DC1 step-6 (M2/M3)** is reserved by D-139's ORDERING RULING for **its own GA-R5 exchange** (steps 4-6 "were NOT put"; step 6 "needs its own GA-R5 exchange ... NOT inferred"). So M2/M3 are a **future decision**, not mere execution -- do not run them on this proposal's authority. 2. **OOB v4 timing (M4):** values move to `10.13.*` under the owed re-IP ruling -> defer to it; don't carve v4 now to renumber later. (OPS/ordering.) 3. **DC1 apex prefixes (M1) now vs fold into rebuild:** recommend **now** -- v6 GUA is re-IP-independent, the apex is the design source of truth (D-141), and M1 executes D-139 already ruled (gated mutation). --- ## 6. Geneve encap gate -- BUILT 2026-08-09 (was ruled-not-built) D-139/D-101 named geneve-over-v6 a verification gate; nothing executable asserted it. **Now delivered:** `scripts/geneve-encap-assert.sh` (+ `tests/geneve-encap-assert/run-tests.sh`, 16/16) enumerates every chassis `Encap` IP and asserts **single-family, == the ruled family** (`--expect-family v6`, parameterized -- not a hardcoded v6) AND **every geneve tunnel `ofport>=0`** (the bracket-bug check the family-only version missed); proven in both failing directions. Wired into phase-04 Step 12.2. **Still owed** -- D-139's OTHER gate: **every container's `metal-admin` leg is IPv4** (the D-139 CARRIED RISK; more critical once `data-tenant` goes v6-only, leaving `metal-admin` the last dual-stack container plane) -- not yet built. --- ## 7. Limits / what I did NOT do - Apex-only proposal. **No MAAS-side measurement** (S2/S3/S5 need rack access; not asserted absent). - The OVN-v6-geneve capability is established generically (web sources); the **deployment-specific datapath is unproven** and is the section 5.1 gate. The `addrs[0]`<->D-134-auto-pick link is an inference joining two records, not this incident's independently measured mechanism. - "0 addresses in the apex" corroborates the 2026-07-31 live "zero ranges" finding (same direction); it is a design-level read, not a fresh live re-measure. D-139 established zero-ranges != can't-allocate. - No mutation performed. Every M-item is a gated action awaiting approval; M2/M3 additionally await their own GA-R5.