# geneve-over-v6 overlay FAILURE -- root cause (dc0 activation checkpoint, 2026-08-08)

Status authority is `docs/CURRENT-STATE.md`; this is the root-cause record for the
checkpoint "verify-live geneve-over-v6" wrap-gate item. Surfaced by the 1-test-LB smoke
test (D-021 amphora build had just been fixed). All claims below are MEASURED this session.

## Symptom
- 1 test LB `24bd326f-611c-4a8c-a348-e9c67f5d156c` (VIP 198.51.100.21) stuck `PENDING_CREATE`.
- Amphora nova instance `66542b9f-...` is ACTIVE (image `775ebeba-...` boots -> image validated),
  on lb-mgmt-net at `fc00:5b7a:7bdc:bd86:f816:3eff:fe2b:9473`, but Octavia amphora = `BOOTING`.
- octavia health-manager log: repeated `ConnectTimeout` to `[fc00:...:9473]:9443` (NOT ssl/cert/
  refused -> not a version/agent problem).
- `lb-mgmt-sec-grp` correctly allows `tcp IPv6 ::/0 9443` + `ipv6-icmp` (SG is NOT the cause).
- ping6 from o-hm0 (`fc00:5b7a:7bdc:bd86:f816:3eff:fef9:2dc8`, same /64) -> amphora = **100% loss**
  (even ICMPv6, which the SG allows) -> a same-subnet OVERLAY delivery failure, cross-node.

## Root cause (measured)
OVN geneve **encap IPs are split across address families** between containerized control
chassis and metal compute chassis, so no control<->compute geneve tunnel can form:

| Chassis | Role | `ovn-sbctl` Encap geneve ip | Family |
|---|---|---|---|
| `juju-d4f45a-0-lxd-13` | octavia/o-hm0 (control, LXD) | `10.12.16.1` | IPv4 |
| `juju-d4f45a-1-lxd-15` | control (LXD) | `10.12.16.2` | IPv4 |
| `juju-d4f45a-2-lxd-13` | control (LXD) | `10.12.16.3` | IPv4 |
| `vr1-dc0-compute-01`   | amphora host (metal) | `[2602:f3e2:f02:30::120]` | IPv6 |
| `vr1-dc0-compute-02`   | compute (metal) | `[2602:f3e2:f02:30::121]` | IPv6 |

Corroboration on BOTH compute chassis (`ovs-vsctl list interface`) -- INCLUDING
`vr1-dc0-compute-01`, the amphora's actual host (`ovn-chassis/1`): geneve tunnels to the three
control chassis carry IPv4 `remote_ip=10.12.16.{1,2,3}` while the local endpoint is IPv6 ->
cross-family, cannot encapsulate. Only the compute<->compute tunnel (`[2602:f3e2:f02:30::12x]`,
v6<->v6) is family-consistent. NOTE on evidence weight: `bfd_status` read empty on all tunnels,
but that is AMBIGUOUS (OVN geneve does not necessarily populate BFD) and is NOT relied on -- the
DECISIVE evidence is the encap family split + the 100% ICMPv6 loss on the tested control->compute
path. The v6<->v6 compute<->compute data path is UNTESTED (no second instance existed to ping),
so it is NOT claimed healthy.

## Mechanism -- why control got v4, compute got v6
The octavia control chassis is an **LXD container** whose data/geneve-plane interface
(`eth2`) has ONLY `10.12.16.1/22` (IPv4); it has NO v6 address on the data-tenant plane
(its only v6 is `o-hm0`, the lb-mgmt OVERLAY it serves, not the underlay). ovn-chassis
therefore derives an IPv4 `ovn-encap-ip`. Metal compute nodes carry IPv6 on the same plane
and derive a v6 encap. Governing prior art: **D-134 records LXD container addresses are
AUTO-PICKED (not carved)** -> the auto-pick landed IPv4-only for the containerized chassis,
diverging from the carved metal's IPv6. D-101 intends geneve-over-v6 on the data-tenant
underlay and names it a Phase-0 verification gate; this is the live proof that gate is not
clean.

SAME PLANE, family-selection defect (NOT a binding error): `lib-net.sh` PLANE_CIDRS confirms
`10.12.16.0/22` = the `data-tenant` plane, and `data-tenant` is in `SPACES6` (v6-capable); the
v6 side is the GUA `2602:f3e2:f02:30::/64` seen on metal. Both node types bind the same
`data-tenant` space -- the divergence is address-family SELECTION on one dual-stack plane
(containers got a v4-only address there, metal got v6), not a mis-binding. Note also that neither
side matches D-101's "IPv6-only **ULA** data-tenant" literally (compute uses a GUA, control v4).

## Roosevelt-delta (why this outlives throwaway dc0)
Any v6 build with CONTAINERIZED OVN chassis (octavia/ovn-chassis-octavia, and any LXD-hosted
ovn-chassis) must give those containers a v6 data-tenant address so geneve encap is v6 and
matches metal. The auto-picked LXD addressing (D-134) does not currently guarantee this. This
is the containerized-chassis-v6-underlay problem; it will recur on the 10.13 rebuild. Warrants a
new D-number (GA-R3: architectural + Roosevelt-delta) -- PROPOSED; number to be assigned by
grep-for-next-free AT ruling time (operator ruling owed), not pre-allocated here.

## Fix landscape (all gated; none applied)
- **Durable (rebuild):** ensure containerized OVN chassis get a v6 address on the data-tenant
  plane (carve/assign, not auto-pick) so `ovn-encap-ip` is v6 cloud-wide; OR pin the geneve
  encap family consistently. This is the real fix and belongs in the rebuild bundle/addressing.
- **Live dc0 workaround (to complete the smoke test only):** add a v6 data-tenant address to
  the 3 control-node OVN-chassis containers and set `ovn-encap-ip` to it (family-consistent
  with compute). Non-trivial (the containers have no v6 there today); throwaway-only.
- **Alternative:** flip compute encap to v4 to match control -- REJECTED direction (inverts the
  geneve-over-v6 design intent, D-101).

## Evidence
This session's captured output (transcript): `ovn-sbctl show`, `ovs-vsctl list interface`
(bfd_status empty), octavia health-manager log ConnectTimeout, ping6 100% loss, SG rule list,
`ip addr` on octavia/leader (eth2 v4-only on data). Full changelog at session close.