Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260809-geneve-v6-rootcause-gate.md

changelog 2026-08-09 -- geneve-over-v6 root cause (bracket bug) LIVE-CONFIRMED + executable gate

Session body (GA-R2, one per session). Status lives ONLY in docs/CURRENT-STATE.md. Branch dc-dc-stage5-preconditions. Live work ran FROM the dc0 rack 172.31.0.2 (vvr1-dc0) via voffice1 (D-138). dc0 is throwaway (10.13 rebuild follows); live mutations were test scaffold, all torn down.

Item 1 -- geneve-over-v6 ROOT-CAUSED (bracket bug) + LIVE-CONFIRMED working

Extends the 2026-08-08 family-split root cause. Diagnostic (bounded, after a fable advisory caught an INVALID first test -- pings sourced from an OVN localport/ovnmeta namespace, which OVN never tunnels by design; that wrong "v6 broken" conclusion was retracted). REAL cause of the v6 tunnel failing: ovn-chassis 24.03 sets external_ids:ovn-encap-ip BRACKETED ("[2602:f3e2:f02:30::120]"); OVS geneve rejects it (bad geneve 'remote_ip'), tunnel ofport -1. Isolated test proved kernel 5.15/OVS 3.3 accept an UNbracketed v6 geneve remote_ip (ofport valid) and reject the bracketed form. Live fix + confirm on dc0: ovs-vsctl set open_vswitch . external_ids:ovn-encap-ip=<unbracketed> on both computes -> ovn0-vr1-d-0 ofport -1->14 -> real VM->VM cross-compute ping over geneve-over-IPv6 = 8/8, 0% loss. geneve-over-v6 WORKS on this stack; v4-forced is OFF the table.

  • Files: docs/audit/geneve-over-v6-rootcause-20260808.md (RETRACTION + RESOLVED + LIVE-CONFIRMED sections, with the tested sequence), docs/CURRENT-STATE.md (section 1 UPDATE 2026-08-09).
  • Revert: git checkout <prev> -- docs/audit/geneve-over-v6-rootcause-20260808.md docs/CURRENT-STATE.md. Live: the manual encap override is transient (charm re-asserts on next hook) and on throwaway dc0; no revert needed. Test scaffold (geneve-proof-*, lb-smoke) already deleted.

Item 2 -- NEW GATE: scripts/geneve-encap-assert.sh (+ harness 16/16)

Makes the phase-04 "geneve-over-v6 verified" gate EXECUTABLE (was ruled-not-built). Asserts C1 (all chassis Encap.ip single-family, optional --expect-family v6 for the D-139 ruled family) AND C2 (every geneve tunnel ofport >= 0 -- catches the bracket bug that C1 alone misses). Offline (fixture-testable) + --live (gathers via juju on the rack). Harness proves BOTH failing directions (split -> C1 FAIL; ofport -1 -> C2 FAIL) + refuse-not-pass on empty + rc-2 usage.

  • Files: scripts/geneve-encap-assert.sh, tests/geneve-encap-assert/run-tests.sh (16 pass).
  • Revert: git rm scripts/geneve-encap-assert.sh tests/geneve-encap-assert/run-tests.sh.

Item 3 -- Runbook phase-04 CORRECTED (Step 12.2 gate + Step 6 note)

Step 12.2 was FAMILY-ONLY (would PASS on a bracketed v6 encap with no working tunnel) and cited the retired ULA (D-139 is full-GUA). Rewrote it to lead with bash scripts/geneve-encap-assert.sh --live, add the C2 tunnel-health check + the bracket-bug fix, and correct ULA->GUA. Step 6's "geneve family follows the interface once the plane is v6-only" note gained a measured CAVEAT (necessary-not-sufficient: containerized-chassis v6 carve + unbracketed encap).

  • File: runbooks/dc-dc-phase4-juju-bundle-per-dc.md (Step 12.2, Step 6).
  • Revert: git checkout <prev> -- runbooks/dc-dc-phase4-juju-bundle-per-dc.md.

Item 4 -- GUA carve completion proposal (context; superseded framing corrected)

docs/audit/gua-carve-completion-proposal-20260809.md (Rev 2, fable-reviewed) carries the DC1 GUA carve gap + missing-assignment scan + matrix. Its section-5.1 geneve decision is RESOLVED by Item 1 (v6 viable). Proposal-doc update to reflect the resolution is the next task (operator flow "A then B").

  • Revert: git rm docs/audit/gua-carve-completion-proposal-20260809.md (if unwanted).

Owed (rebuild / ruling)

  • D-number for the containerized-chassis-v6 + bracket finding (amendment vs new -- operator's call).
  • Confirm the fixed ovn-chassis charm revision (LP charm-ovn-chassis IPv6 bracket family) OR a persistent post-deploy unbracket override; wire into the deploy.
  • neutron overlay_ip_version=6 (tenant MTU ~1422) for the v6 overlay.