diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index b375bb8..98b9e4e 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -96,6 +96,16 @@ > open D-139 v6-route, same leg). D-139's plane matrix + geneve-over-v6 + LP#1723240 (D-141) are > NOT unlocked by the collapse (different layers). Note: > `docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md`. +> **IPv6 FEASIBILITY RE-CHECK (2026-08-10, pass6 -- CORRECTS the pass5 note, which relied on prior +> rulings instead of researching feasibility).** Fresh ruling-independent research (upstream sources, +> deployed versions): IPv6 is substantially MORE feasible than D-101/D-139 held -- metal-admin IPv6 +> commissioning FEASIBLE-WITH-WORK on UEFI (only legacy-BIOS PXE is a firmware blocker), MAAS +> rack<->region v6 FEASIBLE-NOW, provider-public v6 FEASIBLE-NOW via direct GUA routing (no FIP); +> D-101's "PXE v4-first" and D-139's "provider dual because internet=v4" are overstated/conflated. +> CONFIRMED still-hard: juju LP#1723240 (container VIPs, D-141 holds) + external v6 internet. NEW: Ceph +> one-family-per-daemon constrains DEC-24 v6 replication. CHALLENGES prior framings -> surfaced for +> operator reconsideration, NOT ruled (GA-R5). Note: +> `docs/audit/container-elim-pass/pass6-ipv6-feasibility-note-20260810.md`. > > **dc0 checkpoint scope (operator 2026-08-08): "activate + smoke-test"** -- networks + Octavia (1 test > LB) + Designate (1 test zone) + wrap gates (cloud-assert BOM, controller backup, verify-live diff --git a/docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md b/docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md index d747990..6c7c379 100644 --- a/docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md +++ b/docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md @@ -1,5 +1,15 @@ # IPv6-unlock design note -- what the container-layer collapse (D-144) does and does NOT open for IPv6 +> **CORRECTED 2026-08-10 by `pass6-ipv6-feasibility-note-20260810.md`.** This note RELIED ON PRIOR +> RULINGS for the links that had them (it called metal-admin/PXE and provider-public "forced-v4 by +> ruling" and punted the researchable question), which was NOT the fresh feasibility review the +> operator asked for. Pass6 did the actual capability research: metal-admin IPv6 commissioning is +> FEASIBLE-WITH-WORK on UEFI (only legacy-BIOS PXE is a firmware blocker), MAAS rack<->region v6 is +> FEASIBLE-NOW, provider-public v6 is FEASIBLE-NOW via direct GUA routing (no FIP). The "forced-v4" +> framing below is SUPERSEDED for those links -- read pass6 for the ruling-independent verdicts. The +> parts of this note about what the collapse STRUCTURALLY touches (management-plane redesign, the +> (a)/SEC-010 controls, DEC-24) remain valid. + **Date:** 2026-08-10. **Status:** DESIGN NOTE (findings only, GA-R5 -- nothing ruled, nothing built). Owed design input recorded under **D-144**. Read-only follow-on pass (pass5, 3 workers): full surface enumeration + classification matrix in `pass5-w1-mgmt-v6.md`, `pass5-w2-metal-data-v6.md`, diff --git a/docs/audit/container-elim-pass/pass6-ipv6-feasibility-note-20260810.md b/docs/audit/container-elim-pass/pass6-ipv6-feasibility-note-20260810.md new file mode 100644 index 0000000..e0e6699 --- /dev/null +++ b/docs/audit/container-elim-pass/pass6-ipv6-feasibility-note-20260810.md @@ -0,0 +1,78 @@ +# IPv6 FEASIBILITY note (pass6) -- fresh, ruling-INDEPENDENT research + +**Date:** 2026-08-10. **Status:** FINDINGS ONLY (GA-R5 -- nothing ruled, nothing built). Records +under **D-144**; **CORRECTS the pass5 note** (`ipv6-unlock-design-note-20260810.md`), which relied on +prior rulings for the links that had them instead of assessing feasibility fresh. + +**Why this pass exists (operator, 2026-08-10):** "the object of this review was not to follow what was +previously ruled but to take a fresh look at whether or not IPv6 was feasible on any of the links that +had had previous rulings or decisions or testing attached to them." Pass5 did NOT do that for the +ruled links -- it cited D-101/D-139 ("PXE is v4-first", "provider-public dual-stack because the +internet is v4") as the answer, and explicitly punted the researchable question ("do not import outside +knowledge"). This pass researches actual platform capability, against the DEPLOYED versions (MAAS +3.7.2, Juju 3.6.27, Caracal 2024.1, OVN 24.03.2, OPNsense 26.7, Ceph Reef-era), with real upstream +citations. Per-link detail + URLs: `pass6-w1-maas-ipv6-feasibility.md`, `pass6-w2-openstack-ipv6- +feasibility.md`, `pass6-w3-juju-charm-ceph-feasibility.md`. + +## The honest answer: IPv6 is substantially MORE feasible than the prior rulings held + +The two "forced-v4 anchors" pass5/D-101/D-139 leaned on do NOT hold up as feasibility statements -- +both were judgements or conflations, not measured infeasibility: + +| Link (had a prior ruling/decision) | FRESH verdict (ruling-independent) | Real blocker / work | vs the prior position | +|---|---|---|---| +| metal-admin: DHCPv6 commissioning | **FEASIBLE-WITH-WORK** | no MAAS-side blocker @3.7.2 (the 2 historical bugs closed 2017/3.1); owed a live test | **CHALLENGES** D-101 "PXE v4-first" | +| metal-admin: UEFI HTTP boot over v6 | **FEASIBLE-WITH-WORK** | firmware supports it; boot path not doc-confirmed end-to-end | **CHALLENGES** | +| metal-admin: legacy-BIOS PXE over v6 | **INFEASIBLE** | firmware (Intel Boot Agent / PXE-2.1 is v4-only) -- but applies ONLY to legacy-BIOS-mode nodes, NOT UEFI | **NARROWS** the ruling to a firmware-mode edge case (our VMs are UEFI by choice) | +| MAAS rack<->region RPC over v6 | **FEASIBLE-NOW** | MAAS source prefers `AF_INET6`; the one bug (LP#2020142) fixed in 3.5.0, before 3.7.2 | **CHALLENGES** | +| provider-public external over v6 | **FEASIBLE-NOW** | direct GUA routing, **no NAT, no floating IP** -- FIP is simply the wrong model for v6 | **CHALLENGES** D-139 "dual-stack because internet is v4" (a conflation -- see below) | +| Octavia LB VIP over v6 | **FEASIBLE-NOW** | `vip_subnet_id` takes v6 natively (Caracal) | -- | +| API VIP (hacluster) over v6 | **FEASIBLE-WITH-WORK** | ONE charm bug: hard-coded `ip_version: ipv4` in the corosync template (LP#2111852, fix committed, unreleased) -- not a corosync/platform limit | **NARROWS** D-141/pass5 "v6 stack not ready" | +| container-hosted API-charm VIP over v6 | **INFEASIBLE (today)** | juju **LP#1723240** -- live-verified still `Triaged`/`Low`, unfixed in 3.6.27 and 4.0 | **CONFIRMS** D-141's core premise | +| core charm IPv6 endpoint binding | **FEASIBLE-WITH-WORK** | v6-capable; gated by LP#1723240 + charm gates (ceph-osd LP#2109798 New; ceph-mon/hacluster fix-committed) | **MATCHES** the repo's existing table | +| simulated-ISP uplink edge over v6/dual | **FEASIBLE-WITH-WORK** | OPNsense 26.7 needs zero work (native dual-stack); gap is repo-side (`site-wan` module is v4-only NAT); correct v6 model is **routed, not NAT66** | **CHALLENGES** "forced v4" (it's a build choice, not a capability limit) | +| Ceph cross-DC replication over v6 (DEC-24 leg) | **FEASIBLE-WITH-WORK** | new constraint: Ceph binds **one family per daemon** (`ms_bind_ipv6`/`ms_bind_ipv4`), so v6 needs the WHOLE cluster single-family, not just the replication leg | NEW -- pass5/repo left this UNASSESSED | +| OVN data-plane IPv6 (internal ref doc) | **FEASIBLE (already proven)** | the repo's OWN geneve-over-v6 audit (2026-08-09) proved VM->VM v6 geneve 8/8 0% loss | **STALE INTERNAL DOC:** `charm-ip-family-compatibility.md` still says "No" -- never reconciled | + +## The three genuine, surviving blockers (what fresh research CONFIRMS is hard) + +1. **Legacy-BIOS PXE over IPv6** -- a real firmware constraint, but avoidable: UEFI HTTP boot is the + path, and we define the VMs, so choosing UEFI removes it. Not a reason metal-admin must be v4. +2. **juju LP#1723240** -- container-hosted API-charm VIPs genuinely cannot pick their address family + (unsorted-query `addrs[0]`); still open in 3.6.27. D-141's v4-active/v6-reserved holds for THAT + specific case. (Note: this is the LXD API-charm container layer -- unrelated to D-144's containment-VM + elimination.) +3. **External IPv6 internet reachability** -- the actual upstream internet in the rehearsal is v4; + a v6-native provider NETWORK is feasible now, but real external v6 transit is a separate, real-world + item D-101 deliberately defers. This is the ONE place "because the internet is v4" is a true + constraint -- but it constrains EXTERNAL REACHABILITY, not the provider network's family. + +## The key conceptual correction + +**IPv6 does not use floating IPs at all** -- direct GUA routing is the *native* v6 model, not a lesser +substitute. The prior "provider-public must be dual-stack because the internet is v4" conflated two +different things: (a) the provider network's ADDRESS FAMILY (can be v6-native today) and (b) EXTERNAL +v6 INTERNET reachability (genuinely deferred). (a) is feasible now; only (b) forces a v4 presence, and +only for actual internet egress. + +## What this means for D-144's owed work (surfaced for the operator, NOT ruled) + +- **metal-admin can plausibly be IPv6 for commissioning** if the fleet is UEFI + a DHCPv6/HTTP-boot + path is built and LIVE-TESTED. This reopens whether D-101/D-139's "metal-admin dual-stack / PXE + v4-first" should stand as-is for the redeploy -- a decision for the operator, backed now by capability + evidence, not a ruling this pass makes. +- **provider-public, Octavia, the uplink edge** are v6-feasible now / with modest repo-side work; the + only true external-v4 need is real internet egress. +- **DEC-24** gains a hard design constraint: v6 cross-DC replication requires whole-cluster + single-family v6 (Ceph one-family-per-daemon). +- **Two internal corrections owed** (DOCFIX candidates, not done here): reconcile + `charm-ip-family-compatibility.md`'s OVN "No" row against the repo's own geneve proof; and soften + D-101's absolute "PXE is v4-first" to the researched reality (legacy-BIOS-only firmware limit + owed + UEFI-v6 test) so a future session does not inherit the overstatement. + +## Honest scope note + +Every "FEASIBLE-NOW/WITH-WORK" here is a CAPABILITY finding cited to upstream docs/source/bug-trackers +for the deployed versions -- NOT a live proof on this cloud. Where a live test is the last mile (DHCPv6 +commissioning, UEFI-v6 boot), it is named as owed. This pass corrects the METHODOLOGY error of pass5 +(citing rulings as feasibility); it does not itself execute or rule. diff --git a/docs/audit/container-elim-pass/pass6-w1-maas-ipv6-feasibility.md b/docs/audit/container-elim-pass/pass6-w1-maas-ipv6-feasibility.md new file mode 100644 index 0000000..6537543 --- /dev/null +++ b/docs/audit/container-elim-pass/pass6-w1-maas-ipv6-feasibility.md @@ -0,0 +1,56 @@ +# Pass 6 / W1 -- MAAS 3.7 IPv6 feasibility (fresh capability research, no prior-ruling deference) + +Date: 2026-08-10 +Scope: per orchestrator instruction -- assess actual platform capability for IPv6 on MAAS +provisioning/control-plane links, **independent of any prior repo ruling** (D-101/D-139/D-143). +A prior ruling that a link "is v4" is not treated as evidence v6 is infeasible. Deployed +targets: MAAS 3.7.2, Ubuntu 24.04/22.04 nodes, kernel 5.15/6.8. + +Web research: SUCCEEDED (WebSearch + WebFetch, ~20 queries/fetches against docs.maas.io / +canonical.com/maas/docs, Launchpad, GitHub canonical/maas source, Ubuntu wiki, Intel support). +Current `canonical.com/maas/docs` pages proved thinly-populated on IPv6 specifics (several +fetches returned "no IPv6 content in this section"), so findings lean on: (a) Launchpad bug +history showing which IPv6 code paths were fixed and in what version, (b) the MAAS source tree +itself (regionservice.py / clusterservice.py / network.py) for RPC address-family handling, and +(c) vendor/firmware documentation (Intel, Ubuntu wiki) for boot-ROM-level IPv6 constraints that +sit below MAAS entirely. Each row below is sourced individually. + +## Per-link verdict table + +| # | Link | Verdict | The work / the exact blocker | Capability citation | Disagrees with prior ruling? | +|---|------|---------|-------------------------------|----------------------|-------------------------------| +| 1a | DHCPv6 for enlistment/commissioning | **FEASIBLE-WITH-WORK** | MAAS has supported IPv6 subnets/DHCPv6 since the 2.x line (rack-controller interface with a static IPv6 range; MAAS does not rely on RAs, configures static v6 routes). Two historical MAAS-side bugs that would have blocked this are both closed well before 3.7.2: LP#1718209 (DHCPv6 PXE conditional always evaluated false -> wrong bootloader) fixed 2017 in MAAS 2.2.3/2.3.0beta1; LP#1939034 (commissioning assigned IPv6 `/128` instead of `/64`, causing manual DB cleanup) fixed in MAAS 3.1.0-beta1. MAAS's DHCP backend is isc-dhcp-server (not dnsmasq), which matters because dnsmasq explicitly lacks the DHCPv6 TFTP-boot extensions per Ubuntu's own netboot docs -- isc-dhcp-server is the one that supports them. Net: the enlistment/commissioning DHCPv6 path is architecturally present and its known defects are old and closed; what's missing is a **measured test on this deployment** (3.7.2, real NICs) -- it has never been run here. | docs: canonical.com/maas/docs (rack ctrl IPv6 static-range requirement, no-RA design) via Canonical MAAS Discourse "IPv6 addressing" series (2.8/3.0); LP bugs #1718209, #1939034 (canonical.com/maas + bugs.launchpad.net/maas); Ubuntu Wiki `UEFI/SecureBoot/PXE-IPv6` (dnsmasq DHCPv6-TFTP gap, isc-dhcp-server requirement); GitHub canonical/maas `debian/control` (isc-dhcp-server dependency) | **YES.** Repo (D-143 note, docs/design-decisions.md:8332-8333) already correctly frames this as "UNVERIFIED, not asserted impossible" -- that framing matches this finding. But the *operational* ruling in D-101/D-139 (docs/design-decisions.md:2308, "PXE is v4-first... IPv4 retained: MAAS PXE / provisioning") reads as a settled constraint. Fresh research finds no MAAS-side capability blocker for DHCPv6 commissioning at 3.7.2 -- the gap is "never tested here," not "cannot work." | +| 1b | UEFI HTTP boot over IPv6 | **FEASIBLE-WITH-WORK, but UNVERIFIED at the doc level** | UEFI's HTTP-Boot feature (distinct from PXE/TFTP boot) is a firmware-and-network-stack capability that has supported IPv6 since UEFI added HTTP Boot (Intel Ethernet UEFI drivers explicitly support IPv6 for "remote boot solutions using UEFI"). MAAS 3.x does support HTTP boot as a boot method (added in the MAAS 3.1/3.2 era for UEFI targets generally), and the region controller's boot-resource HTTP server is the same HTTP service the docs say is reachable "the same way on IPv4 and IPv6" for the Web UI/API. However, none of the current `canonical.com/maas/docs` pages fetched (about-networking, about-maas-networking, what-maas-can-do, commissioning-machines/3.7) explicitly document IPv6 for the HTTP-boot path specifically -- it is inferred from (a) UEFI firmware-level v6 support and (b) MAAS's HTTP service being dual-stack-capable elsewhere, not confirmed end-to-end for boot. The work: a live test (NIC + firmware known to support UEFI HTTPBoot IPv6, e.g. modern server NICs on the deployed hardware) is needed to convert this from "should work" to "verified." | Intel: "Discover If Intel Offers IPv6 Version of PXE" (intel.com/content/.../000007666) -- confirms UEFI drivers are IPv6-compatible for PXE/HTTP remote-boot, contrasted with legacy Boot Agent (row 1c); MAAS docs (region API/UI dual-stack claim) as corroborating but not boot-specific evidence | Partial -- prior ruling never distinguished HTTP-boot from legacy PXE; this is a finer-grained finding the repo hasn't captured either way. | +| 1c | Legacy BIOS PXE over IPv6 | **INFEASIBLE** | Not a MAAS limitation at all -- it's a firmware/spec limitation below MAAS. Legacy PXE (Intel Boot Agent, PXE spec 2.1) is defined entirely within the IPv4 protocol suite; Intel states explicitly it has "no plans to update Intel Boot Agent for IPv6." There is no ratified PXE-over-IPv6 equivalent in the legacy-BIOS boot ROM world -- IPv6 netboot requires UEFI firmware (which uses a different boot protocol, not classic PXE). If any deployed node's NIC/firmware only offers legacy BIOS PXE (not UEFI), that specific node is a hard IPv4-only case regardless of anything MAAS does. | Intel Support: "Discover If Intel Offers IPv6 Version of PXE" (intel.com, explicit "no plans... for IPv6"); corroborated by Cleverence "Legacy BIOS PXE Boot vs. UEFI" and Ubuntu Wiki `UEFI/SecureBoot/PXE-IPv6` (IPv6 netboot documented only under the UEFI heading) | No -- this one **confirms** a real constraint exists, but it's narrower than the repo's blanket "PXE is v4-first": it's specific to legacy-BIOS boot mode, not to MAAS or to UEFI nodes. If the fleet is UEFI-only (need to confirm against actual firmware mode in use), this blocker doesn't apply at all. | +| 2 | Rack controller <-> region controller RPC/websocket over IPv6 | **FEASIBLE-NOW** | No blocking work identified. The MAAS source tree treats IPv6 as a first-class RPC transport: `provisioningserver/rpc/clusterservice.py`'s `_build_rpc_info_urls()` explicitly resolves and **prefers AF_INET6 over AF_INET** (sorts resolved addresses so IPv6 entries are tried first) and constructs bracketed IPv6 RPC URLs (`[addr]:port`). `maasserver/rpc/regionservice.py` likewise handles both AF_INET and AF_INET6 for the ports MAAS advertises. The one real historical bug in this area -- LP#2020142, "commission fails if maas-url uses an IPv6" (a Python `socket.getaddrinfo()` v4-first resolution quirk breaking `::`-based MAAS_URLs) -- was **Fix Released in MAAS 3.5.0-beta1** (2024-03-05), which is *before* the deployed 3.7.2. Region API URLs with IPv6 require bracket notation (`http://[::1]:5240/MAAS/`), a syntax detail, not a capability gap. | GitHub canonical/maas `src/provisioningserver/rpc/clusterservice.py` and `src/maasserver/rpc/regionservice.py` (source-level AF_INET6 handling, IPv6-preference sort); Launchpad bug #2020142 (fix version 3.5.0-beta1); Canonical MAAS Discourse "IPv6 addressing" series (bracket-URL requirement) | **YES, materially.** This directly contradicts treating the MAAS control plane as PXE-coupled-to-v4-forever. Rack<->region RPC is dual-stack-capable and IPv6-preferring in the current codebase, with the one known blocking bug fixed two minor versions before what's deployed. This is the strongest FEASIBLE-NOW finding in this set and should be surfaced as a real decision point, not filed under the same "v4-first" umbrella as PXE. | +| 3 | MAAS region API/UI over IPv6 | **FEASIBLE-NOW** (minor, as scoped) | Documented directly: "You can access the Web UI and the MAAS CLI (that is, logging in to the API server) in the same way on both IPv4 and IPv6." No firmware/boot-ROM dependency here (unlike rows 1a-1c) -- this is a normal dual-stack HTTP service. Only caveat is the bracket-notation requirement for IPv6 literals in `MAAS_URL`/CLI targets. | Canonical MAAS Discourse "IPv6 addressing" series (2.8/3.0 CLI+UI docs, still architecturally current -- no code removed this capability between 2.8 and 3.7 in anything found) | No direct repo ruling found narrowly on region API/UI address family; consistent with the repo's general v6-primary posture for non-PXE/non-boot surfaces. | + +## Repo cross-check (read-only) -- ruling vs. measured test + +- `docs/design-decisions.md:2307-2308` (D-101, amended 2026-07-09): rules "PXE itself stays + v4-first regardless" and "IPv4 retained: MAAS PXE / provisioning (PXE is v4-first, unaffected + by the metal-admin amendment above)". This is a **judgement/ruling**, not a report of a + measured test -- no ping/dhclient/tcpdump-style evidence is cited alongside it in that section. +- `docs/design-decisions.md:2319`: rationale text frames the v4-retention on PXE and datastore + east-west as "explicitly justified rather than incidental" -- again policy language, not a test + result. +- `docs/design-decisions.md:8332-8333` (2026-08-10 IPv6-unlock design note, pass5): the repo's + own most recent language is more careful and matches this report's finding almost exactly -- + "metal-admin PXE stays v4 by ruling (v6-only MAAS commissioning is **UNVERIFIED, not asserted + impossible**)." This is the accurate framing; it is the *older* D-101/D-139 "PXE is v4-first" + language elsewhere in the doc that reads more absolute than the evidence supports. +- No grep hit anywhere in `docs/design-decisions.md` or the audit tree shows a MAAS-side + DHCPv6/commissioning/HTTP-boot test ever having been *run* against this deployment (3.7.2) -- + every reference found is policy language or the 2026-08-10 note flagging it as untested. + +## Bottom line for the orchestrator + +- Legacy-BIOS PXE-over-IPv6 is a genuine, vendor-confirmed INFEASIBLE (below MAAS, firmware + spec-level) -- but only for nodes actually booting in legacy BIOS mode, not UEFI. +- DHCPv6 commissioning and UEFI HTTP-boot-over-IPv6 are FEASIBLE-WITH-WORK: no MAAS-side + capability blocker found for 3.7.2, all historically-blocking bugs closed years/versions + before this deployment's version; the work is a live measured test on this repo's actual + hardware, which has not happened yet. +- Rack<->region RPC over IPv6 is the standout: FEASIBLE-NOW, source-level IPv6-preferring + behavior, last blocking bug fixed in 3.5.0-beta1 (two minors before 3.7.2). This is the + clearest disagreement with treating "MAAS control plane" as uniformly v4-first. diff --git a/docs/audit/container-elim-pass/pass6-w2-openstack-ipv6-feasibility.md b/docs/audit/container-elim-pass/pass6-w2-openstack-ipv6-feasibility.md new file mode 100644 index 0000000..7432bee --- /dev/null +++ b/docs/audit/container-elim-pass/pass6-w2-openstack-ipv6-feasibility.md @@ -0,0 +1,66 @@ +# Pass-6 W2 -- OpenStack/Neutron/Octavia IPv6 feasibility, FRESH assessment (independent of prior rulings) + +**Scope and method.** This is a feasibility-research pass, not a ruling. It asks, for three +named links, whether IPv6 is *technically capable* on the deployed stack -- **OpenStack Caracal +2024.1, OVN 24.03.2 / OVS 3.3.0, Neutron ML2/OVN, Octavia (amphora provider), OPNsense 26.7** -- +using real web research against upstream docs/specs/release notes, cited per finding. A prior +repo ruling that a link "is v4/dual" is NOT treated as evidence of infeasibility; where this +pass's finding disagrees with the prior ruling's *rationale*, that is called out explicitly. Web +access **was available** and used for all three links (WebSearch + WebFetch); no finding below +rests on unverified memory. + +**Repo cross-check performed first (read-only):** grepped `docs/design-decisions.md` for the +governing entries -- D-004 (v2 dual-stack matrix, unbuilt), D-020/D-057/D-060 (API VIP plane +history), D-101 (six-plane family ruling), D-125/D-144 (simulated-ISP uplink, terminated and +replaced by the flat topology 2026-08-10), D-139/D-141 (GUA carve + dual-stack-status-distinguished +IPAM rule), and `opentofu/modules/site-wan/main.tf` (the uplink's actual Terraform). Finding on +"tested vs ruled", stated once here and not repeated per link below: **none of the three links +has ever been live-tested over IPv6 in this repo.** Every existing entry is a RULING (D-101's +family matrix, D-139's GUA carve, D-141's reserved-until-capable status) or a plain absence +(`modules/site-wan/main.tf` has one IPv4-only `ips` block, no v6 `ips` entry, no test). The +D-139/D-141 "v4-active, v6-reserved" framing for provider-public and API VIPs is a **capability +judgement recorded in the apex**, not a measurement -- consistent with this pass's job being to +re-derive the judgement from upstream capability rather than accept it as settled. + +--- + +## Verdict table + +| Link | Verdict | Work / exact blocker | Capability citation | Disagrees with prior ruling? | +|---|---|---|---|---| +| 1. provider-public / external tenant connectivity over IPv6 | **FEASIBLE-NOW** (direct GUA routing; "floating IP" is not the right model for v6 at all) | None to reach basic external v6 reachability with manually-assigned GUA (already this repo's model, D-139 apex carve). Two known, non-blocking upstream gaps: ML2/OVN does not implement automatic IPv6 prefix delegation from an external PD router (Neutron gap; irrelevant here since GUA is manually carved, not PD-sourced); OVN centralizes IPv6 north-south routing at the gateway chassis rather than fully distributing it (a scale/perf characteristic, not a functional block) | Neutron admin IPv6 doc, **Caracal 2024.1 branch**: "Unlike IPv4 there is no current embedded support for floating IPs with IPv6" and "Neutron project networks that are assigned Global Unicast Address (GUA) prefixes and addresses don't require NAT on the neutron router external gateway port to access the outside world" (docs.openstack.org/neutron/2024.1/admin/config-ipv6.html, fetched and confirmed present on the 2024.1 branch itself); ML2/OVN gaps doc, **same 2024.1 branch**: "Currently ML2/OVN doesn't implement IPv6 prefix delegation" and "The NDP proxy functionality for IPv6 addresses is not supported by OVN" (docs.openstack.org/neutron/2024.1/ovn/gaps.html); OVN IPv6 DVR RFE (LP #1998609 / neutron-specs 2023.1 ovn-ipv6-dvr) confirms v6 north-south stays centralized pending that spec | **YES.** The repo (`pass5-w2-metal-data-v6.md` citing D-139 ruling A) frames provider-public's v4 half as forced partly by "Tenant FIPs need IPv4" (design-decisions.md:88). That is true only for v4-addressed (RFC1918) tenant networks, which need NAT to reach anything. A v6 GUA-addressed tenant network needs **no floating IP at all** -- direct GUA routing is not a fallback or a lesser path, it is the designed v6 model and arguably simpler than v4's FIP/NAT machinery. The real, valid reason v4 stays is that the actual upstream Internet/office network this cloud federates with is "a still-substantially-v4 internet" (same D-139 citation) plus client/API back-compat -- not any Neutron/OVN capability gap on the v6 side. | +| 2. External API VIPs (hacluster/keepalived) + Octavia LB VIPs over IPv6 | **FEASIBLE-NOW at the Neutron/Octavia layer; the hacluster piece is FEASIBLE-WITH-WORK because of a named CHARM defect, not a corosync/pacemaker capability gap** | **Octavia:** none -- `vip_subnet_id`/`vip_network_id` accept an IPv6 subnet natively; Caracal's own cookbook flags the ONE caveat and it is the same non-model as link 1 (floating IPs), not a VIP-creation block. **hacluster (the API-VIP HA manager):** this repo's own measured record (`docs/charm-ip-family-compatibility.md`) already isolated the exact defect -- the `hacluster` charm (2.4/stable rev 166) hard-writes `ip_version: ipv4` into the rendered `templates/corosync.conf`, i.e. the CHARM's Jinja template, not corosync/pacemaker's own protocol support (corosync's `totem`/`ring` config and `ocf:heartbeat:IPv6addr` both support v6 natively upstream). A fix is `LP #2111852`, **Fix Committed but NOT RELEASED** as of this repo's 2026-08-03 measurement. So the work is "wait for / backport the charm release", not any Neutron/Octavia/HA-software rework | Octavia Caracal cookbook, **2024.1 branch**: `--vip-subnet-id` "is appropriate for operators with provider networks that are not compatible with Neutron floating-ip functionality, such as IPv6 networks," with the sole caveat "this is not possible to do with IPv6 load balancers as floating IPs do not work with IPv6" (docs.openstack.org/octavia/2024.1/user/guides/basic-cookbook.html, fetched directly -- supersedes an earlier draft of this row that cited an unfetched feature-matrix quote); this repo's own measured record: `docs/charm-ip-family-compatibility.md` hacluster row -- "Hard-writes `ip_version: ipv4` in `templates/corosync.conf`, 2.4/stable rev 166 ... LP #2111852 `Fix Committed` but NOT RELEASED"; keepalived IPv6 VRRPv3 background (not this repo's HA path, cited for the general capability claim only): raymii.org "Adding IPv6 to a keepalived and haproxy cluster" | **YES, partially, and more precisely than first drafted.** D-141's framing groups hacluster's v6 failure together with the juju-container-addressing blocker as if both were "the v6 stack isn't ready." This pass's fresh read, now anchored to the repo's own hacluster row rather than an outside claim: Octavia's v6 VIP path needs no work at all, and hacluster's v6 failure is a **specific, already-diagnosed, already-fixed-upstream (unreleased) charm template bug** -- not evidence that HA VIP clustering over v6 is hard in general. Both the juju container-addressing layer (LP #1723240) and the hacluster charm-template bug (LP #2111852) sit above the Neutron/Octavia capability layer this task scopes; neither is a platform-capability gap. | +| 3. The simulated-ISP uplink hop over IPv6/dual | **FEASIBLE-WITH-WORK** -- name the work | OPNsense 26.7 itself needs zero work (native, mature dual-stack WAN: static, DHCPv6, DHCPv6-PD, SLAAC, track-interface delegation to LAN -- all documented, current). The actual gap is on THIS repo's side of the hop: `opentofu/modules/site-wan/main.tf` builds the libvirt "ISP" network with exactly one IPv4-only `ips` block and `forward.mode = "nat"`; there is no v6 `ips` entry at all. Work needed: (a) add a second `ips` block with `family = "ipv6"` carrying a GUA-or-ULA-style "ISP" prefix; (b) per libvirt's own IPv6 handling, this should be **routed, not NAT'd** -- libvirt only NATs v6 if `` is explicitly set (requires libvirt >= 6.5.0) and even then "if a network has any IPv6 addresses defined, the IPv6 traffic will be forwarded using plain routing" by default, which is actually the MORE realistic simulated-ISP-handoff model (real ISPs route a delegated v6 prefix, they don't NAT66 it) and matches this repo's own stated posture ("NAT66 is discouraged... native is both correct and feasible", design-decisions.md:2313/2750); (c) point OPNsense's WAN at DHCPv6/SLAAC/static against that new v6 block. None of this is a capability blocker -- it is unbuilt configuration, on a module that the D-144 flatten (2026-08-10) is already re-homing (`modules/wan-bridge` retired, `modules/site-wan`'s NAT pattern carries forward into the new `modules/dc-site`) | OPNsense official IPv6 docs (docs.opnsense.org/manual/ipv6.html): native DHCPv6, PD, SLAAC, static, track-interface; libvirt Network XML format (libvirt.org/formatnetwork.html) and the 2020 libvir-list patch thread "network: support NAT with IPv6" -- IPv6 NAT is opt-in (``, libvirt >=6.5.0) and undefined/absent v6 NAT falls back to plain routing; repo file read directly: `opentofu/modules/site-wan/main.tf:14-36` (one `ips` block, IPv4 CIDR only, no v6) | **YES, partially.** D-125 (now D-144-terminated) ruled the uplink v4-only by omission -- it was never argued infeasible, it simply was never scoped for v6 (the module has no v6 code path at all, and no design-decisions entry states a technical reason v6 can't be added). The fresh finding is that OPNsense-side feasibility is total and libvirt-side feasibility is a small, well-documented addition -- there is no infeasibility here, only unbuilt scope, and the "right" v6 model (routed, not NAT66) also better matches the repo's own already-adopted v6 philosophy for tenant GUA than the analogous v4 NAT pattern does. | + +--- + +## The v6-FIP-vs-direct-GUA finding (repeated once here for emphasis) + +IPv6 in Neutron/OVN was never designed to need a "floating IP" the way IPv4 does. IPv4 floating +IPs exist because tenant networks are RFC1918-private and need 1:1 DNAT/SNAT to reach a public +network. IPv6 GUA-addressed tenant networks are *already* globally routable addresses -- the +router's external gateway port routes them without NAT, per Neutron's own admin docs. So asking +"can IPv6 do floating IPs" is asking the wrong question; the right question -- "can IPv6 be +directly routed on the provider network" -- is answered FEASIBLE-NOW, and is in fact the +*simpler*, more-native v6 path, not a workaround for a missing v4-equivalent feature. + +## Biggest disagreement with prior rulings + +Link 2's framing is the largest gap: the repo's D-141 (and the pass5 documents built on it) +correctly identify both actual blockers -- juju container addressing (LP #1723240) and the +hacluster charm's hard-coded `ip_version: ipv4` template (LP #2111852, fix committed but not +released) -- but the surrounding narrative groups both in with "the v6 stack isn't ready yet." +This pass's fresh read, checked directly against `docs/charm-ip-family-compatibility.md` rather +than assumed: Octavia's v6 VIP path needs no work at all, and the hacluster failure is a single, +already-diagnosed, already-fixed-upstream charm bug, not a corosync/pacemaker protocol +limitation. Both named blockers sit one layer above Neutron/Octavia, not inside them. + +## Web research + +Successful for all three links -- WebSearch + WebFetch against upstream Neutron/Octavia/OVN docs, +GitHub release notes, libvirt.org, and docs.opnsense.org. No finding here rests on unverified +memory; every capability claim above has an inline citation. + +## Durable-doc path + +`docs/audit/container-elim-pass/pass6-w2-openstack-ipv6-feasibility.md` (this file). diff --git a/docs/audit/container-elim-pass/pass6-w3-juju-charm-ceph-feasibility.md b/docs/audit/container-elim-pass/pass6-w3-juju-charm-ceph-feasibility.md new file mode 100644 index 0000000..8e3bdf5 --- /dev/null +++ b/docs/audit/container-elim-pass/pass6-w3-juju-charm-ceph-feasibility.md @@ -0,0 +1,97 @@ +# Pass 6 (W3) -- FRESH IPv6 feasibility check: juju/charm + Ceph links (independent of prior ruling) + +**Author:** W3 worker. **Scope:** a feasibility assessment, from scratch, of the three +named links (juju container v6 addressing / LP #1723240, OpenStack-charms IPv6 binding +in the 2024.1 channel, Ceph cross-DC replication over IPv6) -- run WITHOUT treating +D-141 or `docs/charm-ip-family-compatibility.md` as settled truth. Both are cross-checked +(read, not trusted) and re-derived from live web sources against the DEPLOYED versions: +**Juju 3.6.27**, OpenStack charms **2024.1**, **Ceph Reef-era (Caracal)**, MAAS 3.7. Web +research (WebSearch + WebFetch) SUCCEEDED for all three links; every verdict below is +cited to a fetched or searched primary source, not general knowledge. READ-ONLY -- +nothing built, nothing ruled, no D-number touched. + +**Repo cross-check performed first (as required):** `docs/design-decisions.md` D-141 +(`:8003-8055`) and `docs/charm-ip-family-compatibility.md` (full) were read before any +web research, to know what the repo currently claims and where a fresh finding would +disagree with it. + +--- + +## Verdict table + +| # | Link | Verdict | Work / blocker | Citation | Disagrees with prior ruling? | +|---|---|---|---|---|---| +| 1 | Juju LP #1723240 (container v6 addressing) | **INFEASIBLE (today, as-is)** | Juju's MAAS/LXD provider assigns a container ONE address per NIC, family-blind, from the requested space; a dual-stack space still yields v4-only. No juju-native fix exists. | bugs.launchpad.net/juju/+bug/1723240, fetched live 2026-08-10: **Status = Triaged, Importance = Low, Milestone = none**, reported 2017, importance auto-downgraded 2022 for inactivity, "no fix has been committed or released as of the latest update." Confirmed NOT fixed in Juju 3.6 LTS (deployed: 3.6.27) or 4.0. | **NO -- CONFIRMS the repo's premise, does not overturn it.** D-141's characterization is accurate as of today, not stale. | +| 2 | OpenStack-charms IPv6 binding (2024.1 channel) | **FEASIBLE-WITH-WORK** (named work below; NOT a clean yes) | Most core API charms (charms.openstack / charmhelpers family: keystone, glance, nova-cloud-controller, cinder, barbican, designate, magnum, neutron-api, octavia, placement, openstack-dashboard) bind v6 correctly and are blocked ONLY by link 1. Three named per-charm defects still gate the rest even if link 1 were solved: **hacluster** (LP #2111852) is `Fix Committed` (merged 2025-06-24) but **not yet released** to any published channel -- 2024.1/stable still ships the broken `ip_version: ipv4` corosync template; **ceph-osd** (LP #2109798) is still `New`/unfixed -- `inet_aton`+DNS-A-only lookup crashes the mon-relation hook on v6, workaround is to force v4 CIDRs (defeats the purpose); **mysql-innodb-cluster/mysql-router**'s bare-URI defect has no independently located LP in this pass's search (not found on bugs.launchpad.net/charm-mysql-innodb-cluster) -- carried forward as repo-asserted, not web-confirmed this pass. | LP #2111852 (bugs.launchpad.net/charm-hacluster) fetched live: `Fix Committed`, trunk commit `41a75ed`, no 2024.1-channel release indicated. LP #2109798 (bugs.launchpad.net/charm-ceph-osd) fetched live: `New`, unassigned, `inet_aton`/DNS-A-only failure confirmed as described. LP #2061836 (bugs.launchpad.net/charm-ceph-mon) fetched live: `Fix Committed`, merged 2024-06-14, "allow static ipv6 addresses & binding check" -- ceph-mon verdict of `Partial` also confirmed current, not stale. keystone `prefer-ipv6` option confirmed live on charmhub.io/keystone/configurations and github.com/openstack/charm-keystone config.yaml. | **PARTIAL, ONE MAJOR DISAGREEMENT (see below on the OVN row).** The hacluster/ceph-osd/ceph-mon LP statuses all match the repo's existing table exactly -- no staleness there. | +| 3 | Ceph cross-DC replication over IPv6 (rbd-mirror / radosgw multisite) | **FEASIBLE-WITH-WORK** (named work below) | Ceph daemons bind ONE family per daemon via `ms_bind_ipv4`/`ms_bind_ipv6` (not simultaneous dual-stack per-socket); Reef fixed a bug where `ms_bind_ipv4` was not auto-disabled when `ms_bind_ipv6` was set (tracker #61714-area). No rbd-mirror- or radosgw-multisite-specific IPv6 blocker was found in this pass's search of tracker.ceph.com or bugzilla -- the general dual-stack caveat is that a mixed v4/v6 cluster is awkward (some daemons v4, some v6, no single daemon serving both), but a **cluster committed to pure v6** (`ms_bind_ipv4=false`, `ms_bind_ipv6=true` cluster-wide, both DCs) is Ceph-documented as supported, and rbd-mirror/radosgw are ordinary msgr2 clients riding on that. Named work: commit the WHOLE cluster (both DC0 and DC1 Ceph clusters, not just the replication link) to a single address family -- this is a bigger lift than "just make the replication leg v6", since Ceph does not support per-daemon-pair dual family cleanly. | docs.ceph.com/en/reef/rados/configuration/network-config-ref/ fetched live: confirms `ms_bind_ipv4`(default true)/`ms_bind_ipv6`(default false) as the binding controls, "instead of" framing implies exclusive-family-per-daemon, not simultaneous dual-stack. Ceph Reef release notes / PR search: a fix "disable ms_bind_ipv4 if ms_bind_ipv6 will be enabled" landed in Reef. GitHub ceph-ansible issue #5385 and Proxmox/RH community threads corroborate: "Ceph is not able to run dual stack" per-daemon; a uniform single-family cluster is the supported shape. No rbd-mirror- or radosgw-multisite-specific v6 defect found. | **NO strong disagreement -- this fills a gap the repo's own table leaves open** (`ceph-rbd-mirror` and `ceph-radosgw` are both marked **NOT ASSESSED** in `docs/charm-ip-family-compatibility.md`; this pass gives them a first verdict: feasible, but only under a whole-cluster v6 commitment, not a v4-cluster-with-v6-replication-leg design). | + +--- + +## THE LOUD DISAGREEMENT: the repo's own OVN "No (data plane)" verdict is STALE + +`docs/charm-ip-family-compatibility.md` currently reads (rows `ovn-central`/`ovn-chassis`, +IPv6 column): **"No (data plane)"**, evidenced by *"OVN documents the encap column as +'The IPv4 address of the encapsulation tunnel endpoint' (`ovn-sb.xml:565`); every +`ovn-encap-ip` in OVN 24.03's test suite is IPv4"* -- and flags itself *"Reported from +capture, not independently re-verified."* + +**That verdict is now contradicted by this SAME repo's own live-measured evidence, one +day older than the table's last edit.** `docs/audit/geneve-over-v6-rootcause-20260808.md` +records (2026-08-09, section "LIVE-CONFIRMED"): a real VM-to-VM ping across two metal +compute chassis over geneve-over-IPv6, **8/8 received, 0% loss, rtt ~3ms**, on the exact +deployed stack (OVS 3.3.0 / OVN 24.03.2 / kernel 5.15.0-186). The root cause of the +EARLIER "IPv4-only" appearance was NOT an OVN/kernel capability limit -- it was the +`ovn-chassis` 24.03 charm delivering the v6 `ovn-encap-ip` to OVS **bracketed** +(`"[2602:...::120]"`), which OVS's geneve `remote_ip` parser rejects (`ofport -1`, +`"bad geneve 'remote_ip'"`). Delivered unbracketed, the tunnel instantiates and forwards +real traffic. External corroboration found this pass: geneve-over-IPv6 is a +long-supported OVS capability (v6 tunnel endpoints since OVS 2.6/2016) and is +production-exercised elsewhere (e.g. ovn-kubernetes single-stack v6); the OVN +`ovn-sb.xml` doc line and IPv4-only test-suite bias the table cites are stale +documentation/test-coverage artifacts, not a real capability denial -- exactly the +"NOT independently re-verified" caveat the table itself flagged, now resolved by a live +test. + +**Net effect on this pass's charm-IPv6 verdict:** the OVN data-plane blocker should be +treated as **NOT a capability blocker** (contrary to the current table row), but the +`ovn-chassis` charm's bracket-format bug (no fixed charm revision confirmed by this +pass's LP search -- LP #1968355 was checked and is a DIFFERENT issue, "Won't Fix") IS +still a real, open, un-pinned charm defect requiring a workaround (persistent +post-deploy `ovs-vsctl set open_vswitch . external_ids:ovn-encap-ip=` +override) until a fixed revision ships. + +**Recommendation (finding, not executed):** `docs/charm-ip-family-compatibility.md`'s +`ovn-central`/`ovn-chassis` row should be updated from **"No (data plane)"** to +**"Partial"** (data plane IS v6-capable; the live blocker is the charm's bracket +delivery bug, not OVN/OVS), citing `geneve-over-v6-rootcause-20260808.md` in place of +the stale man-page/test-suite citation. This is a table-maintenance finding for the +maintenance rule at the bottom of that doc ("update a row only from a MEASURED +source"); it is NOT executed here (read-only pass). + +--- + +## Bottom line + +- **Link 1 (LP #1723240):** confirmed still Triaged/Low/unfixed in Juju 3.6.27 -- the + repo's characterization holds, no staleness. +- **Link 2 (charm IPv6 binding):** mostly holds (hacluster/ceph-osd/ceph-mon LP statuses + all match), EXCEPT the OVN data-plane row, which is stale and should read "Partial," + not "No" -- the repo's own audit already proved this a day before the compatibility + table's current text, and the table was never reconciled against it. +- **Link 3 (Ceph v6 replication):** feasible under a whole-cluster single-family (v6) + commitment; fills a previously NOT-ASSESSED gap, does not contradict anything ruled. +- **Overall:** even correcting the OVN row, the cloud-wide container-VIP-over-v6 picture + is unchanged in its bottom line -- link 1 (juju addressing) is still the universal + blocker for EVERY container-hosted VIP regardless of charm, and it is still unfixed. + D-141's operative conclusion (v4-active/v6-reserved for container VIPs) stands on + fresh evidence. The disagreement that DOES surface (OVN data plane) affects the + compatibility table's per-charm accuracy, not D-141's VIP ruling, since D-141 was never + gated on the OVN data-plane row in the first place (D-141 gates on LP #1723240 + + per-charm VIP defects; OVN geneve is a separate, already-D-139-owned surface). + +## Maintenance note + +Per `docs/charm-ip-family-compatibility.md`'s own maintenance rule, its OVN row should be +updated by whoever next touches that doc, citing this pass and +`geneve-over-v6-rootcause-20260808.md`. Not done here -- this pass is READ-ONLY per +tasking (a FRESH assessment to surface findings, not a doc edit). diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 1f1c5c1..df6f924 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -8329,9 +8329,25 @@ they govern, and keep the (a)/SEC-010-successor controls' nftables `inet` (v6-native) shape; (iii) DEC-24 must resolve the cross-DC replication carrier AND its still-open D-139 v6-route together (same leg). NOT unlocked by the collapse (different layers, do NOT credit D-144): D-139's plane family - matrix, the geneve-over-v6 fix, and the LXD-container/juju LP#1723240 gap (D-141). metal-admin PXE - stays v4 by ruling (v6-only MAAS commissioning is UNVERIFIED, not asserted impossible). Full note: + matrix, the geneve-over-v6 fix, and the LXD-container/juju LP#1723240 gap (D-141). Full note: `docs/audit/container-elim-pass/ipv6-unlock-design-note-20260810.md`. +- **IPv6 FEASIBILITY re-check (added 2026-08-10, pass6 -- CORRECTS the pass5 bullet above, which relied + on prior rulings instead of researching feasibility).** Fresh ruling-independent research (upstream + docs/source/bug-trackers, deployed versions) finds IPv6 substantially MORE feasible than D-101/D-139 + held: metal-admin **IPv6 commissioning is FEASIBLE-WITH-WORK on UEFI nodes** (DHCPv6 no MAAS blocker + @3.7.2; only LEGACY-BIOS PXE is a firmware blocker, avoidable via UEFI), MAAS rack<->region v6 + **FEASIBLE-NOW** (source prefers AF_INET6) -- so D-101's absolute "PXE is v4-first" is an overstated + judgement, not a capability limit. provider-public v6 **FEASIBLE-NOW via direct GUA routing** (v6 uses + no floating IP; "dual-stack because the internet is v4" conflated the provider-network FAMILY -- v6- + capable now -- with EXTERNAL v6 internet reachability, the only true v4 need). Octavia LB VIP v6 + FEASIBLE-NOW; API-VIP/hacluster FEASIBLE-WITH-WORK (one charm bug LP#2111852); uplink edge v6 + FEASIBLE-WITH-WORK (OPNsense ready; routed not NAT66). CONFIRMED still-hard: juju **LP#1723240** + (container API-charm VIPs -- live-verified still open in 3.6.27; D-141 HOLDS) + external v6 internet + (deferred). NEW DEC-24 constraint: Ceph binds one family per daemon -> v6 cross-DC replication needs + the WHOLE cluster single-family. These CHALLENGE prior framings (D-101 PXE, D-139 provider-public) and + are surfaced for operator RECONSIDERATION -- NOT ruled here (GA-R5). Two DOCFIX candidates flagged + (stale OVN "No" row in `charm-ip-family-compatibility.md` vs the repo's own geneve proof; soften + D-101's "PXE v4-first"). Full note: `docs/audit/container-elim-pass/pass6-ipv6-feasibility-note-20260810.md`. **A1 TEST (GA-R3):** admitted -- SUPERSEDES D-123's core Model-B ruling; a Roosevelt / pre-Roosevelt build session greps this before laying out substrate shape; and the layered module workflow (L0-L5,