diff --git a/docs/changelog-20260714-office1-lan-ipv6-executed.md b/docs/changelog-20260714-office1-lan-ipv6-executed.md new file mode 100644 index 0000000..da66182 --- /dev/null +++ b/docs/changelog-20260714-office1-lan-ipv6-executed.md @@ -0,0 +1,97 @@ +# 2026-07-14 -- Office1 LAN IPv6 is LIVE (Stage 2 close-out C1: OPEN -> DONE) + +The last build item on Office1. Executed against the live edge and proven on the wire, not by +self-report. Closes the item pinned since 2026-07-13 (D-115 question 3). + +## Operator ruling that shaped it (2026-07-14) + +The session opened with a proposal to build an **IPv4-to-IPv6 NAT** (a second OPNsense, or nginx) +to *simulate* the v6 forwarding the lab does not provide. **Not built, and the proposal was +withdrawn** in favour of: *"configure IPv6 internally as much as possible and we will configure the +IPv6 edge after deployment has been completed."* + +Why that was the right call, recorded so it is not re-proposed: + +- **D-101 already rules it out:** *"native GUA routing, no NAT66 -- NAT66 is discouraged and we own + the edge."* Translation is a REJECTED approach here, not an open one. +- **D-115's amendment already ruled on this exact symptom:** v6 not routing past the lab boundary + is *"a network issue OUTSIDE this cloud and outside this repo -- it needs the lab/upstream + operator, not a change here."* +- **MINIMIZE DELTA TO ROOSEVELT:** Roosevelt has a real ISP carrying real v6. A translation box is + precisely the component Roosevelt will NOT have, so a NAT shim is the *least* faithful possible + simulation -- it would make the runbooks teach a topology that must later be deleted. +- **nginx was not a peer option:** it is an L7 HTTP(S) reverse proxy, not NAT. It proves nothing + about RA, ICMPv6, Ceph-over-v6 or geneve-over-v6 -- i.e. nothing about D-101's family matrix. + +**Re-measured 2026-07-14, confirming D-115:** vcloud holds a GUA and a v6 default route via RA, the +v6 gateway is reachable, but v6 internet is **100% loss** while v4 from the same host is fine. The +lab's upstream advertises RA and a prefix and does not route v6 out. Unchanged, and not ours to fix. + +## What was done + + scripts/opnsense-set-interface-v6.sh --commit 10.10.0.1 lan 2602:f3e2:f01:100::1 64 + POST /api/radvd/settings/addEntry {"entries":{"enabled":"1","interface":"lan","mode":"unmanaged"}} + POST /api/radvd/service/reconfigure + +Address `2602:f3e2:f01:100::1/64` is the D-115 carve value, not an invented one. + +**The address half is NOT API-doable** -- see the D-113 amendment. The RA half IS API-native. Both +halves are now exercised for real, which is what makes them reusable verbatim for the DC0/DC1 edges +at Stage 3. + +## `mode=unmanaged`, deliberately NOT the model's default + +The radvd model defaults to `mode=stateless`, which sets the RA **Other-config flag** -- telling +clients to fetch DNS and other options from a **DHCPv6 server**. MEASURED on the edge: Kea DHCPv6 is +`enabled=0` and no DHCPv6 daemon is running. Accepting the default would have advertised a DHCPv6 +server **that does not exist** -- a quiet misconfiguration that would look fine and produce clients +that hang waiting on it. `unmanaged` (M=0, O=0) is the honest configuration for SLAAC-only. + +Field names, the request key (`entries`, from `addBase('entries','entries')`), the legal `mode` +values, and the apply endpoint were all READ OFF THE EDGE (the model XML and the controller), not +recalled. Nothing here was guessed. + +## PROVEN ON THE WIRE (not by `{"result":"saved"}`) + + edge ifconfig vtnet0 -> inet6 2602:f3e2:f01:100::1 prefixlen 64 [the KERNEL, not the config] + edge radvd running -> prefix 2602:f3e2:f01:100::/64, AdvAutonomous on, AdvOnLink on + RDNSS 2602:f3e2:f01:100::1, no M/O flags [= unmanaged took effect] + voffice1 -> inet6 2602:f3e2:f01:100:5054:ff:fe6a:87e5/64 [SLAAC] + + 2602:f3e2:f01:100::/64 proto ra [RA-learned route] + edge -> voffice1 GUA -> 3 packets, 0.0% loss, 0.838ms avg [TRAFFIC PASSES] + edge ndp -an -> neighbour 52:54:00:6a:87:e5 [matches voffice1's Kea + lease hwaddr -- it IS voffice1] + +**No collateral damage** (`configctl interface reconfigure lan` re-applies the LAN that all of +Office1 sits behind): v4 `10.10.0.1` intact, Kea DHCPv4 still running and listening on udp/67, +edge + `voffice1` + the routed compose net all still reachable. The interface-count assertion held +(3 interfaces, unchanged). + +## A trap worth recording: `accept_ra=0` does NOT mean "RAs ignored" on Ubuntu + +`voffice1` has kernel `net.ipv6.conf.all.forwarding=1` (it runs LXD) and `accept_ra=0` on `enp1s0`. +Classic Linux behaviour says a forwarding host ignores RAs unless `accept_ra=2` -- so this looked +like it could not possibly have autoconfigured. **It did.** netplan/systemd-networkd processes RAs +in **USERSPACE**, independent of the kernel sysctl. Do not diagnose "no SLAAC" from the sysctl alone +on an Ubuntu box; read `ip -6 addr` and look for a `proto ra` route. + +## What this does and does NOT prove + +**Proves** (most of D-101's family matrix): v6 addressing, RA/SLAAC, on-link routing, and the edge +speaking v6 on the LAN. This is the mechanism DC0/DC1 will reuse. + +**Does NOT prove, and cannot in VR1:** internet GUA reachability. The WAN v6 leg is DEFERRED per +D-115 -- it would be a path to nowhere and untestable. Per the 2026-07-14 operator ruling, the +**IPv6 edge is configured AFTER deployment completes.** Do not read this green result as "v6 works +end to end." + +## Revert + + # RA + bash scripts/opnsense-api.sh POST /api/radvd/settings/delEntry/627e3f12-8150-4909-9910-a275f2a0a6fc + bash scripts/opnsense-api.sh POST /api/radvd/service/reconfigure + # address (clear the two leaves, then reconfigure) + bash scripts/opnsense-set-interface-v6.sh --commit 10.10.0.1 lan :: 64 # or clear via the GUI + +(`OPNSENSE_API_HOST=10.10.0.1` for the API calls.) Reverting is not expected to be needed: nothing +depends on the v6 leg yet, and the v4 LAN is untouched by it. diff --git a/docs/dc-dc-deployment-workflow.md b/docs/dc-dc-deployment-workflow.md index 168fa3a..9dc91d8 100644 --- a/docs/dc-dc-deployment-workflow.md +++ b/docs/dc-dc-deployment-workflow.md @@ -132,7 +132,7 @@ | # | Item | State | |---|---|---| -| C1 | Office1 LAN IPv6 deployed (`2602:f3e2:f01:100::1/64` + RA on the edge LAN, via the D-113(a2) REST API -- NEVER a config.xml push). WAN v6 leg DEFERRED: v6 does not egress the lab. | **OPEN** (pinned, D-115 q3) | +| C1 | Office1 LAN IPv6 deployed (`2602:f3e2:f01:100::1/64` + RA on the edge LAN). WAN v6 leg DEFERRED: v6 does not egress the lab. | **DONE 2026-07-14** -- live and PROVEN ON THE WIRE: address on the kernel, radvd advertising the prefix (`mode=unmanaged`), `voffice1` autoconfigured `...:5054:ff:fe6a:87e5` by SLAAC, edge->voffice1 GUA ping 0.0% loss. NOTE the D-113 AMENDMENT: the ADDRESS half is **not** REST-API-doable (`scripts/opnsense-set-interface-v6.sh`); only the RA half is. `docs/changelog-20260714-office1-lan-ipv6-executed.md` | | C2 | The validated carve fed UPSTREAM to `netbox.baldurkeep.com` -- the gate says *authoritative*, and the sandbox is not it. Operator-gated write. | **OPEN** | | C3 | The `$DC` shell-selector collision resolved -- `lib-net.sh`/`lib-hosts.sh` `dc0` means **VR0's** DC0 while the D-117 apex `dc0` means **VR1's**. D-117 Option B is only PARTIALLY executed. Blocks Stage 3 (see gap #19). | **OPEN** | | C4 | The NetBox sandbox loop documented here and in `docs/dc-dc-netbox-buildout-scope.md`. | **OPEN** | diff --git a/docs/session-ledger.md b/docs/session-ledger.md index a545050..d5b2e53 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -113,7 +113,25 @@ the last build item on Office1. **STAGE 2 IS IN CLOSE-OUT, NOT BUILD. It is also the BRANCH MERGE POINT** (see PINNED). The close-out checklist C1..C5 is in `docs/dc-dc-deployment-workflow.md`; do not re-derive it here. -1a. **PINNED -- NEXT AFTER NETBOX: deploy Office1's LAN IPv6** (operator ruling 2026-07-13, D-115 +1a. **DONE 2026-07-14 -- Office1's LAN IPv6 is LIVE and PROVEN ON THE WIRE. C1 is CLOSED.** + Address `2602:f3e2:f01:100::1/64` on the edge LAN (kernel-verified) + radvd advertising the prefix + with `mode=unmanaged`; `voffice1` autoconfigured `2602:f3e2:f01:100:5054:ff:fe6a:87e5` by SLAAC and + edge->voffice1 GUA ping is 0.0% loss. No collateral damage (v4, Kea DHCPv4, compose-net route all + intact). See `docs/changelog-20260714-office1-lan-ipv6-executed.md`. + **OPERATOR RULING 2026-07-14 -- DO NOT RE-PROPOSE A v6 NAT.** A v4-to-v6 NAT (second OPNsense, or + nginx) was proposed to *simulate* the v6 the lab will not forward, and was WITHDRAWN in favour of + *"configure IPv6 internally as much as possible; configure the IPv6 edge AFTER deployment + completes."* It contradicts **D-101** (*"native GUA routing, no NAT66"*) and **D-115**'s amendment + (upstream's problem, not ours), and a translation box is the one component Roosevelt will NOT have + -- maximum delta, zero proof value. nginx is an L7 proxy, not NAT, and proves nothing about + RA/ICMPv6/Ceph-over-v6. + **TRAP recorded:** `voffice1` has `forwarding=1` + `accept_ra=0` yet STILL autoconfigured -- + netplan/systemd-networkd handles RA in USERSPACE, independent of the kernel sysctl. Never diagnose + "no SLAAC" from the sysctl alone on Ubuntu; read `ip -6 addr` + look for a `proto ra` route. + **The WAN v6 leg remains DEFERRED** -- v6 does not egress the lab (re-measured 2026-07-14: v6 + internet 100% loss, v4 fine). Internet GUA reachability is UNPROVABLE in VR1. + +1a-historical. **(superseded -- the original pinned item)** deploy Office1's LAN IPv6 (operator ruling 2026-07-13, D-115 question 3: "register AND deploy dual-stack on Office1 now", sequenced AFTER the NetBox deploy). Add `2602:f3e2:f01:100::1/64` to the OPNsense edge LAN + RA, and let `voffice1` take its address. **CORRECTED 2026-07-14 (D-113 AMENDMENT -- the old instruction here was WRONG):** the ADDRESS half