diff --git a/docs/changelog-20260711-ula-gen-command-fix.md b/docs/changelog-20260711-ula-gen-command-fix.md index 6d79867..59cc8b8 100644 --- a/docs/changelog-20260711-ula-gen-command-fix.md +++ b/docs/changelog-20260711-ula-gen-command-fix.md @@ -15,6 +15,13 @@ `fd50:840e:74e2::/48` (from `docs/dc-dc-netbox-buildout-scope.md` 4c); the command is now framed as the reference method, not a VR1 re-generation step. +> **CORRECTION (2026-07-13, D-118).** The word "ratified" above was **FALSE** when +> written. `fd50:840e:74e2::/48` was a *recommendation* in a scope doc (gap G3); +> **no D-number assigned it**, and `grep ORG_ULA docs/design-decisions.md` returned +> nothing. The value was treated as settled for two days on the strength of this +> sentence alone. It is NOW genuinely ratified, as **D-118**. Recorded because the +> failure mode is reusable: a document asserting authority it does not have. + ## Verification - Ran the corrected command: `fdc0:4468:d6bf::/48` -> parses, /48, in `fc00::/7`. - `bash scripts/repo-lint.sh` -> 0 fail. diff --git a/docs/dc-dc-netbox-buildout-scope.md b/docs/dc-dc-netbox-buildout-scope.md index d766147..4a5594c 100644 --- a/docs/dc-dc-netbox-buildout-scope.md +++ b/docs/dc-dc-netbox-buildout-scope.md @@ -113,9 +113,9 @@ literals. Recommended literals (pending G5 naming + section 5 ratification): ``` # DC1 (v4 inherited; only v6 top-level blocks required) -ORG_ULA_48= DC_GUA_PREFIX=2602:f3e2:f02::/48 --dc dc0 # D-117: VR1 DC0 +ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f02::/48 --dc dc0 # D-117: VR1 DC0 # DC2 (distinct v4 supernet ALSO required -- >= /19; tool re-packs 6 sequential /22s, see 5.3) -ORG_ULA_48= DC_GUA_PREFIX=2602:f3e2:f03::/48 DC1_V4_SUPERNET=10.12.64.0/19 --dc dc1 # D-117: VR1 DC1 +ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f03::/48 DC1_V4_SUPERNET=10.12.64.0/19 --dc dc1 # D-117: VR1 DC1 ``` Run `--verify-only` first. The tool creates the six-plane v4 + the D-101 family-matrix v6 legs. @@ -184,7 +184,7 @@ | Literal | Status | Recommendation | |---|---|---| | `ORG_ULA_48` | unassigned | `fd50:840e:74e2::/48` (ratify) | -| `ORG_ULA_48` | **UNASSIGNED -- BLOCKS the DC import** | `fd50:840e:74e2::/48` recommended below, but **NOT RATIFIED**. No ADOPTED decision assigns it. Needs its own D-number before `dc-dc-prefixes-import.py` can run. | +| `ORG_ULA_48` | **ASSIGNED (D-118)** | **`fd50:840e:74e2::/48`** -- RFC 4193 valid, no collision with Tailscale`s `fd7a:115c:a1e0::/48`. Unblocks the DC import. | | `DC_GUA_PREFIX` **dc0** | **MEASURED** (D-117) | `2602:f3e2:f02::/48` -- the apex's `vr1-dc0` block. Not renamed; the repo adopted the apex's name. | | `DC_GUA_PREFIX` **dc1** | **MEASURED** (D-117) | `2602:f3e2:f03::/48` -- the apex's `vr1-dc1` block. | | `DC2_V4_SUPERNET` | **ASSIGNED (D-115)** | **`10.12.64.0/19`** -- INSIDE the Cloud `/16`, free (DC1 holds `10.12.{4,8,12,16,32,36}`), DC1<->DC2 stay routable. The old `10.13.0.0/19` recommendation was OUTSIDE every allocated block (it would have squatted); D-115 supersedes it. Tool needs >= `/19` and re-packs 6 sequential `/22`s -- see 5.3. | diff --git a/docs/design-decisions.md b/docs/design-decisions.md index bfcfa40..05268fa 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -2943,3 +2943,59 @@ `dc1`/`dc2`=VR1) while `netbox/dc-dc-prefixes-import.py --dc` uses the NEW, apex-aligned meaning (`dc0`/`dc1` = VR1's two DCs). **These two are NOT interchangeable.** Do not copy a `$DC` value from a shell script into `--dc`, or vice versa, without re-reading this entry. + +--- + +## D-118: the org ULA /48 -- ADOPT `fd50:840e:74e2::/48` + +**Status:** **ADOPTED 2026-07-13 (operator ruling).** Closes gap **G3** +(`docs/dc-dc-netbox-buildout-scope.md`) and the D-101 open literal `ORG_ULA_48`. **UNBLOCKS the VR1 +DC six-plane NetBox import**, which could not run without it. + +**Ruled:** `ORG_ULA_48 = **fd50:840e:74e2::/48**`. + +D-101 rules the org ULA `/48` a NetBox-assigned literal, "NOT hardcoded in this decision", and every +tool takes it as a REQUIRED env var with no default. Until now no decision assigned it, so +`netbox/dc-dc-prefixes-import.py` was unrunnable -- supplying a value would have been an inferred +value (hard rule 2). + +**Validation (measured 2026-07-13, not asserted):** + +| Property | Result | +|---|---| +| Inside `fc00::/7` | yes | +| First byte `0xfd` -> **L=1**, locally assigned (RFC 4193) | yes | +| 40-bit Global ID | `0x50840e74e2` -- no discernible structure | +| Prefix length | `/48` -> **256 x `/56`**, and D-101 allocates one `/56` per DC | +| Collides with Tailscale's ULA `fd7a:115c:a1e0::/48` | **NO** | +| Collides with anything else in the apex | no -- the apex holds only the `fc00::/8` / `fd00::/8` containers and Tailscale's `/48` | + +**The Tailscale non-collision is the load-bearing check, not a formality.** `office1-tailscale` runs +with `--accept-routes` on the self-hosted control plane, so an org ULA overlapping +`fd7a:115c:a1e0::/48` would be a LIVE routing conflict on the tailnet, not a paper one. `fd50` vs +`fd7a` -- distinct. + +**Provenance, stated honestly.** `docs/dc-dc-netbox-buildout-scope.md` 4c describes this value as +CSPRNG-generated. That claim originates in an earlier session and **cannot be re-verified now**. It +does not change the ruling: RFC 4193's "SHA1 of time + MAC" construction is a **SHOULD**, not a MUST +-- the actual requirement is a pseudo-random 40-bit Global ID, which this satisfies. Regenerating +would not make the value MORE correct; it would only substitute a different random number, at the +cost of re-editing the 7 places that already carry this one and risking a stale literal surviving in +one of them. Not worth it. + +**Corrects a FALSE claim in the repo.** `docs/changelog-20260711-ula-gen-command-fix.md` called this +"the **ratified** VR1 value" -- it was NOT ratified; no D-number ever assigned it. That phantom +ratification is very likely why the value has been treated as settled for two days while +`grep ORG_ULA docs/design-decisions.md` returned nothing. This entry makes the claim true, and the +changelog is annotated. **Same failure mode as D-117:** a value treated as authoritative because a +document asserted it, with no decision behind it. + +**Consequences.** The DC import can now run: +``` +ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f02::/48 --dc dc0 # VR1 DC0 +ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f03::/48 \ + DC1_V4_SUPERNET=10.12.64.0/19 --dc dc1 # VR1 DC1 +``` +(`DC_GUA_PREFIX` values are MEASURED off the apex per D-117; `DC1_V4_SUPERNET` is assigned by D-115.) +Per D-101/D-111 each DC takes a `/56` of this `/48`, indexed to its GUA site nibble, so the 4th +hextet reads `DC.NN` in BOTH families. diff --git a/docs/session-ledger.md b/docs/session-ledger.md index dd9d251..7eaa38f 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -299,7 +299,36 @@ The **office KEEPS its number** (Off1). Fixed the importer's off-by-one site binding and made it **dry by default**. -### BLOCKED: the VR1 DC six-plane import needs `ORG_ULA_48` RATIFIED +### ~~BLOCKED~~ **UNBLOCKED and DONE** -- D-118 ratified the org ULA; both DCs imported + +**D-118 ADOPTED:** `ORG_ULA_48 = fd50:840e:74e2::/48` (RFC 4193 valid; **no collision with +Tailscale's `fd7a:115c:a1e0::/48`**, which matters because `office1-tailscale` runs `--accept-routes` +-- an overlap would have been a LIVE tailnet routing conflict). Also corrected a **FALSE claim** in +`changelog-20260711-ula-gen-command-fix.md` that called the value "ratified" when no D-number ever +assigned it -- the same document-asserts-authority failure mode as D-117. + +**Imported into the sandbox:** six-plane roles + 2 RIRs + 5 aggregates, then **VR1 DC0 (18 prefixes) +and VR1 DC1 (18)**. The D-117 fix is now **LIVE-PROVEN**, not just offline: `--dc dc0` bound to the +apex site `VR1 DC0` (id=8) and `--dc dc1` to `VR1 DC1` (id=9). ULA reads `DC.NN` in the 4th hextet +exactly as D-111 intends (`fd50:840e:74e2:220::` .. `:350::`). + +**Final state PROVEN by `netbox/sandbox-fidelity-check.py`:** the sandbox is the upstream draft plus +EXACTLY the planned delta (D-115 + D-117 + D-118) -- every shared object field-identical, nothing +lost, nothing stray. 90 -> 134 prefixes. + +### A REAL BUG in `roles-aggregates-import.py`, found by running it for the first time for real + +It checked existence by **slug only**. NetBox also enforces **unique NAMES**. The draft's legacy role +`Replication` (slug `repl`) blocks the six-plane role (slug `replication`, name `Replication`), so it +created 4 roles, **400'd on the 5th, and DIED -- leaving the apex half-populated** with no RIRs and no +aggregates and no rollback. It had **never been run against a NetBox holding the real draft** +(only empty ones) and **shipped with NO harness** -- which is why this survived. + +Fixed: the six-plane role is now named **"Replication Plane"** (slug unchanged -- `lib-net.sh` +SPACES6 and the prefix importer look it up), and the script now **PREFLIGHTS every name/slug and +writes nothing unless the whole plan is viable**. `tests/roles-aggregates-import/` created: 16/16. + +### (historical) the original blocker `netbox/dc-dc-prefixes-import.py --dc dc0` cannot run. It requires `ORG_ULA_48`, and that literal is **still unassigned** -- it is gap **G3** in `docs/dc-dc-netbox-buildout-scope.md`, which *recommends* diff --git a/netbox/roles-aggregates-import.py b/netbox/roles-aggregates-import.py index 9480246..f579930 100644 --- a/netbox/roles-aggregates-import.py +++ b/netbox/roles-aggregates-import.py @@ -101,7 +101,15 @@ ("data-tenant", "Data Tenant", 130, "Six-plane: geneve overlay, ULA-only; tenant GUA delegated separately. D-052/D-101."), # storage (weight 140) already exists -- intentionally omitted. - ("replication", "Replication", 150, + # NAME is "Replication Plane", NOT "Replication" -- the legacy pre-six-plane role + # `repl` ALREADY HOLDS the name "Replication" in the real draft, and NetBox enforces + # UNIQUE NAMES as well as unique slugs. Creating this as "Replication" returns + # 400 {'name': ['role with this name already exists.']}. This script had never met + # a NetBox carrying the draft (only empty ones), so the collision went unseen until + # 2026-07-13 -- when it 400'd MID-RUN and left the roles half-created. + # The slug stays `replication` because that is what lib-net.sh SPACES6 and + # dc-dc-prefixes-import.py look up. + ("replication", "Replication Plane", 150, "Six-plane: Ceph cluster incl. cross-DC leg, ULA-only. D-052/D-101/D-108."), ] @@ -159,6 +167,34 @@ print(f" WOULD {kind:10s} {label}") created += 1 + # --- PREFLIGHT ----------------------------------------------------------- + # Validate EVERY name/slug against NetBox BEFORE writing anything. + # + # On 2026-07-13 this script created 4 roles, then hit a 400 on the 5th (a NAME + # collision with the legacy `repl` role) and DIED -- leaving the apex half + # populated, with no RIRs and no aggregates, and no rollback. An IPAM importer + # that can fail partway through is a tool that corrupts the thing it is meant + # to populate. Check first; write only if the whole plan is viable. + conflicts = [] + for slug, name, _w, _d in ROLES: + if nb.ipam.roles.get(slug=slug): + continue # same slug -> we skip it anyway + clash = nb.ipam.roles.get(name=name) + if clash: + conflicts.append( + f"role name {name!r} is already held by slug {clash.slug!r} (id={clash.id}) " + f"-- cannot create slug {slug!r}. NetBox enforces UNIQUE NAMES, not just slugs.") + for name, slug, _p in RIRS: + if nb.ipam.rirs.get(slug=slug) or nb.ipam.rirs.get(name=name): + continue + # nothing to check: absent by both keys + if conflicts: + print("PREFLIGHT FAILED -- nothing was written:\n") + for c in conflicts: + print(f" * {c}") + die("resolve the name collision(s) above, then re-run. NO changes were made.") + # ------------------------------------------------------------------------- + # --- Roles --- print("== IPAM roles (six-plane; slugs match lib-net.sh SPACES6) ==") for slug, name, weight, desc in ROLES: diff --git a/netbox/sandbox-fidelity-check.py b/netbox/sandbox-fidelity-check.py index a62bfd1..09b4ca0 100755 --- a/netbox/sandbox-fidelity-check.py +++ b/netbox/sandbox-fidelity-check.py @@ -40,10 +40,39 @@ KEY={"ipam/rirs":"slug","ipam/roles":"slug","dcim/regions":"slug","dcim/sites":"slug", "ipam/aggregates":"prefix","ipam/prefixes":"prefix"} -EXPECTED_NEW={"ipam/roles":{"edge"},"dcim/sites":{"vr1-off1"}, - "ipam/prefixes":{"10.10.0.0/16","10.10.0.0/22","10.10.0.0/24","10.10.1.0/24", - "172.30.0.0/16","172.30.1.0/24", - "2602:f3e2:f01:100::/56","2602:f3e2:f01:100::/64"}} +# The FULL planned delta the sandbox is allowed to have over the upstream draft. +# Anything outside this is a tooling bug (a silent drop, or a stray write). +# D-115 the office carve (site vr1-off1, edge role, 8 prefixes) +# D-117 VR1 DC0/DC1 six-plane (36 prefixes -- 18 per DC) +# D-118 the org ULA /48 (RIRs + aggregates) +# G1/G2 six-plane roles + aggregates +def _dc_prefixes(): + out = set() + for v4 in ["10.12.4.0/22","10.12.8.0/22","10.12.12.0/22","10.12.16.0/22", + "10.12.32.0/22","10.12.36.0/22", # VR1 DC0 (inherited) + "10.12.64.0/22","10.12.68.0/22","10.12.72.0/22","10.12.76.0/22", + "10.12.80.0/22","10.12.84.0/22"]: # VR1 DC1 (D-115 /19) + out.add(v4) + for dc in ("f02", "f03"): + out |= {f"2602:f3e2:{dc}:10::/60", f"2602:f3e2:{dc}:10::/64", f"2602:f3e2:{dc}:11::/64"} + for n in ("2", "3"): # DC nibble + for nn in ("20", "30", "40", "50"): + out.add(f"fd50:840e:74e2:{n}{nn}::/60") + out.add(f"fd50:840e:74e2:{n}{nn}::/64") + out.add(f"fd50:840e:74e2:{n}21::/64") # metal-internal shares metal /60 + return out + +EXPECTED_NEW = { + "ipam/roles": {"edge", "provider-public", "metal-admin", "metal-internal", + "data-tenant", "replication"}, + "dcim/sites": {"vr1-off1"}, + "ipam/rirs": {"rfc-4193-ula", "rfc-1918"}, + "ipam/aggregates": {"2602:f3e2::/36", "23.157.124.0/24", "10.0.0.0/8", + "172.16.0.0/12", "fd50:840e:74e2::/48"}, + "ipam/prefixes": {"10.10.0.0/16","10.10.0.0/22","10.10.0.0/24","10.10.1.0/24", + "172.30.0.0/16","172.30.1.0/24", + "2602:f3e2:f01:100::/56","2602:f3e2:f01:100::/64"} | _dc_prefixes(), +} bad=0 for ep,k in KEY.items(): U={r[k]:r for r in up.get(ep,[])} @@ -76,7 +105,7 @@ print(" field-level: IDENTICAL on every shared object") print("\n" + "=" * 70) if bad == 0: - print("SANDBOX IS A FAITHFUL REPLICA + the expected D-115 delta") + print("SANDBOX = upstream draft + EXACTLY the planned delta (D-115/D-117/D-118). Nothing lost, nothing stray.") sys.exit(0) print(f"!! {bad} fidelity problem(s) -- the seeder is dropping or mangling data") sys.exit(1) diff --git a/tests/roles-aggregates-import/run-tests.sh b/tests/roles-aggregates-import/run-tests.sh new file mode 100755 index 0000000..2c8d87d --- /dev/null +++ b/tests/roles-aggregates-import/run-tests.sh @@ -0,0 +1,72 @@ +#!/usr/bin/env bash +# tests/roles-aggregates-import/run-tests.sh +# +# Harness for netbox/roles-aggregates-import.py. OFFLINE -- touches no NetBox. +# +# THIS SCRIPT SHIPPED WITH NO HARNESS, and that is not incidental to the bug it +# carried. It was only ever exercised against an EMPTY NetBox. The moment it met +# one holding the real upstream draft (2026-07-13) it created 4 roles, hit a 400 +# on the 5th, and DIED -- leaving the IPAM apex half-populated with no RIRs and +# no aggregates, and no rollback. +# +# Two properties are pinned here, both learned from that failure: +# 1. NetBox enforces UNIQUE ROLE NAMES, not just unique slugs. The legacy `repl` +# role already holds the name "Replication", so the six-plane role (slug +# `replication`) MUST NOT be named "Replication". +# 2. The importer must PREFLIGHT every name/slug before writing ANYTHING. An +# IPAM importer that can die partway corrupts the thing it exists to populate. +# ASCII + LF. +set -uo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +S="$(cd "$HERE/../../netbox" && pwd)/roles-aggregates-import.py" +pass=0; fail=0 +ok() { pass=$((pass+1)); } +bad() { fail=$((fail+1)); echo " FAIL: $1"; } + +command -v python3 >/dev/null 2>&1 || { echo "FAIL: python3 required"; exit 1; } + +python3 -c "import ast;ast.parse(open('$S').read())" 2>/dev/null && ok || bad "does not parse" +# This script needs pynetbox (unlike the stdlib tools). On a host without it, it +# must say so CLEANLY rather than traceback -- so accept either. +out="$(python3 "$S" --help 2>&1)" +if printf '%s' "$out" | grep -q "pynetbox not installed"; then ok # clean, actionable +elif printf '%s' "$out" | grep -q -- "--commit"; then ok # real help text +else bad "--help neither printed help nor a clean 'pynetbox not installed' message"; fi + +# DRY BY DEFAULT -- this one already was, and must stay so. +grep -q -- '--commit' "$S" && ok || bad "no --commit flag" +grep -q 'default: preview only, no writes' "$S" && ok || bad "no longer documents preview-by-default" + +# 1. THE NAME COLLISION. Naming the six-plane role "Replication" 400s against any +# NetBox carrying the draft, because legacy slug `repl` holds that name. +grep -q '"Replication Plane"' "$S" && ok \ + || bad "the six-plane replication role is not named 'Replication Plane' -- it will 400 against the real draft (legacy slug 'repl' holds the name 'Replication')" +grep -qE '\("replication", "Replication",' "$S" \ + && bad "the six-plane role is named exactly 'Replication' -- COLLIDES with legacy slug 'repl'" || ok +# the slug must NOT drift: lib-net.sh SPACES6 and dc-dc-prefixes-import.py look it up. +grep -q '"replication", "Replication Plane"' "$S" && ok \ + || bad "the replication slug drifted -- lib-net.sh SPACES6 and the prefix importer look up slug 'replication'" + +# 2. PREFLIGHT. Nothing may be written until the whole plan is known viable. +grep -q 'PREFLIGHT FAILED -- nothing was written' "$S" && ok \ + || bad "the preflight is gone -- a name collision can again leave the apex HALF-WRITTEN" +grep -q 'NetBox enforces UNIQUE NAMES, not just slugs' "$S" && ok \ + || bad "the preflight no longer checks NAME collisions (only slugs) -- that IS the bug" +# the preflight must run BEFORE the first create +pf=$(grep -n 'PREFLIGHT' "$S" | head -1 | cut -d: -f1) +cr=$(grep -n 'nb.ipam.roles.create' "$S" | head -1 | cut -d: -f1) +[ -n "$pf" ] && [ -n "$cr" ] && [ "$pf" -lt "$cr" ] && ok \ + || bad "preflight does not run BEFORE the first create -- half-written apex is back" + +# 3. the six-plane slugs must match lib-net.sh SPACES6 exactly. +for slug in provider-public metal-admin metal-internal data-tenant replication; do + grep -q "\"$slug\"" "$S" && ok || bad "six-plane role slug '$slug' is gone" +done +# storage is deliberately NOT created -- it pre-exists upstream and is reused. +grep -q 'already exists -- intentionally omitted' "$S" && ok \ + || bad "the note that 'storage' is reused (not created) is gone" + +echo +total=$((pass+fail)) +if [ "$fail" -eq 0 ]; then echo "roles-aggregates-import: $pass/$total PASS"; exit 0; fi +echo "roles-aggregates-import: $fail/$total FAIL"; exit 1