diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 04a878d..bfcfa40 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -2787,7 +2787,15 @@ ## D-117: VR1 site naming -- the apex says DC0/DC1, the repo says DC1/DC2 (G5) -**Status:** **PROPOSED 2026-07-13.** Blocks the NetBox import. Options below; operator rules. +**Status:** **ADOPTED 2026-07-13 (operator ruling: Option B, DC-only).** The REPO moves to the +apex's 0-indexed convention **for the DATACENTERS**. NetBox is the IPAM apex and is NOT bent to fit +the repo. Supersedes the `docs/dc-dc-netbox-buildout-scope.md` G5 recommendation (which proposed the +opposite, Option A) and the D-106 `dc1`/`dc2` zone labels. Closes G5. + +**Operator refinement (verbatim):** *"go ahead and rename the DC numbers. The office number is +fine."* -- so the **OFFICE KEEPS ITS NUMBER**: it stays **Office1**, and its NetBox site is +**`vr1-off1` / "VR1 Off1"**. That site does NOT exist upstream yet, so it is CREATED, not renamed -- +no collision (VR0's `vr0-off0` is untouched), and **nothing deployed changes**. **This is the G5 gap** (`docs/dc-dc-netbox-buildout-scope.md`), promoted to a decision because it blocks the Office1 NetBox import and because the tool that trips over it WRITES BY DEFAULT. @@ -2849,3 +2857,89 @@ **Whichever is chosen, one fix is unconditional:** give `dc-dc-prefixes-import.py` a `--commit` flag so it is DRY BY DEFAULT, matching `roles-aggregates-import.py`. A write-by-default IPAM tool is a standing hazard regardless of how the naming lands. + +### D-117 -- AMENDMENT (2026-07-13): ADOPTED (Option B), scope narrowed by operator ruling + +**Ruled:** VR1's datacenters are **DC0** (`vr1-dc0`, `2602:f3e2:f02::/48`) and **DC1** (`vr1-dc1`, +`2602:f3e2:f03::/48`) -- matching the apex, which already holds both sites. The repo's `dc1`/`dc2` +labels are retired in favour of **`dc0`/`dc1`**. + +VR1's office **remains Office1** -> NetBox site **`vr1-off1` ("VR1 Off1")**, carrying the D-115 carve +(`2602:f3e2:f01:100::/56` + `/64`, `10.10.0.0/22`). Note VR1 therefore reads **DC0/DC1/Off1** while +VR0 reads DC0/DC1/Off0 -- an accepted cosmetic asymmetry. The office site is NEW upstream, so it is +created cleanly and **no deployed object is renamed**. + +**SCOPE -- what B does and does NOT touch.** B is a ruling about **IPAM naming authority**: the +NetBox site record and the selector that resolves to it. It does NOT extend to deployment substrate. + +- **IN scope (renamed, DC ONLY):** the DC half of the NetBox site identity; the importer's `SITES` + + GUA map; the `--dc` selector (`dc1|dc2` -> `dc0|dc1`); `lib-net.sh`/`lib-hosts.sh` DC keys; + OpenTofu DC module labels; forward-looking DC prose. +- **OUT of scope -- NOTHING office-side is renamed.** Per the operator refinement above, `Office1` + keeps its number, so `voffice1`, `office1-opnsense`, the `office1-local`/`office1-wan` networks, + `office1-pool`, the MAAS VM host `office1-lxd`, the machines `office1-netbox` / + `office1-tailscale`, the OPNsense `OFFICE1_LXD_GW` object and the Tailscale node all stand as-is. + **No teardown, no rebuild.** +- **Also out of scope, independently:** deployment substrate is not a carve. Even where a name DID + change (the DC libvirt objects), the rename is a label on empty, unused infrastructure -- see the + table below. + +**TRAP 1 -- this is a SEMANTIC REMAP, not a string replace.** `dc1 -> dc0` AND `dc2 -> dc1`, so +`dc1` denotes different things before and after. A sequential `sed` double-applies and corrupts the +tree. Any mechanical pass must go through a placeholder (`dc1 -> __DCA__`, `dc2 -> __DCB__`, then +`__DCA__ -> dc0`, `__DCB__ -> dc1`). + +**TRAP 2 -- and this one NO mechanical pass can catch.** Existing prose uses "DC0" to mean **VR0's +rehearsal DC** while using "DC1"/"DC2" to mean **VR1's** DCs. The importer's docstring: *"DC1 +INHERITS the DC0 six-plane CIDRs unchanged (D-101 -- 'DC1 equals the DC0 template') ... not the +existing `vr0-dc0` site."* Remap VR1's `dc1 -> dc0` and that reads *"DC0 equals the DC0 template"* -- +two DIFFERENT DC0s (`vr0-dc0` and `vr1-dc0`) that were previously disambiguated by NUMBER ALONE. +**Every "DCn" in prose must be region-qualified by hand** (`VR0 DC0` vs `VR1 DC0`). This is why the +decision text below is ANNOTATED, not rewritten. + +**Treatment by surface kind (adopted with the ruling):** + +| Surface | Treatment | +|---|---| +| Operational/structured (importer, `lib-net.sh`, `--dc`, tofu labels) | Mechanical rename. **This tier ALONE unblocks the NetBox import.** | +| **ADOPTED decision text** (D-101, D-106, D-111, D-115) | **ANNOTATE in place, do NOT rewrite** -- the D-103->D-116 supersession precedent. Ratified prose is a historical record. | +| Forward-looking prose (runbooks, design docs) | Hand-edit, region-qualifying every "DCn" (Trap 2). | +| DC libvirt objects (`dc1-*`/`dc2-*` nets, `dc1-pool`/`dc2-pool`) | **Empty and unused** (0 volumes, no DC VMs). Rename the module labels; let the next **GATED** DC apply reconcile them. **No surprise destroy/recreate.** | +| Historical changelogs | **Never rewritten.** | + +**Unconditional companion fix (also ruled):** `netbox/dc-dc-prefixes-import.py` becomes **DRY BY +DEFAULT** with an explicit `--commit`, matching `roles-aggregates-import.py`. A write-by-default tool +pointed at the IPAM apex is a standing hazard, and it is the mechanism by which this naming bug would +have landed silently. + +### D-117 -- AMENDMENT (2026-07-13): the `$DC` shell selector is a SECOND, COLLIDING namespace + +Found while executing the rename. **`scripts/lib-net.sh` / `lib-hosts.sh` use a THREE-value `$DC` +selector in which `dc0` ALREADY MEANS VR0's DC0** (the deployed rehearsal), with `dc1`/`dc2` meaning +VR1's two DCs: + +``` +dc0|dc1) # lib-net.sh: VR0 DC0 and VR1's first DC share v4 literals (D-101 inheritance) +dc2) # VR1's second DC -- FAILS LOUD, no literals assigned yet +``` + +So the repo has been carrying **two different `dcN` namespaces at once**: the shell libs' (where +`dc0` = VR0) and the NetBox importer's (where `dc0` did not exist and `dc1` = VR1's first DC). That +is Trap 2 in its purest form, and it is why a mechanical `sed` across this rename would have been +**actively dangerous**: renaming VR1's `dc1 -> dc0` in `lib-net.sh` collides HEAD-ON with VR0's +existing `dc0` and would silently hand VR1's selector VR0's deployed literals. + +**Proposed fix (NOT yet executed -- does not block the NetBox import):** region-qualify the shell +selector to **the apex's own slugs** -- `vr0-dc0`, `vr1-dc0`, `vr1-dc1` -- so there is ONE namespace +across NetBox, the shell libs, and the importer, and every identifier is unambiguous by construction. +This is the same principle D-117 already adopts: the apex names things; the repo follows. + +**Blast radius:** `lib-net.sh`, `lib-hosts.sh`, and the scripts that take `$DC` +(`carve-host-interfaces.sh`, `dc-dc-ceph-disk-budget.sh`, `dc-dc-dr-drill.sh`, +`dc-dc-radosgw-multisite.sh`, `dc-dc-rbd-mirror.sh`, `phase-00-maas-standup.sh`, +`reenroll-hosts.sh`) plus their harnesses and `tests/dc-selector/`. + +**Until it is executed, the shell `$DC` selector retains its OLD meaning** (`dc0`=VR0 DC0, +`dc1`/`dc2`=VR1) while `netbox/dc-dc-prefixes-import.py --dc` uses the NEW, apex-aligned meaning +(`dc0`/`dc1` = VR1's two DCs). **These two are NOT interchangeable.** Do not copy a `$DC` value from +a shell script into `--dc`, or vice versa, without re-reading this entry. diff --git a/netbox/dc-dc-prefixes-import.py b/netbox/dc-dc-prefixes-import.py index d805a5f..ad98905 100644 --- a/netbox/dc-dc-prefixes-import.py +++ b/netbox/dc-dc-prefixes-import.py @@ -6,7 +6,7 @@ extends the v1 (netbox/ipv4-prefixes-import.py) single-site IPv4-only import to the VR1 two-DC, dual-stack model ratified in D-101 (docs/design-decisions.md). -Per DC (dc1 or dc2, selected via --dc, mirroring the scripts/lib-net.sh +Per DC (dc0 or dc1, selected via --dc, mirroring the scripts/lib-net.sh $DC selector convention, DOCFIX-151), creates the six-plane structure with the D-101 family matrix: @@ -18,16 +18,27 @@ storage (see below) ULA-only replication (see below) ULA-only -DC1 v4: INHERITS the DC0 six-plane CIDRs unchanged (D-101 -- "DC1 equals the -validated template"), i.e. the SAME literals as netbox/ipv4-prefixes-import.py, -hardcoded here for that reason (this is not an inferred value; it is the -explicit text of an ADOPTED decision). Scoped to a DISTINCT NetBox site -(vr1-dc1), not the existing vr0-dc0 site. +NAMING (D-117, 2026-07-13): VR1's DCs are **VR1 DC0** and **VR1 DC1**, matching +the NetBox apex, which already holds both site records. This script previously +called them dc1/dc2 -- an off-by-one against the apex that would have bound +VR1 DC0's prefixes to the site vr1-dc1 (i.e. the OTHER DC), silently, because +the script also used to write by default. Both defects are fixed; see D-117. +EVERY "DCn" below is REGION-QUALIFIED, because "DC0" is ambiguous across VR0 +and VR1 and that ambiguity is exactly what produced the bug. -DC2 v4: D-101 explicitly states DC2's supernet is "NetBox-assigned... not -hardcoded." No default is provided here -- DC2_V4_SUPERNET is a REQUIRED -environment variable with no fallback. Running with --dc dc2 and no -DC2_V4_SUPERNET set FAILS LOUD rather than guessing or reusing DC1's range. +VR1 DC0 v4: INHERITS the **VR0 DC0** six-plane CIDRs unchanged (D-101 -- "the +first DC equals the validated template"), i.e. the SAME literals as +netbox/ipv4-prefixes-import.py, hardcoded here for that reason (this is not an +inferred value; it is the explicit text of an ADOPTED decision). Scoped to the +DISTINCT NetBox site **vr1-dc0** -- NOT the existing **vr0-dc0** site, which is +the rehearsal environment those literals came from. + +VR1 DC1 v4: D-101 explicitly states the second DC's supernet is "NetBox-assigned +... not hardcoded." No default is provided here -- DC1_V4_SUPERNET is a REQUIRED +environment variable with no fallback. Running with --dc dc1 and no +DC1_V4_SUPERNET set FAILS LOUD rather than guessing or reusing VR1 DC0's range. +(This env var was DC1_V4_SUPERNET before D-117; the old name is REJECTED with a +pointer rather than silently ignored.) IPv6 (both DCs): D-101 names two literals as NetBox-assigned and "NOT hardcoded in this decision": the org ULA /48, and the per-DC GUA carve out @@ -54,10 +65,10 @@ metal :20/:21, data :30, storage :40, replication :50. Each plane gets a /60 container + a /64 active, mirroring the deployed template. - GUA: provider-public only (D-101 family matrix). Out of DC_GUA_PREFIX (the - per-DC site /48, e.g. 2602:f3e2:f02::/48 for dc1): /60 + /64 active at :10, + per-DC site /48, e.g. 2602:f3e2:f02::/48 for VR1 DC0): /60 + /64 active at :10, API-VIP /64 at :11. - ULA: the five internal planes, out of a per-DC /56 of the org ULA /48. The - /56 index is the GUA site nibble (dc1 -> ...:02xx, dc2 -> ...:03xx) so the + /56 index is the GUA site nibble (VR1 DC0 -> ...:02xx, VR1 DC1 -> ...:03xx) so the 4th hextet reads DC.NN in BOTH families. metal-admin (:20) and metal-internal (:21) SHARE the metal /60 (at :20), exactly as provider (:10) / API-VIP (:11) share the provider /60. data-tenant :30, storage :40, replication :50 each @@ -69,19 +80,22 @@ unless --update). NetBox version: 4.x (scope_type/scope_id, not legacy site=). Usage: - # DC1 (v4 inherited, only v6 top-level blocks required): + # DRY-RUN IS THE DEFAULT (D-117). Nothing is written without --commit. + + # VR1 DC0 (v4 inherited from VR0 DC0; only v6 top-level blocks required): NETBOX_URL=https://netbox.baldurkeep.com NETBOX_TOKEN= \\ - ORG_ULA_48=fd00:1234:5678::/48 DC_GUA_PREFIX=2602:f3e2:1000::/40 \\ - python3 dc-dc-prefixes-import.py --dc dc1 + ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f02::/48 \\ + python3 dc-dc-prefixes-import.py --dc dc0 # preview + ... python3 dc-dc-prefixes-import.py --dc dc0 --commit # write - # DC2 (v4 supernet ALSO required, nothing defaulted): + # VR1 DC1 (v4 supernet ALSO required, nothing defaulted): NETBOX_URL=... NETBOX_TOKEN=... \\ - ORG_ULA_48=fd00:1234:5678::/48 DC_GUA_PREFIX=2602:f3e2:2000::/40 \\ - DC2_V4_SUPERNET=10.13.0.0/19 \\ - python3 dc-dc-prefixes-import.py --dc dc2 + ORG_ULA_48=fd50:840e:74e2::/48 DC_GUA_PREFIX=2602:f3e2:f03::/48 \\ + DC1_V4_SUPERNET=10.12.64.0/19 \\ + python3 dc-dc-prefixes-import.py --dc dc1 --commit - # Preview only, no writes: - ... python3 dc-dc-prefixes-import.py --dc dc1 --verify-only + # Verification block only (no plan, no writes): + ... python3 dc-dc-prefixes-import.py --dc dc0 --verify-only KNOWN PRE-EXISTING STALENESS (flagged, not fixed here): netbox/ipv4-prefixes- import.py's role slugs (provider/metal/lbaas-management/openstack-tenant) @@ -113,14 +127,22 @@ sys.exit(1) # ----------------------------------------------------------------------------- -# DC1/DC2 site identity. Slugs are new -- no prior convention existed in this -# repo (grepped opentofu/*.tf and the buildout design; none found) -- chosen -# to match the D-106 Designate naming already ratified (dc1/dc2, vr1 label). +# VR1 DC site identity. +# +# HISTORY (kept deliberately -- this is the bug's own confession): the original +# comment here read "Slugs are new -- no prior convention existed in this repo +# (grepped opentofu/*.tf and the buildout design; none found)". It grepped the +# REPO. The convention lived in the APEX (NetBox), which is precisely where +# D-010/D-103 say naming authority lives. That is how the off-by-one got in. # ----------------------------------------------------------------------------- +# D-117: these slugs are the APEX's, verified live against netbox.baldurkeep.com +# on 2026-07-13 (vr1-dc0 id=43 -> 2602:f3e2:f02::/48; vr1-dc1 id=44 -> f03::/48). +# They are NOT a repo-local invention. Do not "fix" them to dc1/dc2 -- that is the +# exact off-by-one D-117 closed. Prior art for naming lives in the APEX, not here. SITES = { + "dc0": {"slug": "vr1-dc0", "name": "VR1 DC0"}, "dc1": {"slug": "vr1-dc1", "name": "VR1 DC1"}, - "dc2": {"slug": "vr1-dc2", "name": "VR1 DC2"}, } # Fixed plane order -- must match scripts/lib-net.sh's PLANE_NAME exactly @@ -153,7 +175,7 @@ # historical single-plane set is NOT the comparison here -- these are the # CURRENT six-plane values from scripts/lib-net.sh's PLANE_CIDRS, which is # this repo's single source of truth for that layout. -DC1_V4_PREFIXES = { +VR0_DC0_TEMPLATE_V4 = { "provider-public": "10.12.4.0/22", "metal-admin": "10.12.8.0/22", "metal-internal": "10.12.12.0/22", @@ -162,12 +184,12 @@ "replication": "10.12.36.0/22", } -# DC2 v4: NO default. Must come from DC2_V4_SUPERNET (env var), carved into +# VR1 DC1 (the SECOND DC) v4: NO default. From DC1_V4_SUPERNET (env), carved into # six sequential /22s in PLANE_ORDER, exactly mirroring DC1's /22-per-plane # structure. The supernet must be large enough to hold six non-overlapping # /22s (a /19 holds exactly eight /22s -- six used, two spare; anything # smaller than /19 FAILS LOUD). -DC2_MIN_SUPERNET_PREFIXLEN = 19 # a /19 = exactly 8x /22, room for the six planes +DC1_MIN_SUPERNET_PREFIXLEN = 19 # a /19 = exactly 8x /22, room for the six planes def die(msg: str, code: int = 1) -> None: @@ -190,10 +212,28 @@ return nb -def find_or_create_site(nb, slug: str, name: str): +class _PlannedSite: + """A site that does not exist yet, in a run that is not allowed to create it. + + D-117: a dry run against a FRESH NetBox must still be able to show the whole + plan -- that is the entire point of previewing against an empty apex. Bailing + out with "no site id" would make --dry-run useless exactly when it matters + most. Nothing is ever written from one of these: id is None, and every write + path returns before touching the API.""" + def __init__(self, slug: str, name: str): + self.id = None + self.slug = slug + self.name = name + + +def find_or_create_site(nb, slug: str, name: str, commit: bool): + """D-117: a site is a WRITE. Without --commit we return a _PlannedSite stub + instead of creating one, so the rest of the plan can still be printed.""" site = nb.dcim.sites.get(slug=slug) if site is not None: return site, False + if not commit: + return _PlannedSite(slug, name), False # Minimal site record -- status defaults to NetBox's own default (active). created = nb.dcim.sites.create(name=name, slug=slug) return created, True @@ -261,33 +301,33 @@ return net -def validate_dc2_supernet(cidr: str) -> ipaddress.IPv4Network: +def validate_dc1_supernet(cidr: str) -> ipaddress.IPv4Network: try: net = ipaddress.ip_network(cidr, strict=True) except ValueError as exc: - die(f"DC2_V4_SUPERNET='{cidr}' is not a valid CIDR: {exc}") + die(f"DC1_V4_SUPERNET='{cidr}' is not a valid CIDR: {exc}") if net.version != 4: - die(f"DC2_V4_SUPERNET='{cidr}' is not an IPv4 prefix") - if net.prefixlen > DC2_MIN_SUPERNET_PREFIXLEN: + die(f"DC1_V4_SUPERNET='{cidr}' is not an IPv4 prefix") + if net.prefixlen > DC1_MIN_SUPERNET_PREFIXLEN: die( - f"DC2_V4_SUPERNET='{cidr}' (/{net.prefixlen}) is too small to hold " - f"six /22 planes -- need at least a /{DC2_MIN_SUPERNET_PREFIXLEN}" + f"DC1_V4_SUPERNET='{cidr}' (/{net.prefixlen}) is too small to hold " + f"six /22 planes -- need at least a /{DC1_MIN_SUPERNET_PREFIXLEN}" ) - dc1_nets = [ipaddress.ip_network(c) for c in DC1_V4_PREFIXES.values()] - for d1 in dc1_nets: + tmpl_nets = [ipaddress.ip_network(c) for c in VR0_DC0_TEMPLATE_V4.values()] + for d1 in tmpl_nets: if net.overlaps(d1): die( - f"DC2_V4_SUPERNET='{cidr}' overlaps DC1's inherited range {d1} -- " - f"D-101 requires DC2 to be non-overlapping. Choose a different supernet." + f"DC1_V4_SUPERNET='{cidr}' overlaps VR1 DC0's inherited range {d1} -- " + f"D-101 requires VR1 DC1 to be non-overlapping. Choose a different supernet." ) return net -def carve_dc2_v4(supernet: ipaddress.IPv4Network) -> dict: +def carve_dc1_v4(supernet: ipaddress.IPv4Network) -> dict: subnets = list(supernet.subnets(new_prefix=22)) if len(subnets) < len(PLANE_ORDER): die( - f"DC2_V4_SUPERNET yields only {len(subnets)} /22 subnet(s), " + f"DC1_V4_SUPERNET yields only {len(subnets)} /22 subnet(s), " f"need {len(PLANE_ORDER)}" ) return {plane: str(subnets[i]) for i, plane in enumerate(PLANE_ORDER)} @@ -299,9 +339,9 @@ # live template. Replaces the earlier contiguous-plane-index subcarve. See the # D-111 decision record + docs/dc-dc-netbox-buildout-scope.md section 5.1. # -# Per-DC ULA /56 index aligned to the GUA site nibble (dc1 -> ...:02xx, dc2 -> +# Per-DC ULA /56 index aligned to the GUA site nibble (VR1 DC0 -> ...:02xx, VR1 DC1 -> # ...:03xx) so the 4th hextet reads DC.NN in BOTH families. -DC_V6_INDEX = {"dc1": 0x02, "dc2": 0x03} +DC_V6_INDEX = {"dc0": 0x02, "dc1": 0x03} def _sub_at(base: ipaddress.IPv6Network, new_prefixlen: int, index: int) -> ipaddress.IPv6Network: @@ -338,7 +378,10 @@ # ----------------------------------------------------------------------------- -def create_or_report_prefix(nb, cidr: str, role_slug: str, site, description: str, update: bool) -> None: +def create_or_report_prefix(nb, cidr: str, role_slug: str, site, description: str, update: bool, commit: bool) -> None: + """D-117: DRY BY DEFAULT. Without commit=True this writes NOTHING -- it only + prints the plan. This tool used to write by default, which is how an + off-by-one site binding could have landed in the IPAM apex unannounced.""" role = find_role(nb, role_slug) existing = nb.ipam.prefixes.get(prefix=cidr) payload = { @@ -349,9 +392,15 @@ "scope_id": site.id, } if existing is None: + if not commit: + print(f" [dry-run] would CREATE prefix {cidr} role={role_slug} -> site {site.name}") + return created = nb.ipam.prefixes.create(**payload) print(f" CREATED prefix {cidr} (id={created.id}) role={role_slug}") elif update: + if not commit: + print(f" [dry-run] would UPDATE prefix {cidr} (id={existing.id}) role={role_slug} -> site {site.name}") + return existing.update(payload) print(f" UPDATED prefix {cidr} (id={existing.id}) role={role_slug}") else: @@ -378,53 +427,68 @@ def main() -> int: parser = argparse.ArgumentParser(description=__doc__.split("\n\n", 1)[0]) - parser.add_argument("--dc", required=True, choices=["dc1", "dc2"], help="Which DC to import (no default -- explicit, measured selection per the $DC convention)") - parser.add_argument("--update", action="store_true", help="Update existing prefixes in place") - parser.add_argument("--verify-only", action="store_true", help="Skip writes; only print verification block") + parser.add_argument("--dc", required=True, choices=["dc0", "dc1"], help="Which VR1 DC to import (D-117: dc0|dc1, matching the NetBox apex. The old dc1|dc2 labels were off by one -- see D-117)") + parser.add_argument("--update", action="store_true", help="Update existing prefixes in place (requires --commit)") + parser.add_argument("--commit", action="store_true", help="WRITE to NetBox. Default is a DRY RUN that writes nothing (D-117 -- this tool used to write by default)") + parser.add_argument("--verify-only", action="store_true", help="Skip the plan entirely; only print the verification block") args = parser.parse_args() + # D-117: reject the pre-rename env var by NAME rather than ignoring it. An + # operator with muscle memory would otherwise get "required env missing" and + # have no idea the variable was renamed under them. + if os.environ.get("DC2_V4_SUPERNET") and not os.environ.get("DC1_V4_SUPERNET"): + die("DC2_V4_SUPERNET is set, but that name was RETIRED by D-117. VR1's " + "second DC is now 'dc1' (not 'dc2'), so the variable is DC1_V4_SUPERNET. " + "Re-export it under the new name -- refusing to guess.") + dc = args.dc + commit = args.commit site_cfg = SITES[dc] org_ula_48 = validate_ula_48(require_env("ORG_ULA_48", "D-101's org ULA /48 -- not defaulted, must be a real RFC 4193 /48 you generated")) dc_gua_prefix = validate_gua_prefix(require_env("DC_GUA_PREFIX", f"the {dc} carve out of ARIN 2602:f3e2::/32 region-0 /36 cited in D-101 -- not defaulted")) - if dc == "dc1": - v4 = dict(DC1_V4_PREFIXES) + if dc == "dc0": + v4 = dict(VR0_DC0_TEMPLATE_V4) else: - dc2_supernet = validate_dc2_supernet(require_env("DC2_V4_SUPERNET", "D-101 explicitly requires DC2's v4 supernet be NetBox-assigned, not hardcoded")) - v4 = carve_dc2_v4(dc2_supernet) + second_supernet = validate_dc1_supernet(require_env("DC1_V4_SUPERNET", "D-101 explicitly requires the SECOND DC's v4 supernet be NetBox-assigned, not hardcoded (D-115 assigns 10.12.64.0/19)")) + v4 = carve_dc1_v4(second_supernet) v6_recs = carve_v6(dc, org_ula_48, dc_gua_prefix) nb = get_nb() - site, site_created = find_or_create_site(nb, site_cfg["slug"], site_cfg["name"]) - print(f"Connected. Site '{site_cfg['name']}' (id={site.id}){' -- CREATED' if site_created else ''}.") + site, site_created = find_or_create_site(nb, site_cfg["slug"], site_cfg["name"], commit) + if isinstance(site, _PlannedSite): + print(f"[dry-run] would CREATE site '{site.name}' ({site.slug}) -- it does not exist yet.") + else: + print(f"Connected. Site '{site_cfg['name']}' (id={site.id}){' -- CREATED' if site_created else ''}.") - if dc == "dc1": + if dc == "dc0": print( "\nNOTE: this may create prefixes identical to the existing 'vr0-dc0' " - "site's entries (D-101: DC1 inherits DC0's v4 layout unchanged). " + "site's entries (D-101: VR1 DC0 inherits VR0 DC0's v4 layout unchanged). " "NetBox permits duplicate global prefixes (soft-warn, not a hard " "block) but this is a real IPAM-hygiene decision for the operator: " - "either retire/delete the vr0-dc0 entries once DC1 supersedes that " + "either retire/delete the vr0-dc0 entries once VR1 DC0 supersedes that " "rehearsal environment, or knowingly keep both for historical " "record. This script does not decide that for you." ) if not args.verify_only: + print("\n*** DRY RUN -- nothing will be written. Re-run with --commit to apply. ***" + if not commit else "\n*** COMMITTING to NetBox. ***") print("\nIPv4 prefixes:") for plane in PLANE_ORDER: create_or_report_prefix( nb, v4[plane], plane, site, - f"VR1 {dc.upper()} {plane} (v4; D-101)", args.update, + f"VR1 {dc.upper()} {plane} (v4; D-101)", args.update, commit, ) print("\nIPv6 prefixes (D-101 family matrix on the D-111 NN layout):") for cidr, role_slug, kind in v6_recs: create_or_report_prefix( nb, cidr, role_slug, site, f"VR1 {dc.upper()} {role_slug} ({kind}; D-101/D-111)", - args.update, + args.update, commit, ) verify(nb, site, v4, v6_recs) diff --git a/tests/dc-dc-prefixes-import/test_logic.py b/tests/dc-dc-prefixes-import/test_logic.py index 3ca17d6..464efdb 100644 --- a/tests/dc-dc-prefixes-import/test_logic.py +++ b/tests/dc-dc-prefixes-import/test_logic.py @@ -113,36 +113,36 @@ expect_dies("gua unrelated GUA block rejected", T.validate_gua_prefix, "2001:db8::/40") # ----------------------------------------------------------------------------- -# validate_dc2_supernet +# validate_dc1_supernet # ----------------------------------------------------------------------------- -dc2_super = expect_ok("dc2 supernet valid /19 no overlap", T.validate_dc2_supernet, "10.13.0.0/19") -expect_dies("dc2 supernet too small (/20) rejected", T.validate_dc2_supernet, "10.13.0.0/20") -expect_dies("dc2 supernet overlapping DC1 rejected", T.validate_dc2_supernet, "10.12.0.0/19") -expect_dies("dc2 supernet IPv6 input rejected", T.validate_dc2_supernet, "fd00::/19") +dc2_super = expect_ok("second-DC supernet valid /19 no overlap", T.validate_dc1_supernet, "10.12.64.0/19") +expect_dies("second-DC supernet too small (/20) rejected", T.validate_dc1_supernet, "10.12.64.0/20") +expect_dies("second-DC supernet overlapping VR1 DC0 rejected", T.validate_dc1_supernet, "10.12.0.0/19") +expect_dies("second-DC supernet IPv6 input rejected", T.validate_dc1_supernet, "fd00::/19") # ----------------------------------------------------------------------------- -# carve_dc2_v4 +# carve_dc1_v4 # ----------------------------------------------------------------------------- if dc2_super is not None: - v4 = T.carve_dc2_v4(dc2_super) - check(set(v4.keys()) == set(T.PLANE_ORDER), "carve_dc2_v4 covers all six planes") + v4 = T.carve_dc1_v4(dc2_super) + check(set(v4.keys()) == set(T.PLANE_ORDER), "carve_dc1_v4 covers all six planes") all_22 = all(ipaddress.ip_network(c).prefixlen == 22 for c in v4.values()) - check(all_22, "carve_dc2_v4 all /22") + check(all_22, "carve_dc1_v4 all /22") within = all(ipaddress.ip_network(c).subnet_of(dc2_super) for c in v4.values()) - check(within, "carve_dc2_v4 all within supernet") + check(within, "carve_dc1_v4 all within supernet") distinct = len(set(v4.values())) == len(v4) - check(distinct, "carve_dc2_v4 all distinct") + check(distinct, "carve_dc1_v4 all distinct") # ----------------------------------------------------------------------------- # carve_v6 (D-111 NN layout: /60+/64 per plane, deployed net-byte mnemonic) # ----------------------------------------------------------------------------- org_ula = ipaddress.ip_network("fd12:3456:789a::/48") -gua_dc1 = ipaddress.ip_network("2602:f3e2:f02::/48") -recs1 = T.carve_v6("dc1", org_ula, gua_dc1) -recs2 = T.carve_v6("dc2", org_ula, ipaddress.ip_network("2602:f3e2:f03::/48")) +gua_dc0 = ipaddress.ip_network("2602:f3e2:f02::/48") +recs1 = T.carve_v6("dc0", org_ula, gua_dc0) +recs2 = T.carve_v6("dc1", org_ula, ipaddress.ip_network("2602:f3e2:f03::/48")) # structure: 12 records (provider 3, metal-admin 2, metal-internal 1, data/storage/repl 2 each) -check(len(recs1) == 12, "carve_v6 dc1 yields 12 records", str(len(recs1))) +check(len(recs1) == 12, "carve_v6 VR1-DC0 yields 12 records", str(len(recs1))) roles1 = [r for _, r, _ in recs1] check(roles1.count("provider-public") == 3, "carve_v6 provider-public has 3 (/60,/64,VIP)") check(roles1.count("metal-admin") == 2, "carve_v6 metal-admin has 2 (/60,/64)") @@ -153,24 +153,24 @@ # family matrix: provider-public is GUA (within its /48); the rest are ULA (fd..) addrs1 = {c for c, _, _ in recs1} prov = [ipaddress.ip_network(c) for c, r, _ in recs1 if r == "provider-public"] -check(all(n.subnet_of(gua_dc1) for n in prov), "carve_v6 provider-public within the GUA /48") +check(all(n.subnet_of(gua_dc0) for n in prov), "carve_v6 provider-public within the GUA /48") nonprov = [ipaddress.ip_network(c) for c, r, _ in recs1 if r != "provider-public"] check(all(str(n.network_address).startswith("fd") for n in nonprov), "carve_v6 non-provider planes are ULA") # NN offsets land on the deployed mnemonic; DC.NN reads in the 4th hextet -check("2602:f3e2:f02:10::/60" in addrs1, "carve_v6 dc1 provider GUA /60 at :10") -check("2602:f3e2:f02:11::/64" in addrs1, "carve_v6 dc1 provider API-VIP /64 at :11") -check("fd12:3456:789a:220::/64" in addrs1, "carve_v6 dc1 metal-admin ULA /64 at DC.NN :220") -check("fd12:3456:789a:221::/64" in addrs1, "carve_v6 dc1 metal-internal ULA /64 at :221") -check("fd12:3456:789a:230::/64" in addrs1, "carve_v6 dc1 data-tenant ULA /64 at :230") -check("fd12:3456:789a:250::/64" in addrs1, "carve_v6 dc1 replication ULA /64 at :250") +check("2602:f3e2:f02:10::/60" in addrs1, "carve_v6 VR1-DC0 provider GUA /60 at :10") +check("2602:f3e2:f02:11::/64" in addrs1, "carve_v6 VR1-DC0 provider API-VIP /64 at :11") +check("fd12:3456:789a:220::/64" in addrs1, "carve_v6 VR1-DC0 metal-admin ULA /64 at DC.NN :220") +check("fd12:3456:789a:221::/64" in addrs1, "carve_v6 VR1-DC0 metal-internal ULA /64 at :221") +check("fd12:3456:789a:230::/64" in addrs1, "carve_v6 VR1-DC0 data-tenant ULA /64 at :230") +check("fd12:3456:789a:250::/64" in addrs1, "carve_v6 VR1-DC0 replication ULA /64 at :250") m60 = ipaddress.ip_network("fd12:3456:789a:220::/60") check(ipaddress.ip_network("fd12:3456:789a:221::/64").subnet_of(m60), "carve_v6 metal-internal /64 sits inside metal /60") -# dc2 uses the :3NN nibble; the two DCs never overlap +# VR1 DC1 uses the :3NN nibble; the two DCs never overlap addrs2 = {c for c, _, _ in recs2} -check("fd12:3456:789a:320::/64" in addrs2, "carve_v6 dc2 metal-admin at :320 (DC nibble 3)") -check("2602:f3e2:f03:10::/64" in addrs2, "carve_v6 dc2 provider GUA at f03:10") +check("fd12:3456:789a:320::/64" in addrs2, "carve_v6 VR1-DC1 metal-admin at :320 (DC nibble 3)") +check("2602:f3e2:f03:10::/64" in addrs2, "carve_v6 VR1-DC1 provider GUA at f03:10") no_overlap = not any( ipaddress.ip_network(a).overlaps(ipaddress.ip_network(b)) for a, _, _ in recs1 for b, _, _ in recs2 @@ -217,63 +217,111 @@ fake1 = fake_pynetbox.FakeApi() fake1.preseed_roles(T.PLANE_ORDER) -os.environ["DC_GUA_PREFIX"] = "2602:f3e2:f02::/48" +os.environ["DC_GUA_PREFIX"] = "2602:f3e2:f02::/48" # apex: vr1-dc0 +os.environ.pop("DC2_V4_SUPERNET", None) # retired by D-117 +os.environ.pop("DC1_V4_SUPERNET", None) +# --- D-117: DRY RUN IS THE DEFAULT. This is the guard that matters most: the +# --- tool used to write with no flag at all, straight into the IPAM apex. +with captured_stdout() as out0: + rc0 = run_main(["--dc", "dc0"], fake1) +check(rc0 == 0, "main dc0 DRY RUN rc==0", str(rc0)) +check(out0.getvalue().count("CREATED prefix") == 0, + "D-117: dry run (no --commit) CREATES NOTHING") +check(out0.getvalue().count("[dry-run] would CREATE prefix") == 18, + "D-117: dry run PLANS all 18 prefixes", + str(out0.getvalue().count("[dry-run] would CREATE prefix"))) +check("[dry-run] would CREATE site" in out0.getvalue(), + "D-117: dry run PLANS the site too (fresh-NetBox preview must not bail)") +check("DRY RUN" in out0.getvalue(), "D-117: dry run says so, loudly") +check(len(fake1.ipam.prefixes._items) == 0, + "D-117: dry run left the NetBox EMPTY (no side effects)", + str(len(fake1.ipam.prefixes._items))) + +# --- now commit for real with captured_stdout() as out1: - rc1 = run_main(["--dc", "dc1"], fake1) -check(rc1 == 0, "main dc1 first run rc==0", str(rc1)) + rc1 = run_main(["--dc", "dc0", "--commit"], fake1) +check(rc1 == 0, "main dc0 --commit rc==0", str(rc1)) created_count = out1.getvalue().count("CREATED prefix") -check(created_count == 18, "main dc1 first run creates 18 prefixes (6 v4 + 12 v6)", str(created_count)) -check("vr0-dc0" in out1.getvalue(), "main dc1 prints the vr0-dc0 collision note") +check(created_count == 18, "main dc0 --commit creates 18 prefixes (6 v4 + 12 v6)", str(created_count)) +check("vr0-dc0" in out1.getvalue(), "main dc0 prints the vr0-dc0 (VR0 rehearsal) collision note") +dc0_site = fake1.dcim.sites.get(slug="vr1-dc0") +check(dc0_site is not None, "D-117: --dc dc0 created the APEX site vr1-dc0 (NOT vr1-dc1)") +check(fake1.dcim.sites.get(slug="vr1-dc1") is None, + "D-117: --dc dc0 did NOT touch vr1-dc1 -- the off-by-one is dead") with captured_stdout() as out2: - rc2 = run_main(["--dc", "dc1"], fake1) -check(rc2 == 0, "main dc1 second run (idempotent) rc==0") -check(out2.getvalue().count("CREATED prefix") == 0, "main dc1 second run creates nothing new") -check(out2.getvalue().count("EXISTS prefix") == 18, "main dc1 second run reports all 18 as EXISTS") + rc2 = run_main(["--dc", "dc0", "--commit"], fake1) +check(rc2 == 0, "main dc0 second run (idempotent) rc==0") +check(out2.getvalue().count("CREATED prefix") == 0, "main dc0 second run creates nothing new") +check(out2.getvalue().count("EXISTS prefix") == 18, "main dc0 second run reports all 18 as EXISTS") # ----------------------------------------------------------------------------- -# main() end-to-end -- dc2 happy path (fresh site, distinct v4/GUA) +# main() end-to-end -- VR1 DC1 (the SECOND DC): fresh site, distinct v4/GUA # ----------------------------------------------------------------------------- fake2 = fake_pynetbox.FakeApi() fake2.preseed_roles(T.PLANE_ORDER) -os.environ["DC_GUA_PREFIX"] = "2602:f3e2:f03::/48" -os.environ["DC2_V4_SUPERNET"] = "10.13.0.0/19" +os.environ["DC_GUA_PREFIX"] = "2602:f3e2:f03::/48" # apex: vr1-dc1 +os.environ["DC1_V4_SUPERNET"] = "10.12.64.0/19" # D-115 with captured_stdout() as out3: - rc3 = run_main(["--dc", "dc2"], fake2) -check(rc3 == 0, "main dc2 run rc==0", str(rc3)) -check(out3.getvalue().count("CREATED prefix") == 18, "main dc2 run creates 18 prefixes", str(out3.getvalue().count("CREATED prefix"))) -dc2_site = fake2.dcim.sites.get(slug="vr1-dc2") -check(dc2_site is not None, "main dc2 creates the vr1-dc2 site") + rc3 = run_main(["--dc", "dc1", "--commit"], fake2) +check(rc3 == 0, "main dc1 (second DC) run rc==0", str(rc3)) +check(out3.getvalue().count("CREATED prefix") == 18, "main dc1 creates 18 prefixes", + str(out3.getvalue().count("CREATED prefix"))) +check(fake2.dcim.sites.get(slug="vr1-dc1") is not None, "main dc1 binds to the APEX site vr1-dc1") +check(fake2.dcim.sites.get(slug="vr1-dc2") is None, "D-117: the invented vr1-dc2 site is NEVER created") # ----------------------------------------------------------------------------- -# main() -- dc2 with DC2_V4_SUPERNET missing FAILS LOUD +# main() -- second DC with DC1_V4_SUPERNET missing FAILS LOUD # ----------------------------------------------------------------------------- -del os.environ["DC2_V4_SUPERNET"] +del os.environ["DC1_V4_SUPERNET"] fake3 = fake_pynetbox.FakeApi() fake3.preseed_roles(T.PLANE_ORDER) try: - run_main(["--dc", "dc2"], fake3) - no("main dc2 without DC2_V4_SUPERNET fails loud", "did not raise SystemExit") + with captured_stdout(): + run_main(["--dc", "dc1", "--commit"], fake3) + no("main dc1 without DC1_V4_SUPERNET fails loud", "did not raise SystemExit") except SystemExit: - ok("main dc2 without DC2_V4_SUPERNET fails loud") + ok("main dc1 without DC1_V4_SUPERNET fails loud") # ----------------------------------------------------------------------------- -# argparse: missing/invalid --dc +# D-117: the RETIRED env var name is rejected, not silently ignored +# ----------------------------------------------------------------------------- +os.environ["DC2_V4_SUPERNET"] = "10.12.64.0/19" # muscle memory +os.environ.pop("DC1_V4_SUPERNET", None) +try: + with captured_stdout(): + run_main(["--dc", "dc1", "--commit"], fake_pynetbox.FakeApi()) + no("D-117: retired DC2_V4_SUPERNET is rejected by name") +except SystemExit: + ok("D-117: retired DC2_V4_SUPERNET is rejected by name") +os.environ.pop("DC2_V4_SUPERNET", None) + +# ----------------------------------------------------------------------------- +# argparse: missing/invalid --dc (dc2 is now an INVALID choice -- D-117) # ----------------------------------------------------------------------------- fake4 = fake_pynetbox.FakeApi() fake4.preseed_roles(T.PLANE_ORDER) -os.environ["DC2_V4_SUPERNET"] = "10.13.0.0/19" +os.environ["DC1_V4_SUPERNET"] = "10.12.64.0/19" try: - run_main([], fake4) + with captured_stdout(): + run_main([], fake4) no("main missing --dc fails loud") except SystemExit: ok("main missing --dc fails loud") try: - run_main(["--dc", "dc3"], fake4) + with captured_stdout(): + run_main(["--dc", "dc2"], fake4) + no("D-117: --dc dc2 (the old off-by-one label) is REJECTED") +except SystemExit: + ok("D-117: --dc dc2 (the old off-by-one label) is REJECTED") + +try: + with captured_stdout(): + run_main(["--dc", "dc3"], fake4) no("main invalid --dc choice fails loud") except SystemExit: ok("main invalid --dc choice fails loud") @@ -282,6 +330,30 @@ # Summary # ----------------------------------------------------------------------------- print() + +# --------------------------------------------------------------------------- +# D-117 REGRESSION GUARDS. These are the whole reason this rename happened: +# the tool bound VR1 DC0's prefixes to the site vr1-dc1 (the OTHER DC), and it +# WROTE BY DEFAULT, so it would have landed silently in the IPAM apex. +# The slugs below are the APEX's, measured live 2026-07-13. Do not "fix" them. +# --------------------------------------------------------------------------- +check(T.SITES["dc0"]["slug"] == "vr1-dc0", "D-117: --dc dc0 binds to site vr1-dc0") +check(T.SITES["dc1"]["slug"] == "vr1-dc1", "D-117: --dc dc1 binds to site vr1-dc1") +check("dc2" not in T.SITES, "D-117: the off-by-one 'dc2' selector is GONE") +check(T.DC_V6_INDEX["dc0"] == 0x02, "D-117: VR1 DC0 -> GUA site nibble 02 (== apex vr1-dc0 f02::/48)") +check(T.DC_V6_INDEX["dc1"] == 0x03, "D-117: VR1 DC1 -> GUA site nibble 03 (== apex vr1-dc1 f03::/48)") + +# dry-by-default: --commit must exist, and the writers must take a commit flag. +import inspect +_src = inspect.getsource(T) +check("--commit" in _src, "D-117: --commit flag exists") +check("commit" in inspect.signature(T.create_or_report_prefix).parameters, + "D-117: create_or_report_prefix is commit-gated") +check("commit" in inspect.signature(T.find_or_create_site).parameters, + "D-117: find_or_create_site is commit-gated (a site is a WRITE)") +check("DC2_V4_SUPERNET" in _src and "RETIRED by D-117" in _src, + "D-117: the retired DC2_V4_SUPERNET env var is rejected by name, not ignored") + if F == 0: print(f"ALL PASS ({P} checks)") sys.exit(0)