| 2026-07-14 |
Fix the WAF User-Agent gap: the two pynetbox importers could never write upstream
...
A C2 blocker, measured 2026-07-14: upstream netbox.baldurkeep.com 403s the default
Python User-Agent (references/platform-traps.md) -- curl->200, urllib/pynetbox->403.
The stdlib tools (prod-draft-dump/sandbox-seed/d115-office-carve) already send an
accepted UA, but roles-aggregates-import.py and dc-dc-prefixes-import.py built
pynetbox.api() with the DEFAULT UA and never overrode it. So they would 403 against
upstream -- they could only ever have written to the WAF-less sandbox, never to the
apex C2 must feed. The gap was invisible because every real run so far hit the sandbox.
Fix: get_nb() sets nb.http_session.headers["User-Agent"] = "curl/8.5.0" in both
(same value/rationale as the stdlib tools; pynetbox exposes its requests.Session).
Tests: prefixes-import 85->86 (the fake now has .http_session; the UA is asserted at
RUNTIME, not by source grep). roles-aggregates 19->20. GAUNTLET ALL GREEN (58).
Still open for C2 (logged in the changelog): pynetbox is NOT on vcloud, so the two
importers must run on office1-netbox (has pynetbox 7.0.0); the sandbox re-verify
needs the operator-held token; the upstream write stays operator-gated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
NetBox write-path hardening: the bugs that would have corrupted the apex at C2
...
Found by adversarial review during D-119; independent of the naming work. These
are what would have bitten during C2 (the production IPAM write Stage 2 needs).
1. roles-aggregates-import.py STILL died half-way through a production write. The
preflight added on 2026-07-13 -- after this same script committed 4 roles, 400'd
on the 5th and left the apex half-populated -- only covered role-NAME collisions.
The ARIN RIR lookup and validate_ula_48() both still ran BELOW the first write,
so an apex without ARIN, or a typo'd ORG_ULA_48, would commit 5 roles (and the
RIRs) and THEN die, with no rollback. The exact failure the preflight was added
to prevent, still armed one section lower. The RIR half of the preflight loop was
a literal no-op. Both checks moved INTO the preflight.
A preflight that does not cover every die() downstream of the first write is not
a preflight.
2. sandbox-fidelity-check.py could return a FALSE GREEN -- and it is the script we
have been citing as PROOF the sandbox is a faithful replica. It field-compared
only SHARED objects; the planned delta was checked merely as a SUBSET of
EXPECTED_NEW. With ZERO of the 36 DC prefixes created, extra={} -> unexpected={}
-> it printed "Nothing lost, nothing stray" and exited 0. EXPECTED_NEW was an
upper bound masquerading as an assertion. It is now BOTH bounds, and delta
prefixes are checked for scope and role (the 17-prefix scope-drop bug relocated
into the delta, where the old check was structurally blind).
3. dc-dc-prefixes-import.py died mid-loop on a missing role -- find_role() was
resolved lazily INSIDE the write loop. Against upstream as it stands TODAY (no
six-plane roles), a --commit creates the site + 4 prefixes then dies on the 5th:
a half-written datacenter. Now a whole-plan preflight resolves every role first.
4. The test fake was KINDER THAN REALITY at the failure point: fake_pynetbox.get()
returned matches[0] on a multi-match where real pynetbox RAISES. NetBox permits
duplicate prefixes (the importer's own docstring anticipates them vs vr0-dc0), so
this would blow up in production while the harness sailed through. It now raises.
TESTS: tests/sandbox-fidelity-check/ is NEW (10/10) -- the checker shipped with NO
harness at all, which is exactly why the false green survived; T3 reproduces it, and
T1 derives its expectation from the checker's own EXPECTED_NEW rather than re-typing
it. roles-aggregates-import 16->19, asserting BY POSITION that ARIN and the ULA are
validated before the first create (the old harness only checked the WORD "PREFLIGHT"
appeared -- green while the landmine was armed).
GAUNTLET ALL GREEN (58). repo-lint 0 fail. Zero NetBox writes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
D-119: region-qualify the VR1 DC namespace -- C3 / gap #19 CLOSED (Stage 3 unblocked)
...
THE BUG: the token `dc0` meant TWO DIFFERENT CLOUDS depending on the file --
VR0's LIVE testcloud in scripts/lib-net.sh, VR1's FIRST DC in the NetBox
importer. One string, two clouds, one of them in production.
THE RULING (operator): the repo adopts the apex's names verbatim.
vr0-dc0 VR0's DC0 the LIVE testcloud (a DIFFERENT region)
vr1-dc0 VR1's FIRST DC GUA 2602:f3e2:f02::/48
vr1-dc1 VR1's SECOND DC GUA 2602:f3e2:f03::/48
Bare dc0/dc1/dc2 are now REJECTED LOUDLY everywhere.
D-119 COMPLETES D-117, it does not reverse it: it executes D-117's own amendment
across the three surfaces D-117 never touched. ZERO NetBox writes -- the apex was
already correct and self-consistent (MEASURED: f02::/48 -> vr1-dc0, f03::/48 ->
vr1-dc1); the REPO was the only surface out of step. Renaming the apex to match
the repo was considered and REJECTED (production IPAM write; VR1 would become the
only 1-indexed region; and vr1-dc1 would mean VR1's FIRST DC while vr0-dc1 means
VR0's SECOND).
THE PRIZE: the importer's DC->site map is now an IDENTITY -- there is no offset
table left to get wrong, and an assert enforces it. The original defect was
precisely a WRONG LOOKUP TABLE. The bug class is deleted, not defended.
TWO REAL BUGS the naming fix ALONE would NOT have closed (found by review sweep):
- BREAK-1: descriptions were built with f"VR1 {dc.upper()} ..." -- under D-119 that
renders "VR1 VR1-DC0 provider-public" on all 36 prefixes. Same class as the
original bug (deriving a label by munging a token), hidden in the description
field where slug-focused review missed it. Now looked up from SITES[name].
- BREAK-2 (the important one): DC_GUA_PREFIX was NEVER cross-checked against --dc.
`--dc vr1-dc0 DC_GUA_PREFIX=<f03>` was ACCEPTED: it writes the SECOND DC's GUA,
carved with the FIRST DC's ULA nibble, scoped to the FIRST DC's site -- a
silently mis-bound datacenter assembled from two disagreeing sources. Identity-
mapping the slug leaves the ADDRESSING free-floating. Now guarded by EXPECTED_GUA.
ALSO: vdc1/vdc2 -> vvr1-dc0/vvr1-dc1 (D-114 amendment). D-114's own DR primitive is
`virsh destroy vdc1`, which read as "destroy VR1 DC1" while MEANING "destroy VR1
DC0". A mislabelled destroy command in a DR drill is not cosmetic. Neither VM is
built, so it is free. voffice1/Office1 KEEP their number (operator ruling).
Env var: DC1_/DC2_V4_SUPERNET -> VR1_DC1_V4_SUPERNET (both old names rejected).
rbd-mirror's --site-name aligned: `--dc vr1-dc0 --site-name dc1` was a re-created
two-namespace collision on one command line.
TESTS: dc-selector 21->30, prefixes-import 40->82 (identity invariant; EXPECTED_GUA
in BOTH mismatch directions; mismatched run writes NOTHING; matching pair still
succeeds; no munged description). GAUNTLET ALL GREEN (57). repo-lint 0 fail.
The old harnesses PINNED THE WRONG MAPPING -- they would have gone green while
enforcing the bug.
THE tofu apply IS NOT DONE -- IT IS GATED. Plan: 11 to add, 0 to change, 11 to
destroy (libvirt object names are ForceNew; moved{} makes the plan reviewable but
cannot suppress a replace). MEASURED SAFE: all 11 are EMPTY -- no guests, no
volumes, Stage 3 hasn't run. Office1's live objects are not in the plan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
D-117 ADOPTED (B, DC-only): fix the off-by-one site binding + make the importer dry-by-default
...
Operator ruling: rename the DC numbers to match the NetBox apex; the office
keeps its number (Office1 -> new site vr1-off1, so nothing deployed changes).
netbox/dc-dc-prefixes-import.py:
--dc dc1|dc2 -> --dc dc0|dc1, bound to the APEX slugs measured live:
dc0 -> vr1-dc0 (2602:f3e2:f02::/48) dc1 -> vr1-dc1 (f03::/48)
It previously mapped dc1 -> vr1-dc1 while treating f02::/48 as dc1, i.e. it
would have written VR1 DC0's prefixes to the OTHER DC's site.
Now DRY BY DEFAULT with --commit (it used to write with no flag at all).
A dry run against a FRESH NetBox now plans the site too (_PlannedSite) rather
than bailing with 'no site id' -- previewing an empty apex is the whole point.
DC2_V4_SUPERNET is REJECTED BY NAME (retired) rather than silently ignored.
Every 'DCn' in prose is region-qualified: 'DC0' meant VR0's rehearsal DC in
some sentences and VR1's first DC in others. No sed could have caught that.
tests/dc-dc-prefixes-import/: 73 checks ALL PASS, incl. regression guards that
the dry run creates NOTHING, that --dc dc0 never touches vr1-dc1, and that the
invented vr1-dc2 site is never created.
FOUND, RECORDED, NOT YET FIXED (D-117 amendment): scripts/lib-net.sh carries a
SECOND, colliding $DC namespace where dc0 already means VR0's DC0. Renaming
VR1's dc1->dc0 there would collide with VR0. Proposed fix: region-qualify the
shell selector to the apex slugs (vr0-dc0/vr1-dc0/vr1-dc1). Does not block the
import; the two selectors are NOT interchangeable meanwhile.
repo-lint 0 fail. run-tests-all: ALL GREEN (53 harnesses).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ZBxzAbmqwLW7jEmMG23Xw
|
| 2026-07-11 |
D-111: align VR1 v6 subcarve to deployed NN mnemonic; fix ULA-gen doc bug
...
Basing VR1 NetBox on the real netbox.baldurkeep.com modeling (operator
directive). A read-only live export confirmed VR0-DC0/Willamette use an NN
net-byte mnemonic with /60-per-plane + /64-active; dc-dc-prefixes-import.py
did not match it (arbitrary contiguous indices, /64-only).
DOCFIX-182 / D-111: carve_v6 replaces carve_ula/carve_gua -- provider :10
(+API-VIP :11) GUA, metal /60 shared by admin :20 / internal :21, data :30,
storage :40, repl :50, all /60+/64 ULA; the ULA /56 is indexed to the GUA site
nibble so the 4th hextet reads DC.NN in both families. Direct-math _sub_at (no
subnet enumeration -- preserves the DOCFIX-181 no-hang property). Harness
40 -> 53 PASS. D-111 ADOPTED in design-decisions.md; scope-doc sub-decision #1
ratified.
DOCFIX-183: fix the invalid-IPv6 ULA-generation sed in the netem/ULA proposal
doc (4+6 -> 2+4+4 hextets, caught by actually running it); ratified VR1 ULA is
fd50:840e:74e2::/48.
repo-lint 0 fail; tests/dc-dc-prefixes-import 53/53.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
| 2026-07-10 |
DOCFIX-181: fix carve_gua subnet-enumeration hang (gauntlet was stalling)
...
netbox/dc-dc-prefixes-import.py::carve_gua() list()-ed every /64 subnet of the
GUA prefix (2**24 = ~16.7M for D-101's /40 example) only to use the first ->
hung/OOM. Real import runs would hang identically, not just the test. Fixed to
take the first /64 lazily via next() (O(1)).
Added a faulthandler watchdog to tests/dc-dc-prefixes-import/test_logic.py so a
future blocking regression self-aborts with a traceback after 30s instead of
silently stalling the whole run-tests-all.sh gauntlet -- the false-green that
let the ledger claim "40/40" for a test that never actually completed.
tests/dc-dc-prefixes-import: 40/40 in 0.05s (was: hung indefinitely).
Full gauntlet: ALL GREEN (52 harnesses). repo-lint 0 fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
DOCFIX-160: adversarial review pass finds and fixes 3 real issues
...
Ran a fresh-eyes review subagent across tonight's full deliverable set
(lib-net.sh/lib-hosts.sh selectors, the NetBox DC-DC pipeline, all seven
Stage 1-7 runbooks). Found and fixed:
- A real Python control-flow bug in tests/dc-dc-prefixes-import/
test_logic.py: a `return` inside `try` made the `else` clause
unreachable, silently leaving 3 happy-path assertions uncounted.
37/37 -> 40/40 after the fix (independently reproduced before trusting
the finding).
- A self-contradictory pair of comments in dc-dc-prefixes-import.py about
/19-vs-/22 subnet arithmetic (no functional bug, wrong prose).
- Stale self-referential text in the Stage 4 runbook, written before
Stage 3 existed and never reconciled once it did.
Corrected the 40/40 count in the workflow tracker and session ledger.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
DOCFIX-152: NetBox multi-DC/dual-stack import pipeline (gap #3 mechanism)
...
Adds netbox/dc-dc-prefixes-import.py, extending the v1 single-site IPv4-only
NetBox import to VR1's two-DC dual-stack model (D-101). DC1's v4 planes are
hardcoded (D-101: inherited from DC0 unchanged, this is decision text, not
an inference); DC2's v4 supernet, the org ULA /48, and the per-DC GUA carve
are all required env vars with no defaults, failing loud rather than
inventing any of them. 37/37 tests against a fake in-memory NetBox client
(no live NetBox reachable this session). Closes the tooling half of gap #3;
the real literals still need operator/NetBox assignment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
