| 2026-07-14 |
D-119: region-qualify the VR1 DC namespace -- C3 / gap #19 CLOSED (Stage 3 unblocked)
...
THE BUG: the token `dc0` meant TWO DIFFERENT CLOUDS depending on the file --
VR0's LIVE testcloud in scripts/lib-net.sh, VR1's FIRST DC in the NetBox
importer. One string, two clouds, one of them in production.
THE RULING (operator): the repo adopts the apex's names verbatim.
vr0-dc0 VR0's DC0 the LIVE testcloud (a DIFFERENT region)
vr1-dc0 VR1's FIRST DC GUA 2602:f3e2:f02::/48
vr1-dc1 VR1's SECOND DC GUA 2602:f3e2:f03::/48
Bare dc0/dc1/dc2 are now REJECTED LOUDLY everywhere.
D-119 COMPLETES D-117, it does not reverse it: it executes D-117's own amendment
across the three surfaces D-117 never touched. ZERO NetBox writes -- the apex was
already correct and self-consistent (MEASURED: f02::/48 -> vr1-dc0, f03::/48 ->
vr1-dc1); the REPO was the only surface out of step. Renaming the apex to match
the repo was considered and REJECTED (production IPAM write; VR1 would become the
only 1-indexed region; and vr1-dc1 would mean VR1's FIRST DC while vr0-dc1 means
VR0's SECOND).
THE PRIZE: the importer's DC->site map is now an IDENTITY -- there is no offset
table left to get wrong, and an assert enforces it. The original defect was
precisely a WRONG LOOKUP TABLE. The bug class is deleted, not defended.
TWO REAL BUGS the naming fix ALONE would NOT have closed (found by review sweep):
- BREAK-1: descriptions were built with f"VR1 {dc.upper()} ..." -- under D-119 that
renders "VR1 VR1-DC0 provider-public" on all 36 prefixes. Same class as the
original bug (deriving a label by munging a token), hidden in the description
field where slug-focused review missed it. Now looked up from SITES[name].
- BREAK-2 (the important one): DC_GUA_PREFIX was NEVER cross-checked against --dc.
`--dc vr1-dc0 DC_GUA_PREFIX=<f03>` was ACCEPTED: it writes the SECOND DC's GUA,
carved with the FIRST DC's ULA nibble, scoped to the FIRST DC's site -- a
silently mis-bound datacenter assembled from two disagreeing sources. Identity-
mapping the slug leaves the ADDRESSING free-floating. Now guarded by EXPECTED_GUA.
ALSO: vdc1/vdc2 -> vvr1-dc0/vvr1-dc1 (D-114 amendment). D-114's own DR primitive is
`virsh destroy vdc1`, which read as "destroy VR1 DC1" while MEANING "destroy VR1
DC0". A mislabelled destroy command in a DR drill is not cosmetic. Neither VM is
built, so it is free. voffice1/Office1 KEEP their number (operator ruling).
Env var: DC1_/DC2_V4_SUPERNET -> VR1_DC1_V4_SUPERNET (both old names rejected).
rbd-mirror's --site-name aligned: `--dc vr1-dc0 --site-name dc1` was a re-created
two-namespace collision on one command line.
TESTS: dc-selector 21->30, prefixes-import 40->82 (identity invariant; EXPECTED_GUA
in BOTH mismatch directions; mismatched run writes NOTHING; matching pair still
succeeds; no munged description). GAUNTLET ALL GREEN (57). repo-lint 0 fail.
The old harnesses PINNED THE WRONG MAPPING -- they would have gone green while
enforcing the bug.
THE tofu apply IS NOT DONE -- IT IS GATED. Plan: 11 to add, 0 to change, 11 to
destroy (libvirt object names are ForceNew; moved{} makes the plan reviewable but
cannot suppress a replace). MEASURED SAFE: all 11 are EMPTY -- no guests, no
volumes, Stage 3 hasn't run. Office1's live objects are not in the plan.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
| 2026-07-13 |
D-116 ADOPTED: no Office1-local GitBucket -- keep using git.baldurkeep.com
...
Operator ruling: "we've decided not to build a separate one on the test cloud. We will continue to
use the current setup and create new repos for deployment tasks as needed."
SUPERSEDES the GitBucket half of D-103 (which gave Office1 three service VMs) and removes GitBucket
from Stage 2's build list entirely. Office1's MAAS-composed service machines are now exactly TWO --
office1-netbox and office1-tailscale -- and both are LIVE.
WHAT THIS ACTUALLY CLOSES, and why it is a simplification rather than a deferral: the Stage 2 runbook
carried an OPEN QUESTION about the Office1-local GitBucket -- "what it mirrors, and under what repo
path, is not decided" -- because a SECOND GitBucket alongside the pre-existing git.baldurkeep.com
raised a real authority/mirroring question nobody had answered. That question is now MOOT. There is
one git service, it is the existing one, and it is authoritative.
Recorded explicitly in D-116 so nobody misreads the scope: D-107's per-DC ARTIFACT mirror (OS/package
artifacts for node provisioning) is a SEPARATE concern and is NOT affected. And if Roosevelt later
wants a site-local git service, that is a fresh decision against real requirements -- not something
this ruling forecloses.
Applied everywhere it was an instruction, not just recorded:
- runbooks/dc-dc-phase1-office1-standup.md: Step 11 replaced with a TOMBSTONE (kept rather than
deleted -- step numbering is referenced elsewhere, and someone WILL come looking after reading
D-103); removed from the sequence, the compose steps, and the open-questions list.
- docs/dc-dc-deployment-workflow.md: dropped from Stage 2's Build and, importantly, from its GATE --
"GitBucket serving" is no longer a gate condition.
- docs/design-decisions.md: D-103 annotated in place with the supersession, so grepping for GitBucket
lands on the amendment rather than the stale instruction.
- docs/vr1-office1-as-built.md + session ledger updated.
repo-lint 0 fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
D-114 follow-through: fix node-vm's nested-virt defect NOW; voffice1 pinned to a reserved address
...
Operator ruling: fixes land as we find them -- only DC1 DEPLOYMENT is gated, not DC1 fixes.
1. modules/node-vm: UNCONDITIONAL `cpu = { mode = "host-passthrough" }`. Same defect just found in
cloudinit-vm: with no cpu block libvirt hands the guest a generic emulated CPU with NO svm. A DC
node runs nova-compute, so it MUST run KVM guests -- there is no correct value other than this,
hence no knob that could only ever be set wrong (contrast cloudinit-vm, where it IS a real per-VM
decision, and opnsense-edge, which disables svm as correct hardening for a router). Not yet
instantiated, so no live call breaks. Without it the DC nodes would have come up looking healthy
and been unable to launch a single instance -- surfacing in Stage 5 as an inscrutable scheduler error.
2. voffice1 moved off the dynamic pool onto a RESERVED address. It first took 10.10.0.100 -- the
first address of Kea's pool (.100-.199). A MAAS region controller must not hold an address that
can move. Added a Kea host reservation via the D-113(a2) REST API (a second live exercise of that
write path): 52:54:00:6a:87:e5 -> 10.10.0.20, deliberately OUTSIDE the pool. add_reservation ->
"saved", service/reconfigure -> "ok". Rebooted; voffice1 came back on 10.10.0.20, .100 released,
/dev/kvm still present. Done now because nothing depends on the address yet.
3. runbooks/dc-dc-phase1-office1-standup.md rewritten to the D-114 model (was: three sibling service
VMs + an unruled Option A/B fork -- it would have actively misdirected whoever ran it). New
sequence: voffice1 via tofu -> first contact (lease/SSH/nested-KVM) -> MAAS on voffice1 -> LXD
(5.21 LTS) -> register LXD as a MAAS VM host -> MAAS-COMPOSE NetBox/GitBucket/Tailscale. The
MAAS/LXD command lines carry explicit PENDING VERIFICATION markers rather than fabricated flags.
docs/dc-dc-deployment-workflow.md Stage 2 + gap register updated to match.
repo-lint 0 fail. opentofu-validate PASS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
SAFETY SWEEP: mark the config.xml push path as a LIVE HAZARD (no behaviour change)
...
After D-113(a2) was proven (Office1's config is now API-managed), every instruction in the repo
that still said "render a config.xml and push it to the edge" became ACTIVELY DANGEROUS: a full
config.xml push replaces /conf/config.xml wholesale and drops ~667 migration-populated elements
(measured), including the only 2 firewall pass rules on the box. Following the old runbook steps
against the live edge would clobber it.
Warnings and headers only. No behaviour changes, no live system touched.
Marked (5 files):
runbooks/dc-dc-phase1-office1-standup.md DANGER banner; the config.xml render + config-ISO
sub-steps removed as RUNNABLE instructions (they
would clobber the live edge). Image-prep retained;
history preserved in git + the build changelog.
runbooks/dc-dc-phase2-tofu-dc-substrate.md STOP banner on Step 4.
scripts/opnsense-render-config.sh DANGER header at point-of-use. NOT dead: under
D-113(a2) it is to be reduced to a MINIMAL bootstrap
render. Until then, safe only for a brand-new,
not-yet-booted edge.
scripts/opnsense-build-config-iso.sh RETIRED header quoting the upstream source proving
the importer can never fire on a nano image.
docs/dc-dc-deployment-workflow.md STALE-CONTENT warning atop the tooling gap register.
THE SWEEP ALSO CAUGHT A PRE-EXISTING LANDMINE, unrelated to today's work: dc-dc-phase2 Step 4
still instructs building a CONFIG ISO for DC1's edge -- but D-112 established that ISO can never
be read (opnsense-importer probes for a read-only root; on a pre-installed nano the root is
writable and a factory /conf/config.xml already exists, so it bootstrap_and_exit 0's without
enumerating a single device). That runbook has been telling anyone who follows it to build an
inert artifact and then wonder why the edge came up on factory defaults -- which is EXACTLY the
day lost on 2026-07-12. It was never corrected at the source. Now it is. Its WAN_IF/LAN_IF
"chicken-and-egg" discussion is moot for the same reason: that problem only exists if you try to
seed a full config before first boot, and D-112(c) measures vtnetN AFTER boot.
Changelogs deliberately NOT touched -- they are history, not instructions. Marking history is
noise; marking instructions is safety.
Verification: opnsense-render-config 24 PASS, opnsense-build-config-iso 2 PASS, opnsense-api
21 PASS; repo-lint 0 fail.
STILL OPEN (the actual D-113(a2) migration): reduce the template to a minimal bootstrap, retire
the config-ISO path and the opnsense-edge module's config_seed/cdrom wiring, and rewrite Stage 3's
edge steps around the API. This sweep makes the repo SAFE, not FINISHED.
Revert: git revert this commit (restores the previous, dangerous instructions verbatim).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6Fzre1CxCY8tzwFCudu57
|
| 2026-07-10 |
DOCFIX-163: Office1-local network module + OPNsense edge ownership (gaps #12+16)
...
New opentofu/modules/office1-network (dc-planes-shaped, one shared network
instead of six), instantiated for real in main.tf -- deliberately not a
reused host bridge, since that would live outside OpenTofu's D-103
inventory and break the virsh net-list --all completeness check used
throughout the runbooks. Decided gap #16: Office1 gets its own OPNsense
edge (a fourth site), owned by Stage 2's runbook (new Step 4b). Found a
deeper cross-site gap while wiring the edge's WAN side (no per-site
ISP-uplink network exists for ANY site yet, DC1/DC2 included) -- logged as
new gap #17 rather than defaulted to a mesh-link, which would contradict
D-100's management-only ruling for those legs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
DOCFIX-154/155/156: Stage 2/3/5 runbooks -- Office1 standup, DC substrate, per-DC bundle
...
Three DC-DC runbooks authored in parallel this evening:
- Stage 2 (Office1 headend standup): MAAS-region/NetBox/GitBucket/Tailscale,
each VM-creation step forking an explicit OpenTofu-blocked vs manual-debt
path rather than picking one silently. Flags a real gap: no Office1-local
network is modeled in opentofu/.
- Stage 3 (OpenTofu builds each DC substrate): DC1-first, DC2 hard-gated on
NetBox literals. Walks every open decision (node sizing, MAAS zone/pool,
config.xml tokens, netem params) and stops rather than inventing values.
- Stage 5 (Juju controller + bundle, per DC): describes adapting the
existing phase-01..08 loop to run twice, not duplicating it. Mandates the
$DC selector convention (DOCFIX-151) before every adapted command block.
Surfaces the undesigned IPv6 family-matrix overlay as blocking work.
All three: not yet executed (prep-only session, no live infrastructure
reachable tonight); repo-lint 0 fail/1 documented warn.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
