| 2026-07-12 |
DOCFIX-188: root-cause the OPNsense boot triple-fault -- libvirt memory_unit (guest had 2 MiB)
...
The 2026-07-12 triple-fault was never a CPU/nesting/machine/console problem.
dmacvicar/libvirt >=0.9 reinterprets `memory` as raw libvirt units (KiB), not
MiB (the 0.8-era meaning the modules were authored against). With no
`memory_unit`, `memory = 2048` rendered `<memory unit='KiB'>2048</memory>` ->
QEMU `-m size=2048k` = 2 MiB. boot2 fits in 2 MiB and echoes /boot.config;
/boot/loader does not -- hence the deterministic 262-byte fault, immune to every
knob the prior window turned.
Measured every hop: module input -> tofu state -> domain XML -> virsh dominfo
(Max memory: 2048 KiB) -> live QEMU cmdline.
FOUNDATIONAL: the identical defect was latent in all three VM modules --
opnsense-edge (live blocker), cloudinit-vm (MAAS/NetBox/GitBucket, the next
step), node-vm (Stage 3). Every VR1 VM would have gotten 1024x too little RAM.
- memory_unit = "MiB" on all three libvirt_domain resources (memory_mib = 2048
value untouched -- it now means the 2 GiB always intended).
- Corrected the disproven "svm is ROOT CAUSE" comment in opnsense-edge; the
setting is retained as legitimate hardening, the false claim is not.
- Guard: opentofu-validate.sh S1 + --check-memory-unit flag. `tofu validate`
cannot catch this (memory_unit is optional => omitting it is schema-valid,
which is exactly how it shipped). Harness T3 is a true negative test: the
fixture IS the shipped defect. Harness 1 -> 4 PASS.
- Incident report gains a READ-THIS-FIRST box so nobody re-chases the CPU theories.
Gauntlet ALL GREEN (52 harnesses); repo-lint 0 fail; tofu fmt/validate clean.
Live boot verification remains a separately-gated operator step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y117t1F525ba9r1vXSBETk
|
DOCFIX-187: opnsense-edge module boot fixes + boot-triple-fault incident report
...
Office1 OPNsense edge first-boot debugging. Several correct module fixes made;
the boot is NOT yet resolved (BTX-loader triple-fault under double-nested virt),
documented as an OPEN incident for the next session.
Module (opentofu/modules/opnsense-edge/main.tf), all kept (a working config
needs them, none resolved the fault):
- serial console (nano is serial-only; the module had none) + boot-log capture
- machine q35 -> i440fx (pc)
- disk COW-overlay -> direct per-VM copy (matches virt-install --import)
- CPU host-passthrough with AMD svm disabled (documented AMD nested-virt fix;
note the provider key is `features` plural, not `feature`)
Incident report docs/incident-20260712-opnsense-edge-boot-triplefault.md: full
symptom (boot2 echoes /boot.config then triple-faults loading /boot/loader, a
consistent 262-byte fault confirmed fresh each boot), environment (double-nested
AMD Opteron_G3), everything tried, and ranked next steps (full CPU flags
+kvm_pv_eoi/+kvm_pv_unhalt, add a video device for the -D dual console, mem 4G,
UEFI/OVMF, outer-hypervisor CPU). Ledger checkpoint added; session at context
limit.
repo-lint 0 fail; module tofu validate Success.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
| 2026-07-10 |
DOCFIX-148: OpenTofu module audit pass + provider-docs skill reference
...
Audited every "UNVERIFIED"/"inferred"/"assumed" marker across all 9 modules
built this session rather than leaving them as-is. Found and fixed a real
documentation error: opnsense-edge's comments claimed devices.interfaces
list order directly sets OPNsense's LAN/WAN role. Researching OPNsense's own
interface-assignment model showed this conflates two things -- list order
plausibly controls which vtnetN device number a NIC gets (libvirt's own
docs: auto-assigned PCI addresses "usually match" XML order for a simple
topology), but the actual LAN/WAN role is a separate, explicit mapping set
inside config.xml itself ("vtnet0=WAN" is a convention some guides choose,
not an enforced rule). Corrected main.tf's comments and both
lan_network_name/wan_network_name variable descriptions -- no config.xml
exists in this repo yet, so this fixes documentation accuracy for whoever
writes it next, not current behavior.
Confidence upgraded (not just re-hedged) on three other items: node-vm's
boot={order=N} shape (confirmed the provider 1:1-mirrors libvirt's own XML,
and libvirt's own docs confirm the single-attribute shape), create.content.url
accepting local paths (second independent source), and the genisoimage
flags (confirmed standard, cleanly separated from the still-open question of
whether OPNsense's Importer actually reads the result). Confirmed safe:
maas_vm_host's zone/pool computed-if-unset behavior.
Added .claude/skills/openstack-cloud-ops/references/opentofu-provider-docs.md:
indexes every provider/doc source used this session with exact URLs and
confirmed versions, plus the fetch methodology that actually worked
(Registry doc pages are JS-rendered, use GitHub instead; branch names vary,
check default_branch via the API; real example .tf files beat doc-summarized
prose for syntax questions). Wired into SKILL.md's routing table,
cross-referenced from opentofu/README.md in both directions without
duplicating content.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
DOCFIX-146: opnsense-edge module + nano-image/config-ISO prep scripts
...
Implements the mechanism identified in today's OPNsense deployment research
(cloud-init confirmed unreliable on FreeBSD; the real native mechanism is
OPNsense's Configuration Importer, which scans an attached volume for
/conf/config.xml and gained ISO9660 support specifically for VM/cloud
automation).
scripts/opnsense-prep-image.sh: downloads the nano image, decompresses,
converts raw->qcow2, resizes -- work create.content.url almost certainly
can't do itself (a plain fetch, no decompression). No hardcoded mirror URL;
OPNSENSE_MIRROR_BASE is required explicitly.
scripts/opnsense-build-config-iso.sh: builds the ISO9660 image containing
/conf/config.xml that the Configuration Importer picks up.
opentofu/modules/opnsense-edge: wires both together, mechanically identical
to modules/cloudinit-vm's cdrom-attach shape but with no libvirt_cloudinit_disk
resource (wrong format for this). Explicit lan_network_name/wan_network_name
variables rather than an ordered list, since OPNsense's interface-1-is-LAN
convention makes list-ordering a real place for a silent mistake to hide. No
PXE boot-order attribute used -- this VM boots from its own disk, not the
network, so node-vm's flagged-unverified boot field doesn't apply here.
Both new scripts' harnesses (4/4, 3/3) exercise REAL missing-tool guards --
qemu-img/genisoimage/xorriso are all genuinely absent from this environment.
Two things flagged, not presented as fact: whether the Importer's ISO9660
behavior holds on a real boot, and whether interface list-order reliably
maps to LAN/WAN NIC enumeration for this guest. config_iso_path has no
default -- real config.xml content per site is still undesigned. Not
instantiated in root main.tf.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QrcJx8TUar7pYAvpGJw57A
|
