Newer
Older
openstack-caracal-dc-dc / docs / changelog-20260712-opnsense-edge-boot-fixes.md

Changelog 2026-07-12 -- DOCFIX-187: opnsense-edge module first-boot fixes (boot still OPEN)

Context. First real boot of the Office1 OPNsense edge. Several correct-but-necessary module fixes were made while debugging; the boot is NOT yet resolved (a BTX-loader triple-fault remains -- see docs/incident-20260712-opnsense-edge-boot-triplefault.md). These changes are kept because a working config needs them regardless.

opentofu/modules/opnsense-edge/main.tf

  1. Serial console added (devices.serials, file-backed to /var/lib/libvirt/vr1/staging/${vm_name}-serial.log). OPNsense nano is serial-only; the module had no console, so the guest had no output and faulted with nothing to show. This got the boot from a blank early fault to the loader stage (and gives boot-log capture).
  2. machine q35 -> pc (i440fx). FreeBSD's legacy BTX loader is far more commonly booted on i440fx; q35 is a known early-fault source. (Did not fix this incident, but is the safer, documented choice.)
  3. Disk: COW overlay -> direct per-VM copy (create.content.url = base_volume_path, no backing_store). Matches the documented virt-install --import <nano.qcow2> flow; the nano is designed to boot + auto-expand in place, not through a read-only backing. var.disk_size_bytes is now unused (nano is pre-grown by opnsense-prep-image.sh).
  4. CPU host-passthrough with AMD svm disabled (cpu.features = [{name=svm, policy=disable}]). The documented AMD nested-virt fix (-cpu host,-svm). NOTE: the provider key is features (plural) -- feature validates but is silently dropped.

Important operational note discovered

  • machine and cpu are create-time; the dmacvicar provider plans them as an in-place "change" that libvirt does NOT apply to an existing domain. To actually change them: virsh destroy <dom>; virsh undefine <dom>; tofu apply (recreate). tofu apply -replace hit "domain already exists" mid-replace this session.

Status

  • Module tofu validate -> Success. NOT booting: OPNsense triple-faults at the BTX loader regardless of the above (incident report has the full trail + ranked next steps).

Revert

  • git checkout opentofu/modules/opnsense-edge/main.tf to the pre-DOCFIX-187 revision. (Reverts the serial console + i440fx + direct-disk + svm-disable together.)