# Changelog 2026-07-12 -- DOCFIX-188: ROOT CAUSE of the OPNsense boot triple-fault (libvirt `memory_unit`)

**The 2026-07-12 boot incident is diagnosed. It was never a CPU, nesting, machine-type,
or console problem.** The Office1 OPNsense edge guest was running with **2 MiB of RAM.**

## Evidence (measured this session, read-only, every hop)

| Hop | Measured |
|---|---|
| Module input | `memory_mib = 2048` (`opentofu/main.tf:105`) -- intent: 2 GiB |
| Module wiring | `memory = var.memory_mib` (`modules/opnsense-edge/main.tf:82`), **no `memory_unit`** |
| tofu state | `memory = 2048` |
| Rendered libvirt XML | `<memory unit='KiB'>2048</memory>` |
| `virsh dominfo` | `Max memory: 2048 KiB` |
| **Actual QEMU cmdline** | **`-m size=2048k`** -- i.e. **2 MiB** |

**Why it presents as a BTX triple-fault:** FreeBSD's `boot2` is tiny and fits in 2 MiB, so
it happily echoes `/boot.config` (the deterministic 262-byte serial capture) -- and then
triple-faults the instant it hands off to `/boot/loader`, which does not fit. This is
exactly why the fault was immune to every CPU/machine/disk/console change tried on
2026-07-12: none of them touched the cause.

## The defect

`dmacvicar/libvirt` **>= 0.9** changed `memory` semantics. Per the provider's own schema
(`tofu providers schema -json`):

- `memory` -- *"interpreted in libvirt memory units (typically **KiB** unless a unit is
  specified elsewhere)"*
- `memory_unit` -- *"Sets the unit for the domain's main memory value ... KiB, MiB, or GiB"*

The old 0.8-era meaning of `memory` was MiB. The modules were authored against that
assumption. With no `memory_unit`, libvirt defaults to KiB, so **every VM this tree creates
gets 1024x too little RAM.**

## Fix -- all three VM modules (identical one-line defect)

`memory_unit = "MiB"` added to the `libvirt_domain` resource in each:

1. `opentofu/modules/opnsense-edge/main.tf` -- the Office1 edge (**the live blocker**).
2. `opentofu/modules/cloudinit-vm/main.tf` -- MAAS / NetBox / GitBucket VMs (Stage 2, the
   **next** thing to be built; would have hit this identically).
3. `opentofu/modules/node-vm/main.tf` -- DC node VMs (Stage 3).

Only `opnsense-edge` was ever instantiated, which is the only reason this surfaced there
first. The `memory_mib = 2048` value is UNCHANGED -- with the unit correct it now means the
2 GiB that was always intended.

**Also corrected (same file, flagged not smuggled):** the `opnsense-edge` comment asserting
that disabling AMD `svm` was the *"ROOT CAUSE of the first-boot triple-fault"*. That claim
is disproven by this session's evidence (it never resolved the fault). The `svm = disable`
setting itself is RETAINED -- it is legitimate nested-virt hardening for a guest with no
business seeing `svm` -- but the comment now says it was tried and did not fix anything.
Leaving a false root-cause claim in the tree would have misled the next session.

## Guard -- so this cannot silently recur

`tofu validate` **cannot** catch this: `memory_unit` is optional, so omitting it is
schema-valid. That is precisely how it shipped. Following the DOCFIX-137 precedent (make
the standing lesson enforceable at the source):

- `scripts/opentofu-validate.sh` gains **S1**: a static scan failing any `libvirt_domain`
  that sets `memory` without `memory_unit`. Runs first, needs no tofu binary and no network.
- New `--check-memory-unit` flag runs S1 alone, so the harness can exercise it without
  paying for `tofu init`.
- `tests/opentofu-validate/` gains fixtures + T3/T4/T5. **T3 is the negative test**: the
  fixture is the exact defect that shipped, and S1 rejects it. T5 asserts the real tree is
  clean.

## Verification

- `tests/opentofu-validate/run-tests.sh`: **4 PASS / 0 FAIL** (was 1 PASS).
- `scripts/opentofu-validate.sh`: S1 PASS + fmt + init + validate **Success**.
- `scripts/repo-lint.sh`: **0 fail** (1 documented legacy warn).
- `scripts/run-tests-all.sh`: **ALL GREEN (52 harnesses)**.
- `tofu plan`: `0 to add, 1 to change, 0 to destroy` -- domain only; the 11 GiB direct-copy
  disk volume is NOT touched.

## Live apply -- SEPARATELY GATED, NOT DONE IN THIS CHANGE

The repo fix above is module source only (no live mutation). Booting the edge is a separate
operator-gated step. **Do not trust the plan's "update in-place":** max-boot-memory is
create-time to libvirt, the same trap DOCFIX-187 recorded for `machine`/`cpu` (provider
plans in-place, libvirt ignores it). The domain must be recreated:

    cd opentofu && source ~/vr1-stage1.env
    virsh -c qemu:///system destroy  office1-opnsense   # currently paused/faulted
    virsh -c qemu:///system undefine office1-opnsense   # leaves volumes intact
    tofu apply

Re-run `tofu plan` AFTER the `undefine` -- the `1 to change / in-place` plan captured above
is the *pre-undefine* plan and is NOT what will execute. Post-undefine, tofu refreshes, sees
the domain gone, and switches to create. Confirm it then reads `1 to add, 0 to destroy`
(disk + config_seed volumes untouched by the recreate).

**Pass criteria (do NOT declare success on `tofu apply` exiting 0):**
1. **Authoritative:** `virsh dominfo office1-opnsense` -> `Max memory: 2097152 KiB` (2 GiB).
   Anchor on this number; it is unambiguous. Do NOT anchor on the QEMU `-m` string -- with
   `unit='MiB'` libvirt may normalize and emit `-m size=2097152k`, which is **success**, not
   failure. (Stated as a guess here, not a measurement, until the apply runs.)
2. The serial log grows **past 262 bytes** into the loader/kernel.
3. The router actually works: WAN 172.30.1.2 reachable, LAN DHCP serving `office1-local`.

**Pre-staged fallbacks (so a surprise does not cost another session):**
- If `dominfo` still shows the wrong size, the provider is not honouring `memory_unit`: fix
  is `memory = var.memory_mib * 1024` (leaning on the KiB default), NOT more debugging.
- If it clears 262 bytes and boots further but does not reach a working router, that is a
  **sizing** question, not a regression of this fix -- check OPNsense 26.1's documented
  minimum RAM (the incident report's "3 GB" is UNVERIFIED) and bump to 4096 if warranted.
  Clearing the triple-fault and reaching a working router are two separate bars.

## Revert (per item)

- Module fix: remove the `memory_unit = "MiB"` line from the three `libvirt_domain`
  resources (restores the 2 MiB bug -- do not).
- Comment correction: `git revert` the doc hunk in `modules/opnsense-edge/main.tf`.
- Guard: delete the S1 block + `--check-memory-unit` in `scripts/opentofu-validate.sh`, drop
  T3/T4/T5 and `tests/opentofu-validate/fixtures/`.
- Whole change: `git revert <sha>`. Nothing here is live-state dependent.