# opentofu/ -- VR1 (DC-DC) infrastructure-as-code

Owns create/destroy of the "physical" layer per D-103: libvirt domains (node
VMs), all virtual networks (dark-fiber mesh, per-DC planes, ISP-edge), the
OPNsense edge VMs, and the Office1 service VMs. Runs from the Office1 operator
VM against the vcloud host libvirt. See `docs/dc-dc-buildout-design.md` and
`docs/dc-dc-deployment-workflow.md` (Stage 1-2, tooling gap register item 2)
for the surrounding plan.

**Status: SCAFFOLD, UNVALIDATED.** No `tofu`/`terraform` binary was available
in the session that authored this -- nothing here has been run through
`tofu init`/`validate`/`plan`, let alone `apply`. Before trusting any of it:

```
bash scripts/opentofu-validate.sh
```

on a machine with the binary and network access to the provider registry, and
read the result before proceeding.

## Scope of this delivery

Built:
- `modules/dc-planes` -- the six per-DC plane segments (D-052/D-100) as
  isolated `libvirt_network` resources.
- `modules/mesh-link` -- one inter-site L2 segment per leg of the D-100
  dark-fiber triangle (DC1<->DC2, DC1<->Office1, DC2<->Office1).
- `modules/dc-storage-pool` -- a directory-backed `libvirt_pool` per DC/Office1
  for future node-VM disk images.
- `modules/node-vm` (2026-07-09) -- one MAAS-managed OpenStack node VM per
  call: a BLANK `libvirt_volume` boot disk + a `libvirt_domain` with PXE-boot
  priority on its first network interface. This is the D-103 "OpenTofu creates
  the node-VM libvirt domains (SHIM)" need for Stage 3 -- deliberately NOT the
  cloud-init/pre-built-image pattern (see below), since MAAS images these VMs
  itself after enlistment. NOT instantiated in root `main.tf` yet: node count/
  memory/vcpu/disk sizing is a Phase-0 host/disk-budget decision
  (buildout-design Section 3) that hasn't been made -- call the module once
  real values exist, don't invent placeholder specs to wire it in sooner.
- Root wiring for DC1 (planes + pool, using DC1's inherited DC0 CIDRs) and
  Office1 (pool only), plus all three mesh links.

**Syntax bug fixed 2026-07-09:** `dc-planes`, `mesh-link`, and `dc-storage-pool`
originally used classic HCL block syntax (`domain { name = ... }`) for
`libvirt_network`'s `domain`/`mtu` and `libvirt_pool`'s `target`. Building the
domain module surfaced real example `.tf` files from the provider's own repo
(`examples/domain_with_network.tf`, `examples/alpine_cloudinit.tf`) that
consistently use attribute-style nested objects instead (`os = { ... }`,
`devices = { ... }`) -- this provider has moved to a schema style where nested
structures are object-typed attributes, not blocks. All three modules were
corrected to match once this was confirmed; see each file's syntax-note
comment. This was not previously verifiable from the doc-summarization passes
alone (the docs describe fields as "nested blocks" without showing the actual
call syntax) -- real example code is what settled it.

Deliberately NOT built yet, and why:
- **The cloud-init/pre-built-image VM pattern** (a `libvirt_cloudinit_disk` +
  a downloaded base `libvirt_volume` + a domain referencing both), needed for
  Office1's own service VMs and for OPNsense -- confirmed as a real, distinct
  provider pattern via `examples/alpine_cloudinit.tf`, but not yet built as a
  module here. Different enough from `node-vm` (image source, `backing_store`
  copy-on-write, `libvirt_cloudinit_disk`'s `user_data`/`meta_data`/
  `network_config`) to warrant its own module rather than bolting it onto
  `node-vm`.
- **DC2's plane networks.** D-101 has not assigned DC2's supernet yet (it's an
  explicit open sub-item). `main.tf` has the DC2 module block written and
  commented out, ready to uncomment once NetBox assigns real CIDRs -- do not
  fill in a guessed default to make it "work" sooner.
- **`tc netem` WAN simulation.** `libvirt_network` has no netem-equivalent
  argument; this is an OS-level `tc qdisc` setting applied to the bridge
  interface a `mesh-link` module creates, which needs either a `null_resource`
  provisioner or a small companion script -- not yet written. The exact
  parameters are themselves a Phase-0 open item (D-100), so this was going to
  need a second pass regardless.
- **OPNsense edges -- four separable pieces, not one, all currently blocked or
  unstarted:** (a) the network substrate is done (`modules/mesh-link`, above);
  (b) the VM/image itself needs the cloud-init/base-image module (above, still
  unbuilt) -- `modules/node-vm`'s blank-PXE-boot pattern does NOT apply to
  OPNsense, which needs a real bootable image -- and no image source (official
  OPNsense qcow2/ISO vs. a custom build) has been chosen either; (c) OPNsense's
  own routing/firewall config is a `config.xml` the
  libvirt provider never touches -- pre-baked image vs. cloud-init first-boot
  vs. API/CLI-driven provisioner is an open choice, not started; (d) `tc
  netem` WAN simulation is a separate OS-level `tc qdisc` setting with no
  Terraform-native equivalent at all, needing its own provisioner/script, and
  whose parameters are still an unruled D-100 open item -- a decision, not
  just scripting, is needed first. Full breakdown in
  `docs/dc-dc-deployment-workflow.md` gap register item 4.
- **MAAS-region / NetBox / GitBucket service VMs.** Depend on the deferred
  domain module (they're VMs too).

## Schema notes (read before extending)

Verified this session against the provider's actual current docs
(`docs/resources/*.md`) AND real example `.tf` files (`examples/*.tf`) in
`dmacvicar/terraform-provider-libvirt`, tag `v0.9.8`, fetched 2026-07-09 --
NOT from training-data memory of older provider versions, which would have
been wrong (and initially WAS wrong here once, see the syntax-bug note above):

- **Nested structures are attribute-style objects (`key = { ... }`),
  everywhere, not classic HCL blocks (`key { ... }`).** Confirmed directly
  from `examples/domain_with_network.tf` and `examples/alpine_cloudinit.tf`
  (`os = { ... }`, `devices = { ... }`, `target = { format = { ... } }`).
  This is a provider-wide convention, not resource-specific -- assume it
  applies to any nested field you haven't personally confirmed otherwise.
- `libvirt_network`'s isolation control lives at `forward.mode` (a nested
  attribute), not a top-level `mode` argument as older examples show. This
  scaffold OMITS the `forward` attribute entirely for isolated planes/links
  (documented native-libvirt behavior for a private, unforwarded switch)
  rather than guess a `mode = "none"` value this session could not confirm is
  still valid. Re-check this specific choice first if `tofu validate` objects.
- `libvirt_domain.devices.disks[].source.volume` takes `{ pool = ..., volume =
  ... }` (pool name + volume name, both strings); `.target` takes `{ dev =
  "vda", bus = "virtio" }`; `.driver` takes `{ type = "qcow2" }`. Confirmed
  directly from `examples/alpine_cloudinit.tf`.
- `libvirt_domain.devices.interfaces[].source.network` takes `{ network =
  <libvirt-network-name> }` (attach to a libvirt-managed network, what this
  repo's planes/mesh-links are); `.source.bridge` takes `{ bridge =
  <os-bridge-name> }` (attach to a plain OS bridge instead) -- confirmed from
  `examples/domain_with_network.tf`. Not used here (planes/mesh-links are
  libvirt-managed networks), but relevant if OPNsense ends up needing a direct
  bridge attach to a host NIC.
- `libvirt_volume.target.format.type` (nested twice: `target = { format = {
  type = "qcow2" } }`) and `libvirt_volume.backing_store = { path = ...,
  format = { type = "qcow2" } }` (copy-on-write from another volume's path)
  -- confirmed from `examples/alpine_cloudinit.tf`.
- **UNVERIFIED, flagged in `modules/node-vm/main.tf`:** the per-device `boot`
  attribute's internal shape (used to set PXE-first boot order). Confirmed to
  EXIST as a field name on both `interfaces[]` and `disks[]` entries (matching
  native libvirt's per-device `<boot order='N'/>`), but `{ order = N }` is an
  inference by pattern, not confirmed from a real example. Check this first
  with `tofu providers schema -json` before relying on PXE-first boot working
  as written.
- `libvirt_pool`'s `target` is likewise attribute-style (`target = { path =
  ... }`); types like `dir` are native libvirt storage-pool vocabulary
  (stable, long-standing, not provider-specific) -- reasonably high
  confidence.

## Conventions carried from the rest of this repo

- No hardcoded repo name (repo-lint L9 would need extending to cover `.tf`
  files if this becomes a recurring class of finding -- not yet done, flag if
  it comes up).
- Every variable without a safe, already-ratified default requires explicit
  input (no invented fallbacks) -- same "never use an inferred value"
  discipline as everywhere else in this repo.
- DC1's plane CIDRs are a second, hand-maintained copy of
  `scripts/lib-net.sh`'s `PLANE_CIDRS`. Keep them in sync manually until a
  generator exists to derive one from the other (DOCFIX candidate, not
  actioned).