diff --git a/docs/changelog-20260710-opentofu-scaffold-first-validation.md b/docs/changelog-20260710-opentofu-scaffold-first-validation.md new file mode 100644 index 0000000..31e8474 --- /dev/null +++ b/docs/changelog-20260710-opentofu-scaffold-first-validation.md @@ -0,0 +1,74 @@ +# Changelog 2026-07-10 -- DOCFIX-179: OpenTofu scaffold, first real validation + apply + +**Context.** First-ever execution of the `opentofu/` tree against a real `tofu` +binary (OpenTofu v1.12.3), on the vcloud host, executing +`runbooks/dc-dc-phase0-vcloud-prep.md` Steps 8-11. The tree had carried a +**SCAFFOLD, UNVALIDATED** banner since authoring (no `tofu` had ever run it). +Two real bugs blocked `tofu init`/`plan`; both fixed here, then the Stage-1 +module slice was validated, planned, and applied clean (13 libvirt objects). + +## What changed + +### FIX 1 -- per-module `required_providers` (the `tofu init` blocker) +- `tofu init` failed: child modules use `libvirt_*`/`maas_*` resources but did + not declare their own `required_providers`, so OpenTofu inferred the + `libvirt_` prefix as **`hashicorp/libvirt`** (nonexistent) instead of + `dmacvicar/libvirt`. Provider SOURCE mapping is NOT inherited by child + modules -- only provider CONFIGURATION is. +- Added `versions.tf` (source-only; version stays pinned at the root) to every + module using a provider resource: 8 libvirt modules (`dc-planes`, `mesh-link`, + `dc-storage-pool`, `office1-network`, `node-vm`, `base-image`, `cloudinit-vm`, + `opnsense-edge`) + 1 maas module (`maas-vm-host`). `netem-link` uses only the + built-in `terraform_data`, so it needs none. Fixed comprehensively (all + modules, not just Stage 1's four) so Stage 2/3 do not re-hit it. + +### FIX 2 -- `tofu fmt` (the whole never-fmt'd scaffold) +- `tofu fmt -recursive opentofu/` normalized 11 files (attribute alignment, + comment reflow) that had drifted because no `tofu` had ever formatted them. + +### CHANGE 3 -- `provider "maas"` deferred to Stage 3 +- The root `provider "maas"` block forced `tofu plan` to demand `maas_api_url` + AND the **sensitive** `maas_api_key` for a Stage-1 plan that creates ZERO + MAAS resources -- which would bake a fake key into `phase0.tfplan` and + `terraform.tfstate` (the DOCFIX-175 plaintext-secret surface). Operator ruled + (2026-07-10) to fix structurally rather than use the runbook's placeholder + workaround. Removed the `provider "maas"` block from `main.tf` and the + `maas_api_url`/`maas_api_key` variables from `variables.tf`, with in-file + notes to re-add them at Stage 3 (when `maas-vm-host` is instantiated). `maas` + stays in the ROOT `versions.tf` `required_providers` (version pinned, + `.terraform.lock.hcl` complete for Stage 3). + +### FIX 4 -- repo-lint must skip OpenTofu's `.terraform/` provider cache +- After the first real `tofu init`, `repo-lint` FAILED L1 (ASCII) on a + downloaded provider plugin's README inside `opentofu/.terraform/`. DOCFIX-175 + gitignored `.terraform/` for git, but `repo_lint.py`'s `all_text()` scans the + filesystem, not git. Added `".terraform" in p.parts` to `all_text()`'s skip + set (alongside `.git`/`__pycache__`) -- third-party, gitignored, not repo + content. `tests/repo-lint/run-tests.sh` -> 34/34 PASS. + +### Result +- `tofu validate` -> Success. `tofu plan` -> 13 to add / 0 / 0, with NO MAAS + vars required. `tofu apply` -> **13 added, 0 changed, 0 destroyed** (6 DC1 + planes + dc1/office1/dc2 pools + office1-local + 3 mesh legs, all MTU 9000). +- `.terraform.lock.hcl` is now generated -- **commit it** (pins provider + hashes: `dmacvicar/libvirt` 0.9.8, `canonical/maas` 2.7.2). +- `opentofu/README.md` status banner updated: Stage-1 module slice VALIDATED + + APPLIED; the not-yet-instantiated Stage 2/3 modules remain UNVALIDATED. + +## Verification +- `bash scripts/opentofu-validate.sh` (== `tofu fmt -check` + `init` + `validate`) + -> clean after the fixes. +- `bash scripts/repo-lint.sh` -> 0 fail. + +## Revert +- `git rm opentofu/modules/*/versions.tf` (the 9 added files) -- restores the + original `tofu init` inference failure. +- Re-add the `provider "maas"` block to `main.tf` and the two `maas_*` variables + to `variables.tf` from this changelog / git history. +- `tofu fmt` reverts are cosmetic; `git checkout` the 11 files to undo. +- (Live infra: `runbooks/dc-dc-teardown-rollback.md` / `tofu destroy` -- not part + of a repo revert.) + +## Follow-up (logged) +- Stage 3's runbook must re-add the `provider "maas"` block + the two variables + when it wires `maas-vm-host` -- noted in `main.tf`/`variables.tf` in-file. diff --git a/docs/changelog-20260710-phase0-runbook-as-executed.md b/docs/changelog-20260710-phase0-runbook-as-executed.md new file mode 100644 index 0000000..1fbf07a --- /dev/null +++ b/docs/changelog-20260710-phase0-runbook-as-executed.md @@ -0,0 +1,49 @@ +# Changelog 2026-07-10 -- DOCFIX-180: dc-dc-phase0 runbook as-executed corrections + +**Context.** End-of-Stage-1 runbook update. Per the operator's model (keep a +DOCFIX log through a whole phase, then update the runbook at phase close), the +corrections accrued during the first real execution of +`runbooks/dc-dc-phase0-vcloud-prep.md` (2026-07-10) are now folded back into the +runbook as draft updates. The live execution itself is DOCFIX-179 (scaffold) + +DOCFIX-178 (prereq installers); this entry is the runbook-text reconciliation. + +## What changed in `runbooks/dc-dc-phase0-vcloud-prep.md` + +1. **Top banner** -- marks the runbook FIRST EXECUTED 2026-07-10 and points at + the DOCFIX-178/179/180 changelogs. +2. **Known-gap section -> RESOLVED.** The `provider "maas"` block DID force + `tofu plan` to demand both `maas_api_url` AND the sensitive `maas_api_key` + for a zero-MAAS plan. Recorded the operator's structural resolution (defer the + provider block + both vars to Stage 3; DOCFIX-179) instead of the placeholder + workaround. Also corrected a stale reference ("Step 4" -> it surfaces at + Step 8/9). +3. **Step 3 (MTU).** Added the two-MTU-domains clarification: the measured host + *uplink* MTU (1500) is the ISP/WAN-edge value (Stage 2/3), while `underlay_mtu` + drives host-internal jumbo-capable virtio bridges -> set it to jumbo (9000) + for the internal fabric. First run: measured 1500, ruled `underlay_mtu=9000`. +4. **Step 5 (pools).** Added the non-pristine-host reality (a prior VR0 topology + + `wan` were present): inventory first, tear down leftovers gated, KEEP `wan` + as a gap-#17 ISP-uplink model. Replaced the bare `sudo mkdir` with the + self-owned-parent pattern (`install -d -o $USER -g libvirt /var/lib/libvirt/vr1` + once, then sudo-free subdirs). Recorded that DC2's storage pool IS wired now + (planes still deferred). +5. **Step 8 (init/validate).** Recorded the first-`init` failure + its two fixes + (per-module `required_providers`; `tofu fmt -recursive`) and the README banner + flip to STAGE-1-VALIDATED. +6. **Step 9 (plan).** Added `-input=false` (non-interactive safety) and corrected + the expected plan from 11 to the real **13 resources** (added the omitted + `office1_network` and the newly-wired `dc2_storage`). + +## Verification +- `bash scripts/repo-lint.sh` -> 0 fail (1 documented legacy WARN). +- No script changed here (doc-only), so no harness; the tooling changes these + notes describe are covered by DOCFIX-178/179's harnesses. + +## Revert +- `git checkout runbooks/dc-dc-phase0-vcloud-prep.md` to the pre-DOCFIX-180 + revision (removes the 6 as-executed edits), and + `git rm docs/changelog-20260710-phase0-runbook-as-executed.md`. + +## Follow-up (logged) +- The other `dc-dc-phaseN` runbooks will get the same end-of-phase as-executed + treatment as each is first-executed (their own DOCFIX numbers). diff --git a/docs/changelog-20260710-prereq-install-scripts.md b/docs/changelog-20260710-prereq-install-scripts.md new file mode 100644 index 0000000..31cd57f --- /dev/null +++ b/docs/changelog-20260710-prereq-install-scripts.md @@ -0,0 +1,60 @@ +# Changelog 2026-07-10 -- DOCFIX-178: workstation prerequisite install scripts + +**Context.** First real execution of `runbooks/dc-dc-phase0-vcloud-prep.md` on the +vcloud host surfaced that `tofu` was not installed, and the runbook's Step 6 only +hand-waved ("install per OpenTofu's official instructions ... not prescribed +here"). Operator ruling (2026-07-10): a runbook must not assume prereq runtimes +are present, and new operators should have individual, repo-provided install +scripts to get a workstation/workspace ready. This delivers that. + +## What changed + +### NEW: `scripts/prereqs/` (idempotent Debian/Ubuntu installers) +- `lib-prereq.sh` -- shared helpers (sourced): `pq_have`, `pq_require_apt`, + `pq_parse_mode` (--check/--dry-run/--help), `pq_run` (dry-run-aware runner). +- `install-opentofu.sh` -- OpenTofu `tofu` >= 1.6.0 via the official **deb** + method (lands `/usr/bin/tofu`, the command name every runbook step + + `scripts/opentofu-validate.sh` expect; the `opentofu` snap may not expose it). + Version-checks and skips if already new enough. +- `install-virtualization.sh` -- libvirt-daemon-system + qemu-kvm + libvirt-clients + + virtinst, and adds the invoking user to the `libvirt` group (re-login noted). +- `install-jq.sh` -- jq (a known cause of harness-gauntlet failures when absent). +- `install-image-tools.sh` -- qemu-utils (`qemu-img`) + `xorriso`, for the Stage + 2/3 OPNsense build path (`opnsense-prep-image.sh`, `opnsense-build-config-iso.sh`). +- `check-prereqs.sh` -- read-only report (all of the above + curl/python3/sudo), + exit 0 all-required-present / 1 missing; each miss names its fixer script. +- `install-all.sh` -- orchestrator (virtualization -> tofu -> jq -> image-tools) + + final check; passes `--dry-run` through. +- `README.md` -- quick start, per-script table, scope/conventions. +- Each installer supports `--check` (read-only) and `--dry-run` (mutate nothing); + `sudo` is used only for the actual package/group mutation, never in check/dry-run. + Non-Debian/Ubuntu OS -> fail loud (exit 3), no package-manager guessing. + +### NEW: `tests/prereqs/run-tests.sh` +- Exercises the parse/guard/`--check`/`--dry-run` paths only (never a real apt + install -- same posture as the opnsense-* harnesses). Includes the safety + assertion that `pq_run` under dry-run does NOT execute its command. **24/24 PASS.** + +### MODIFIED: `runbooks/dc-dc-phase0-vcloud-prep.md` +- New **Prerequisites** section (before "Known gap") with the one-pass + `check-prereqs`/`install-all` flow + a per-prereq table pointing at the installers. +- Step 6's vague "install per official instructions" replaced with a concrete + `bash scripts/prereqs/install-opentofu.sh` pointer + the >=1.6.0 floor. + +## Verification +- `bash tests/prereqs/run-tests.sh` -> `prereqs: 24/24 PASS`. +- `bash scripts/repo-lint.sh` -> 0 fail (1 documented legacy WARN). +- `bash scripts/ledger-scan.sh` -> DOCFIX next-free was 178 before this entry. + +## Revert +- `git rm -r scripts/prereqs tests/prereqs docs/changelog-20260710-prereq-install-scripts.md` +- Revert the two `runbooks/dc-dc-phase0-vcloud-prep.md` edits (Prerequisites + section + Step 6 paragraph) back to their pre-DOCFIX-178 text. + +## Follow-up (logged, not actioned) +- The other `dc-dc-phaseN` runbooks should gain the same short "see + `scripts/prereqs/`" pointer in their intros (their prereq sets overlap: tofu, + libvirt, and Stage 2/3 additionally need qemu-img/xorriso). One-line-per-runbook + sweep, its own DOCFIX. `scripts/prereqs/` itself is already stage-agnostic. +- macOS/RHEL install paths (currently exit-3 fail-loud) if a non-Ubuntu control + point ever appears. diff --git a/docs/dc-dc-deployment-workflow.md b/docs/dc-dc-deployment-workflow.md index 40ca0e8..05de60f 100644 --- a/docs/dc-dc-deployment-workflow.md +++ b/docs/dc-dc-deployment-workflow.md @@ -43,7 +43,16 @@ | **Reuse vs new** | NEW. No VR0-DC0 analog -- that testcloud IS the single vcloud host, already prepared; this is preparing the host to carry TWO independent DC substrates plus Office1. | | **Authoring status** | **Runbook WRITTEN 2026-07-09: `runbooks/dc-dc-phase0-vcloud-prep.md`** -- command-level steps (host identify -> measure CPU/RAM/disk/nested-KVM/MTU -> enable nested KVM -> prepare pool paths -> install/confirm OpenTofu -> tfvars -> init/validate/plan/apply -> post-apply gate verify), each MUTATION individually gated per this repo's discipline. Flags a real open structural question up front (the `provider "maas"` block in `main.tf` is unconditional even though this stage only touches libvirt resources -- may force `maas_api_url`/`maas_api_key` to be set at `plan` time with nothing yet using them; noted as a possible DOCFIX candidate, not silently worked around). NOT YET EXECUTED against real infrastructure -- this is the first runbook that will be, tomorrow morning, per the operator's own stated plan. `opentofu/` SCAFFOLD started 2026-07-09: `modules/dc-planes` (six per-DC planes) + `modules/mesh-link` (dark-fiber legs) + `modules/dc-storage-pool` wired for DC1 + Office1 -- see `opentofu/README.md` for what's built vs deliberately deferred (node-VM/domain resources, DC2 CIDRs, netem). UNVALIDATED: no `tofu` binary available to run `scripts/opentofu-validate.sh` yet -- this runbook's Step 8 is the first real run. | -**State:** RUNBOOK WRITTEN, NOT YET EXECUTED. +**State:** **DONE -- executed 2026-07-10** (first DC-DC runbook run against real +infrastructure; vcloud host, OpenTofu v1.12.3). As-built: nested KVM already-on; +`underlay_mtu=9000` (jumbo, operator ruling -- the isolated planes/mesh are +host-internal virtio bridges, jumbo-capable regardless of the 1500 ISP uplink); +Ceph size=3 disk-budget plausibility PASS (4.60 TiB margin); 13 libvirt objects +applied (6 dc1 planes + dc1/office1/dc2 pools + office1-local + 3 mesh legs, all +MTU 9000). Prior VR0 topology torn down (`wan` kept as a gap-#17 model). DC2 +PLANES deferred (Option B -- awaits NetBox supernet); DC2 storage pool wired. +Delivered DOCFIX-178 (prereq installers), DOCFIX-179 (scaffold: per-module +required_providers + fmt + maas-provider deferral). See those changelogs. --- diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 92acb7e..a86fde7 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -1528,6 +1528,82 @@ delivery). Remote backend + real locking explicitly flagged as Roosevelt/production-scope, not invented here. +## Jumphost stream -- session 2026-07-10 (cont.): VR1 Stage 1 FIRST REAL EXECUTION (dc-dc-phase0) + +First-ever execution of a DC-DC runbook against real infrastructure. Logged +session `dc-dc-phase0-vcloud-prep` (~/as-executed/2026-07-10-dc-dc-phase0-vcloud-prep.log). +On the ACTUAL vcloud host this time (hostname `vcloud`, AMD, 256 vCPU / 1 TiB / +10 TiB, Ubuntu 24.04) -- NOT the Windows box the 2026-07-09 session flagged. + +- **Steps 1-3 DONE (read-only audit + recorded decisions):** + - Step 1: on-host, libvirt reachable `qemu:///system`, user in libvirt group. + `libvirt_uri="qemu:///system"`. + - Step 2: nested KVM ALREADY ON (`kvm_amd/nested=1`) -> **Step 4 SKIPPED**. + No running domains. + - Step 3: uplink `enp1s0` MTU 1500. **Operator ruling: underlay_mtu=9000 + (jumbo)** for the host-internal dark-fiber/plane fabric (the module planes + are addressless isolated virtio bridges, jumbo-capable regardless of the + 1500 ISP uplink; 1500 governs future WAN-edge nets only). Ceph size=3 + disk-budget PLAUSIBILITY PASS (4.60 TiB margin @ conservative 9 TiB total, + 3 OSD nodes/DC x 500G, 0.5 overhead) -- real budget deferred to Stage 5. +- **FINDING: host was NOT pristine** -- carried a prior hand-built VR0 topology + (7 `vr0-dc0-*` planes + `vr0-off0-1_lan` + `vr0-ptp-dc0-off0` + a NAT `wan` + net) and the `default` pool (/var/lib/libvirt/images). **Operator ruled: tear + down the 9 orphaned vr0 nets, KEEP `wan`** (dual-stack NAT-out-enp1s0, GUA + 2602:f3e2:fe:10::/64 within the org ARIN block -- a ready model for gap #17's + ISP-uplink network; Stage-2 finding). Teardown DONE, gated (net 1 individually, + nets 2-9 after durable approval, stop-on-anomaly). Only `wan` + `default` + remain. +- **Step 5 DONE:** pool dirs `/var/lib/libvirt/vr1/{dc1,office1,dc2}`. Operator + granted a self-owned parent `/var/lib/libvirt/vr1` (setgid libvirt) so agent + creates subdirs without sudo. Non-secret session env `~/vr1-stage1.env` + created (TF_VAR_* + log path; NEVER holds maas_api_key). +- **Step 7 DONE (repo wiring):** `opentofu/dc-dc-phase0.auto.tfvars` (gitignored, + DOCFIX-175) from measured values; **`module "dc2_storage"` ACTIVATED** + + `dc2_pool_path` var added (operator: "wire dc2 while we're here"). **DC2 PLANES + DEFERRED (Option B, operator ruling)** -- wait for NetBox to assign the D-101 + supernet/ULA/GUA; do not invent CIDRs. DC2 mesh legs already wired. +- **DELIVERED -- DOCFIX-178:** `scripts/prereqs/` idempotent workstation prereq + installers (opentofu/virtualization/jq/image-tools + check-prereqs + install-all + + lib + README) + `tests/prereqs/run-tests.sh` (24/24) + a Prerequisites + section in the phase0 runbook + Step 6 concretized. Operator ruling: runbooks + must not assume prereq runtimes; give new operators install scripts. repo-lint + 0 fail. Changelog `docs/changelog-20260710-prereq-install-scripts.md`. +- **Steps 6-11 DONE -- STAGE 1 COMPLETE (2026-07-10).** tofu installed via + scripts/prereqs/install-opentofu.sh (OpenTofu v1.12.3). Step 8 init/validate + needed DOCFIX-179 fixes: per-module `required_providers` (child modules do NOT + inherit provider SOURCE, OpenTofu inferred nonexistent hashicorp/libvirt) + + `tofu fmt` of the never-fmt'd tree. Step 9 plan: the provider "maas" Known Gap + demanded BOTH maas_api_url + the SENSITIVE maas_api_key for a zero-MAAS plan + -> operator ruled STRUCTURAL FIX (Option B): deferred the provider "maas" + block + both vars to Stage 3 (DOCFIX-179); maas stays in root required_providers + (lock file complete). Step 10 apply: 13 objects, 0 changed/0 destroyed; state + chmod 600 (no secret in it now). Step 11 gate: all 4 bullets PASS, MTU 9000 + verified on all 10 nets. .terraform.lock.hcl to be committed. +- **NOC redesign (mid-session): RAISED then RETRACTED by operator** -- a + 4-site regional model (Office0/NOC0/DC0/DC1, NOC-hub fiber) was floated, then + withdrawn ("misunderstood an outside conversation; nothing changes"). No design + change; existing VR1 (DC1/DC2/Office1 triangle) stands. Logged so the pivot + isn't re-litigated. +- **DOCFIX candidates ACCRUING for the stage-completion commit (draft runbook + updates per operator's standing instruction):** (1) Step-3 MTU-domain + clarification (uplink 1500 vs jumbo internal underlay); (2) non-pristine-host + + `wan`-as-gap-#17 note for Steps 5/11; (3) Step-9 expected-resource list is + stale (omits `office1_network`) and now also gains `dc2_storage` -> real plan + is 13 resources: 6 dc1 planes + 3 pools (dc1/office1/dc2) + 4 nets + (office1-local + 3 mesh); (4) Step-7 tfvars-vs-gitignore (DOCFIX-175 gitignores + what Step 7 says to commit); (5) other dc-dc-phaseN runbooks want a + `scripts/prereqs/` pointer (per DOCFIX-178 follow-up). +- **DELIVERED -- DOCFIX-180:** dc-dc-phase0 runbook as-executed corrections + folded in at Stage-1 close (operator model: DOCFIX-log through a phase, update + the runbook at phase end). Top banner + Known-gap RESOLVED (maas deferral) + + Step-3 two-MTU-domains + Step-5 non-pristine-host/wan/pool-parent + Step-8 + init/required_providers + Step-9 -input=false and 13-resource plan. Changelog + `docs/changelog-20260710-phase0-runbook-as-executed.md`. repo-lint 0 fail. +- **Numbers after DOCFIX-180:** DOCFIX next-free 181 (re-scan before assigning). +- **STAGE 1 CLOSED. Deliverables DOCFIX-178/179/180 ready to commit** (operator + ruled: commit). Next: Stage 2 (dc-dc-phase1-office1-standup) when directed. + diff --git a/opentofu/.terraform.lock.hcl b/opentofu/.terraform.lock.hcl new file mode 100644 index 0000000..51c8b6c --- /dev/null +++ b/opentofu/.terraform.lock.hcl @@ -0,0 +1,71 @@ +# This file is maintained automatically by "tofu init". +# Manual edits may be lost in future updates. + +provider "registry.opentofu.org/canonical/maas" { + version = "2.7.2" + constraints = "2.7.2" + hashes = [ + "h1:0/+f28bV5/BqmlBZdmQiTWmzVnjn8/3VNwU+7RLHi1E=", + "h1:0NQfNgN9iJEcuT3WK2+vC9Ss/C1cnyepVP6uahpvSSo=", + "h1:3LC/Yd6KHwKKUkeAOi0g6lRO3Aj6+vMwc5xGf6ARRcY=", + "h1:Be77+i/FQe/egck+r/Bi/qtUVTtB1+BXnwVS7rhQwGI=", + "h1:Cd7Ed7paJzuEhCpkcauEl18JkfRnvX7zK2KnbZfOeQ0=", + "h1:GcMMY3WdhRt6xJ6uXSoKRxs0tpmt43GL4a/a/gBlJV8=", + "h1:IdwSAz83vIz0gQvRt9VT+rs9F0+1MgLIi2IjU3U/8gA=", + "h1:JWEzos28aI09FfC9nW8Odb9h/J2KWnBwoOAsTz0OZ1g=", + "h1:JY1cOBhZyAYVKEWkM2TU/z+CgXI0GJ53HIdzIgmORRM=", + "h1:M4bq+7JZB9hf2hPzr1x85dAhbWcmMBDCoqGvYcSc/hk=", + "h1:PTbVBZc9uhi9zztYpAiWIRHc7zNr4Uwr5wR/atlbuI0=", + "h1:WNzlm/e2Rn6mVhLEgG5yBvMJEpcUCvSQxExvY77pAEo=", + "h1:jc8UQ6JbaFzi+zxpf8AzfZXs94sMIWYwu34adBDWvBk=", + "h1:u+mlHkUKpqQFmDB1zRtBFC7Nl+gMDvFw0edd71q5FH0=", + "zh:07de224747a55736794b771cfe42bcd7a84a0656120df73f3bba8fa5a7869fed", + "zh:0aa77c14a1e4826cc1b345b6dc61086a2d9c20a77b151b7677718544e275636e", + "zh:2a4a659601e97d10ff8d07871fa74dd895946f5e683fb7fb4a6050b78beb3012", + "zh:4627c138c5b36d36612bea1d6d5ed895b5edc082f15bdc2af3e3f6c1153d0568", + "zh:4a948b51b701b1e60794e7cee3f7f5183838fd2e1e53898ddd824a3d31e7b267", + "zh:54d987940d7e6c591a1d4b837adf01b6cace17698227649fa72da8113d8c3531", + "zh:5b111497a82b898478fbea60b244d7e932039bf95460df9d984d0f05e2567854", + "zh:68ed5d31ac46fb0e79b69e981021f83166600d222d7b2adecbea9393f8ff93b9", + "zh:7fd8f29946937cd2c2716603081fb5c71f524465cc4b0bfeeeacab13eb927515", + "zh:824e97b9b68192bf04cc56e94e7d279468d1c4d48c2a98d9f71f86b804053500", + "zh:ce5669c6279fa6cd72ab8e39ccffeb90be3f0d30d7b4077f154b8cb1fad710c0", + "zh:d508aa6858a9212e6b71924ec9e0ddeaada4bab74e4e040ce036ef4189e5076a", + "zh:db819ed07eaafbab90f8292b0a6bc4551c3bb7dcbd1006af67e68719f36720b0", + "zh:f42d9b7c7b0a8adcd1135f295b40425d5a47191d14468a5725f665d027aa9647", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} + +provider "registry.opentofu.org/dmacvicar/libvirt" { + version = "0.9.8" + constraints = "0.9.8" + hashes = [ + "h1:/9TSq9ibeojseTyT6Jx3z/dbvT5uw8ka3/18NKonrQc=", + "h1:BN5hy1RaHlpcV0BDJYrSOEklK4YNdZCpiXOOGn0ce9k=", + "h1:MNJ6dcFru4DwQM7maIUAADJIHn3aF+L8XsnRJlleNaM=", + "h1:P1Cx0VZGTe6cpWpQ4JFOPJJb+Z+N50t4e1VA76T1tAA=", + "h1:Pm9du7Rzsg1bhLq+XnW9VaLSjTu0Yytr9AOdP9FLQvM=", + "h1:Q/JvNhRskJqDKPJoO9DW3+Ons3mk8kc++zlT4TucyFw=", + "h1:Tzk+wRzhboY3F3wB1u0y3ySMMd4ZPtJbOYlTbbkxZc0=", + "h1:Uuk8gkDEzmXfOTW/nm8GMG2aen7n/+X5GsTAf7eVRJc=", + "h1:cRBlnB/TG4+l0y0i42QLU0JoLDBxxJEwCguLXS9Rv8A=", + "h1:jn+hA9lIJzH5T0550IaKNsHxEAbyTGtvYWm02JCxexM=", + "h1:pKAwAcCGEwsKvVy5eIxdmb1gLhTTxB0xGVi8/jqj0uI=", + "h1:wS5VlGinhpnMCva0tKwLRPRQxmMUl80L05AueWshzNg=", + "h1:yqZeKoJ+EZc3687/+ZBqBmtwzvBPLNwaEHW74+bSc6Y=", + "zh:061e5187853729e1d8ba20938402ad6e778b4097436925d0bef7741c8aa26ee1", + "zh:69a2ac8ee0cb0c1581bdc9a1296d89abcd1808229ff0a2fa20678ba4f7472944", + "zh:8b02bf349191b4e7d9529d620da9e9265de5bd01d6983bb079589acdf80dda7a", + "zh:9620872ed1da1ff421e85dd4ba2a110c284f3f0d06369c36a9aa75d501155fad", + "zh:9ae7c932b7fb6f62ff20b863a15a7cd713415faa4c789472dc403062808366cb", + "zh:a369c3e271b49e085940e6181fe6c1086c2126eebf5c7ad2b1db8fc8792dd30c", + "zh:a45ad97d7192bea104b32521839dd705a330f13364cf1007dc912b6a0b6be3c8", + "zh:b69f2a13ff688ad4b79e14deb6ef6341187864f50960bc0fd9c682f6077b4318", + "zh:c20b467ff0ac76431a7bdb99662b9414b922ea603327509518c0179416cdfea3", + "zh:c22d45913bd1f00a33d1a4b80547e4a2854409a298a0868c1730124f03a210ad", + "zh:cf82bb14884fcbdcd57620cee248994b1633ea246fa49fd668d2f1739abb982d", + "zh:ddc1cb4417206dd46eb22c3b81204b3a5edda03c4436e6a5914d541fcfd4c27f", + "zh:e6b1983b8f1b9658448d0d09635619cc75dde314e6b4d7fd16d827f9e4bdb767", + ] +} diff --git a/opentofu/README.md b/opentofu/README.md index 3c71088..08bc8ce 100644 --- a/opentofu/README.md +++ b/opentofu/README.md @@ -10,17 +10,25 @@ for where every provider fact below was sourced from and the fetch methodology that worked -- read that first if extending any module here. -**Status: SCAFFOLD, UNVALIDATED.** No `tofu`/`terraform` binary was available -in the session that authored this -- nothing here has been run through -`tofu init`/`validate`/`plan`, let alone `apply`. Before trusting any of it: +**Status: STAGE-1 VALIDATED + APPLIED (2026-07-10); Stage 2/3 modules still +UNVALIDATED.** First real `tofu` run: 2026-07-10 on the vcloud host (OpenTofu +v1.12.3), executing `runbooks/dc-dc-phase0-vcloud-prep.md`. The root module + +the four Stage-1 module types (`dc-planes`, `dc-storage-pool`, `mesh-link`, +`office1-network`) passed `tofu init`/`validate`/`plan` and were `apply`-ed +clean (13 libvirt objects, all MTU 9000). Two fixes were required (DOCFIX-179): +per-module `required_providers` (child modules do NOT inherit provider SOURCE +mapping -- without their own `versions.tf`, OpenTofu infers `hashicorp/libvirt`, +which does not exist), and `tofu fmt -recursive`. The root `provider "maas"` +block was also deferred to Stage 3 (it forced Stage 1 to supply a sensitive key +for zero MAAS resources). STILL UNVALIDATED -- the modules not yet instantiated +from root: `node-vm`, `base-image`, `cloudinit-vm`, `opnsense-edge`, +`maas-vm-host`, `netem-link` (Stage 2/3). Re-run before trusting those, on a +machine with the binary + provider-registry access: ``` bash scripts/opentofu-validate.sh ``` -on a machine with the binary and network access to the provider registry, and -read the result before proceeding. - ## Scope of this delivery Built: diff --git a/opentofu/main.tf b/opentofu/main.tf index 07d069c..24606da 100644 --- a/opentofu/main.tf +++ b/opentofu/main.tf @@ -6,17 +6,18 @@ uri = var.libvirt_uri } -provider "maas" { - api_url = var.maas_api_url - api_key = var.maas_api_key -} - -# ---- modules/maas-vm-host and modules/netem-link are NOT instantiated below -# yet: both need real, measured inputs (a MAAS zone/pool that exists, the -# vcloud host's SSH target, real netem parameters) that don't exist yet -- -# see docs/dc-dc-deployment-workflow.md gap register items 2/4/11 and -# opentofu/README.md. The provider block above is wired so the modules are -# ready to call once those values are real. ---- +# ---- The provider "maas" CONFIG block is deliberately DEFERRED to Stage 3 +# (DOCFIX-179, decided on the first real `tofu plan`). No module instantiated in +# Stage 1-2 uses the maas provider (modules/maas-vm-host is a Stage-3 module, +# commented out below), yet a configured provider "maas" block here forced +# `tofu plan` to demand maas_api_url AND the SENSITIVE maas_api_key for a plan +# that creates ZERO MAAS resources -- which would bake a fake key into +# plan/state (the DOCFIX-175 plaintext-secret surface). maas stays in +# versions.tf's required_providers (version pinned, lock-file complete); Stage 3 +# re-adds this provider block + its two variables when it actually instantiates +# modules/maas-vm-host. modules/netem-link is likewise not instantiated yet +# (needs a real bridge/SSH target + netem params). See gap register items +# 2/4/11 and opentofu/README.md. ---- # ---- DC1: inherits the DC0 six-plane v4 layout unchanged (D-101) ---- @@ -34,11 +35,11 @@ target_path = var.dc1_pool_path } -# ---- DC2: NOT YET WIRED. ---- -# D-101 has not assigned DC2's supernet yet (open sub-item). Once NetBox -# assigns it, add a `dc2_planes` variable (same shape as `dc1_planes` in -# variables.tf) and uncomment/adapt this block -- do not fill in guessed -# CIDRs to make this "work" sooner. +# ---- DC2: PLANES deferred (operator ruling 2026-07-10, Option B) -- wait for +# NetBox to assign D-101's supernet/ULA/GUA before wiring dc2_planes. Do NOT +# fill in guessed CIDRs to make it "work" sooner. When NetBox assigns it, add a +# `dc2_planes` variable (same shape as `dc1_planes`) and uncomment this block. +# The storage pool + mesh legs are NOT address-dependent, so they ARE wired now. # # module "dc2_planes" { # source = "./modules/dc-planes" @@ -47,12 +48,12 @@ # mtu = var.underlay_mtu # planes = var.dc2_planes # } -# -# module "dc2_storage" { -# source = "./modules/dc-storage-pool" -# dc_name = "dc2" -# target_path = var.dc2_pool_path -# } + +module "dc2_storage" { + source = "./modules/dc-storage-pool" + dc_name = "dc2" + target_path = var.dc2_pool_path +} # ---- Office1 headend: its own storage pool for MAAS-region/NetBox/GitBucket # service-VM disks (D-103). Office1 does not get a dc-planes module -- the diff --git a/opentofu/modules/base-image/variables.tf b/opentofu/modules/base-image/variables.tf index 33c194e..008ba2f 100644 --- a/opentofu/modules/base-image/variables.tf +++ b/opentofu/modules/base-image/variables.tf @@ -20,5 +20,5 @@ is not guaranteed the same way). Do not assume this module covers OPNsense without checking that first. EOT - type = string + type = string } diff --git a/opentofu/modules/base-image/versions.tf b/opentofu/modules/base-image/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/base-image/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/cloudinit-vm/main.tf b/opentofu/modules/cloudinit-vm/main.tf index 9ebd8e5..d36b768 100644 --- a/opentofu/modules/cloudinit-vm/main.tf +++ b/opentofu/modules/cloudinit-vm/main.tf @@ -34,12 +34,12 @@ backing_store = { path = var.base_volume_path # the caller passes module..path - # straight through -- a real attribute - # reference, matching - # examples/alpine_cloudinit.tf's - # `libvirt_volume.alpine_base.path` exactly, - # just threaded across a module boundary - # instead of within one file. + # straight through -- a real attribute + # reference, matching + # examples/alpine_cloudinit.tf's + # `libvirt_volume.alpine_base.path` exactly, + # just threaded across a module boundary + # instead of within one file. format = { type = "qcow2" } diff --git a/opentofu/modules/cloudinit-vm/variables.tf b/opentofu/modules/cloudinit-vm/variables.tf index 0815632..9a920ee 100644 --- a/opentofu/modules/cloudinit-vm/variables.tf +++ b/opentofu/modules/cloudinit-vm/variables.tf @@ -31,7 +31,7 @@ + volume name; libvirt's actual pool-relative path convention was not independently confirmed this session (see main.tf's note on this). EOT - type = string + type = string } variable "network_names" { @@ -48,7 +48,7 @@ hasn't been done yet, and inventing placeholder cloud-config content would misrepresent it as a real decision. Pass the actual config once written. EOT - type = string + type = string } variable "meta_data" { @@ -64,5 +64,5 @@ 3), so a real static address from NetBox is required here, not a plausible-looking default that would silently fail to reach anything. EOT - type = string + type = string } diff --git a/opentofu/modules/cloudinit-vm/versions.tf b/opentofu/modules/cloudinit-vm/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/cloudinit-vm/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/dc-planes/variables.tf b/opentofu/modules/dc-planes/variables.tf index ae2fd94..d21eb3a 100644 --- a/opentofu/modules/dc-planes/variables.tf +++ b/opentofu/modules/dc-planes/variables.tf @@ -18,10 +18,10 @@ EOT type = map(object({ cidr = string # anchor fact only -- NOT applied to the libvirt_network - # resource (isolated planes carry no forward/ips block, see - # main.tf); recorded here purely so `tofu plan` output and - # `terraform output` reflect the intended assignment for - # cross-checking against NetBox. + # resource (isolated planes carry no forward/ips block, see + # main.tf); recorded here purely so `tofu plan` output and + # `terraform output` reflect the intended assignment for + # cross-checking against NetBox. })) } diff --git a/opentofu/modules/dc-planes/versions.tf b/opentofu/modules/dc-planes/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/dc-planes/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/dc-storage-pool/variables.tf b/opentofu/modules/dc-storage-pool/variables.tf index 82c3f8a..e052d08 100644 --- a/opentofu/modules/dc-storage-pool/variables.tf +++ b/opentofu/modules/dc-storage-pool/variables.tf @@ -12,5 +12,5 @@ Section 3). MUST be a real, measured path on the vcloud host -- never invented; no default. EOT - type = string + type = string } diff --git a/opentofu/modules/dc-storage-pool/versions.tf b/opentofu/modules/dc-storage-pool/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/dc-storage-pool/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/maas-vm-host/variables.tf b/opentofu/modules/maas-vm-host/variables.tf index 8d0bd34..d688787 100644 --- a/opentofu/modules/maas-vm-host/variables.tf +++ b/opentofu/modules/maas-vm-host/variables.tf @@ -14,7 +14,7 @@ the root module in practice, but is kept as its own input here rather than silently assumed identical. EOT - type = string + type = string } variable "zone" { diff --git a/opentofu/modules/maas-vm-host/versions.tf b/opentofu/modules/maas-vm-host/versions.tf new file mode 100644 index 0000000..332837a --- /dev/null +++ b/opentofu/modules/maas-vm-host/versions.tf @@ -0,0 +1,11 @@ +# Provider source mapping: maps local name `maas` to canonical/maas so +# `tofu init` does not infer the `maas_` resource prefix as a nonexistent +# hashicorp/maas. Version pinned at the root (opentofu/versions.tf). +# (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + maas = { + source = "canonical/maas" + } + } +} diff --git a/opentofu/modules/mesh-link/versions.tf b/opentofu/modules/mesh-link/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/mesh-link/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/netem-link/variables.tf b/opentofu/modules/netem-link/variables.tf index 49e85db..59616cf 100644 --- a/opentofu/modules/netem-link/variables.tf +++ b/opentofu/modules/netem-link/variables.tf @@ -14,7 +14,7 @@ + passwordless sudo on the vcloud host for the invoking user are assumed prerequisites, not configured by this module. No default -- measured. EOT - type = string + type = string } variable "bridge_name" { @@ -27,7 +27,7 @@ ` (or `tofu show` post-apply) and pass it explicitly here -- no default. EOT - type = string + type = string } variable "netem_args" { @@ -42,5 +42,5 @@ research target, not tc itself); confirm your exact netem_args string with `man tc-netem` or a live `tc qdisc` test before relying on it. EOT - type = string + type = string } diff --git a/opentofu/modules/node-vm/main.tf b/opentofu/modules/node-vm/main.tf index e939cbb..cd73642 100644 --- a/opentofu/modules/node-vm/main.tf +++ b/opentofu/modules/node-vm/main.tf @@ -33,10 +33,10 @@ } resource "libvirt_domain" "node" { - name = var.vm_name - memory = var.memory_mib - vcpu = var.vcpu - type = "kvm" + name = var.vm_name + memory = var.memory_mib + vcpu = var.vcpu + type = "kvm" running = true os = { diff --git a/opentofu/modules/node-vm/variables.tf b/opentofu/modules/node-vm/variables.tf index 102fcdb..1e71a17 100644 --- a/opentofu/modules/node-vm/variables.tf +++ b/opentofu/modules/node-vm/variables.tf @@ -30,5 +30,5 @@ given PXE boot priority (see main.tf's boot-order note; MAAS-managed node VMs PXE-boot per D-103, they are not given a pre-built OS image). EOT - type = list(string) + type = list(string) } diff --git a/opentofu/modules/node-vm/versions.tf b/opentofu/modules/node-vm/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/node-vm/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/office1-network/versions.tf b/opentofu/modules/office1-network/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/office1-network/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/modules/opnsense-edge/variables.tf b/opentofu/modules/opnsense-edge/variables.tf index d86c1d4..31abce0 100644 --- a/opentofu/modules/opnsense-edge/variables.tf +++ b/opentofu/modules/opnsense-edge/variables.tf @@ -34,7 +34,7 @@ create.content.url almost certainly does a plain fetch with no decompression/conversion; that work must already be done. EOT - type = string + type = string } variable "config_iso_path" { @@ -48,7 +48,7 @@ role is design work that has to happen first -- do not point this at a placeholder/empty config.xml to make the module "work" sooner. EOT - type = string + type = string } variable "lan_network_name" { @@ -67,7 +67,7 @@ stays clear in this file even though the real role still comes from config.xml. EOT - type = string + type = string } variable "wan_network_name" { @@ -78,5 +78,5 @@ mesh-link legs (modules/mesh-link) or a per-site ISP-simulated segment are the expected callers here. EOT - type = string + type = string } diff --git a/opentofu/modules/opnsense-edge/versions.tf b/opentofu/modules/opnsense-edge/versions.tf new file mode 100644 index 0000000..a6a130f --- /dev/null +++ b/opentofu/modules/opnsense-edge/versions.tf @@ -0,0 +1,13 @@ +# Provider source mapping for this module. Child modules MUST declare their own +# required_providers so OpenTofu maps the local name `libvirt` to +# dmacvicar/libvirt. WITHOUT this, OpenTofu infers the `libvirt_` resource +# prefix as hashicorp/libvirt (which does not exist) and `tofu init` fails. +# Version is pinned once at the ROOT (opentofu/versions.tf); child modules +# declare source only. (Found on the first real `tofu init`; DOCFIX-179.) +terraform { + required_providers { + libvirt = { + source = "dmacvicar/libvirt" + } + } +} diff --git a/opentofu/variables.tf b/opentofu/variables.tf index c5a4674..905bbc5 100644 --- a/opentofu/variables.tf +++ b/opentofu/variables.tf @@ -3,16 +3,12 @@ type = string } -variable "maas_api_url" { - description = "MAAS API endpoint, e.g. http://:5240/MAAS (confirmed format from the canonical/maas provider's own docs). Measured, no default." - type = string -} - -variable "maas_api_key" { - description = "MAAS API key for the account OpenTofu authenticates as. Retrieved via the operator's own secret-handling process -- never hardcoded, never printed (this repo's own secrets discipline applies here same as everywhere else). No default. Pass via TF_VAR_maas_api_key (never -var, which lands in shell history). CAUTION: sensitive=true suppresses CLI/plan output only -- the real value is still stored in PLAINTEXT in terraform.tfstate regardless of this flag. See opentofu/README.md 'State file handling' (DOCFIX-175) for the required file-permission/backup posture before running tofu apply." - type = string - sensitive = true -} +# maas_api_url / maas_api_key variables are DEFERRED to Stage 3 (DOCFIX-179): +# removed from Stage 1's root because the provider "maas" block that consumed +# them is deferred (see main.tf's note). Re-add both here when Stage 3 +# instantiates modules/maas-vm-host. The SENSITIVE maas_api_key MUST return via +# TF_VAR_maas_api_key only -- never a committed file (opentofu/README.md +# "State file handling", DOCFIX-175). variable "domain_suffix" { description = "Base domain for per-plane libvirt DNS domains. Anchor value per D-106's naming convention (.omega..vr1.cloud.neumatrix.local)." @@ -35,6 +31,11 @@ type = string } +variable "dc2_pool_path" { + description = "Host filesystem path for DC2's libvirt storage pool. Measured, real path on the vcloud host -- no default. NOTE: only DC2's STORAGE POOL is wired (operator ruling 2026-07-10, dc-dc-phase0 Step 5/7). DC2's PLANES stay deferred (Option B) pending NetBox assigning the D-101 supernet -- a storage pool has no address dependency, plane networks do." + type = string +} + variable "dc1_planes" { description = <<-EOT DC1's six planes. DC1 INHERITS the DC0 v4 layout UNCHANGED (D-101) -- diff --git a/runbooks/dc-dc-phase0-vcloud-prep.md b/runbooks/dc-dc-phase0-vcloud-prep.md index 915e286..95f07d7 100644 --- a/runbooks/dc-dc-phase0-vcloud-prep.md +++ b/runbooks/dc-dc-phase0-vcloud-prep.md @@ -1,5 +1,11 @@ # DC-DC Phase 0 -- vcloud host preparation (Stage 1) +> **FIRST EXECUTED 2026-07-10** (vcloud host, OpenTofu v1.12.3) -- run +> end-to-end against real infrastructure. As-executed corrections are folded in +> inline, tagged **(DOCFIX-180)**; the OpenTofu scaffold fixes that run required +> are DOCFIX-179; the workstation prereq installers are DOCFIX-178. See the +> matching `docs/changelog-20260710-*.md`. + Turn the bare vcloud host into a substrate Office1 can deploy VR1 (DC1, DC2, Office1) from. This is the FIRST DC-DC runbook executed against real infrastructure -- everything before this session was repo-only prep (docs, @@ -29,26 +35,49 @@ --- +## Prerequisites -- runtimes on the vcloud host / Office1 control point (DOCFIX-178) + +Do NOT assume these are already installed. This runbook drives a real libvirt +host and a real `tofu` binary; confirm each BEFORE Step 1. Idempotent installer +scripts live in `scripts/prereqs/` (Debian/Ubuntu; each supports `--check` and +`--dry-run`) -- see that directory's `README.md`. One-pass setup on a fresh +workstation/workspace: + +```bash +bash scripts/prereqs/check-prereqs.sh # read-only: what's missing? +bash scripts/prereqs/install-all.sh # install the missing prereqs (uses sudo) +# re-login (or: newgrp libvirt) if the libvirt group was just added, then: +bash scripts/prereqs/check-prereqs.sh # confirm all required present +``` + +| Prerequisite | Needed by | Installer | +|---|---|---| +| libvirt + qemu-kvm + `virsh`; `qemu:///system` reachable; user in `libvirt` group | Steps 1, 5, 11 | `scripts/prereqs/install-virtualization.sh` | +| OpenTofu `tofu` >= 1.6.0 (installs as `/usr/bin/tofu`, not the `opentofu` snap) | Steps 8-10 | `scripts/prereqs/install-opentofu.sh` | +| Provider-registry reach (registry.opentofu.org) for `tofu init` | Step 8 | network (local mirror if airgapped) | +| `sudo`, or a pre-owned pool-parent dir under `/var/lib/libvirt` | Step 5 mkdir | -- | +| jq; qemu-img + xorriso (later dc-dc stages) | ops scripts; Stage 2/3 OPNsense build | `install-jq.sh`, `install-image-tools.sh` | + +If the host is airgapped by policy, the Step 8 `tofu init` registry fetch needs a +local provider mirror instead -- out of scope here; resolve before Step 8. + +--- + ## Known gap, flagged before you start (not fixed in this runbook) -`opentofu/main.tf` declares `provider "maas" { api_url = var.maas_api_url; -api_key = var.maas_api_key }` UNCONDITIONALLY at the root, even though this -Phase-0 stage only touches `libvirt`-backed resources (planes, storage pools, -mesh links) -- no MAAS resource is instantiated yet (Stage 3's job). Depending -on how strictly OpenTofu validates a declared-but-unused provider block, `tofu -init`/`plan` in Step 4 MAY demand `maas_api_url`/`maas_api_key` be set even -though nothing in this stage's plan uses MAAS. If you hit that: -- Cheapest workaround: pass placeholder-shaped-but-clearly-fake values (e.g. - `maas_api_url=http://not-yet-provisioned.invalid:5240/MAAS`) ONLY if OpenTofu - merely wants the variable non-null and does not attempt to actually reach - it during `plan` for a provider with zero resources. Verify this is true - (a `plan`, not `apply`, should never open a real connection) before relying - on it. -- If OpenTofu actually attempts to validate MAAS reachability at `plan` time - even with zero MAAS resources, that is a real structural finding -- LOG it - (a DOCFIX candidate: split the MAAS provider block into a Stage-3-only root - module) rather than working around it by fabricating real-looking - credentials or skipping `tofu validate`. +**RESOLVED on first execution (2026-07-10, DOCFIX-179/180).** The root +`provider "maas"` block DID force `tofu plan` to demand BOTH `maas_api_url` AND +the SENSITIVE `maas_api_key` for a Stage-1 plan that creates zero MAAS resources +(it fails at variable validation, *before* any provider is configured -- so no +real MAAS connection is ever attempted). Rather than the placeholder workaround +(which bakes a fake sensitive key into `phase0.tfplan`/`terraform.tfstate` -- +the DOCFIX-175 surface), the operator ruled the STRUCTURAL fix: the +`provider "maas"` block and the `maas_api_url`/`maas_api_key` variables are now +REMOVED from the root module until Stage 3 instantiates `maas-vm-host` +(DOCFIX-179). `maas` stays in the root `versions.tf` `required_providers` +(version pinned, lock file complete). **Stage 1 therefore needs no MAAS input +at all.** Stage 3's runbook MUST re-add the provider block + both variables when +it wires `maas-vm-host` (noted in `main.tf`/`variables.tf` in-file). --- @@ -147,11 +176,23 @@ ```bash ip -o link show | awk '{print $2, $0}' | grep -i mtu ``` -Identify which interface(s) carry the vcloud host's real uplink (the one -OpenTofu's virtual networks will ultimately ride over, or bridge to, for the -simulated ISP edges) and record its MTU. Per D-101 (folded in from D-102): -"Prefer jumbo (9000) end-to-end... if pinned at 1500, set the reduced tenant -MTU consistently." **Do not assume jumbo** -- use the measured value. +Identify which interface(s) carry the vcloud host's real uplink and record its +MTU. Per D-101 (folded in from D-102): "Prefer jumbo (9000) end-to-end... if +pinned at 1500, set the reduced tenant MTU consistently." + +**As-executed clarification (2026-07-10, DOCFIX-180) -- there are TWO distinct +MTU domains; do not conflate them:** the measured host *uplink* MTU (e.g. +`enp1s0`) is the ISP/WAN-edge MTU and governs the future per-site ISP-uplink +networks (Stage 2/3), which are naturally ~1500 like real internet. But +`underlay_mtu` here drives the six planes + mesh legs + `office1-local`, which +are *host-internal isolated virtio bridges* -- they never traverse the physical +uplink and ARE jumbo-capable regardless of it. Per D-101's explicit "prefer +jumbo end-to-end" intent (and because real dark fiber is jumbo-capable), set +`underlay_mtu` to the JUMBO value (9000) for the internal fabric when the host +supports it -- keeping tenant MTU at 1500 with full geneve headroom and avoiding +the nested-virt+geneve MTU-stacking failure the design itself flags. First run: +measured uplink 1500, ruled `underlay_mtu=9000`. Run the calculator below for +BOTH the measured uplink and the chosen jumbo value to see each verdict. **Decision to record (D-101 MTU sub-policy, buildout-design Section 3) -- use the tested calculator (DOCFIX-162, tooling gap #7, 19/19 tests) instead @@ -242,22 +283,37 @@ ``` Choose paths under whichever filesystem has the room computed in Step 3's disk-budget check. Do not reuse VR0/DC0's existing pool path if this vcloud -host is the SAME physical host that ran the single-DC testcloud (check -`virsh pool-list --all` for any existing pool already pointed at a -candidate path) -- a path collision between VR0's retiring pool and VR1's -new one is exactly the kind of silent-overlap risk this repo's discipline -flags rather than assumes away. +host is the SAME physical host that ran the single-DC testcloud. -**MUTATION** +**As-executed (2026-07-10, DOCFIX-180) -- the vcloud host was NOT pristine.** It +carried a prior hand-built VR0 topology (9 `vr0-*` isolated networks + a `wan` +NAT network) plus the stock `default` pool at `/var/lib/libvirt/images`. +Inventory FIRST: ```bash -sudo mkdir -p /path/you/chose/dc1 -sudo mkdir -p /path/you/chose/office1 +virsh -c qemu:///system net-list --all +virsh -c qemu:///system pool-list --all ``` -(DC2's path is not created yet -- no CIDRs, no plan to instantiate it in -`main.tf` yet either; see `opentofu/main.tf`'s commented DC2 block.) +Tear down leftover prior-generation networks (gated, per +`runbooks/dc-dc-teardown-rollback.md`) for a clean substrate + clean Step-11 +verify -- BUT KEEP a pre-existing `wan` NAT-to-uplink network: it is a +ready-made model for the per-site ISP-uplink network (gap #17) Stage 2/3 needs. +New pool paths must NOT be `/var/lib/libvirt/images` itself (that is `default`'s +path). -Record the two real paths -- they go into Step 7's tfvars as -`dc1_pool_path`/`office1_pool_path`. +**MUTATION** -- `/var/lib/libvirt` is root-owned. Either run the mkdir with +`sudo`, OR (recommended, sudo-free thereafter) have the operator create a +self-owned VR1 parent ONCE, then create the pool subdirs as your own user: +```bash +# one-time, operator (real terminal for the sudo password): +sudo install -d -o "$USER" -g libvirt -m 2775 /var/lib/libvirt/vr1 +# then, no sudo, and DC2's pool too (see below): +mkdir -p /var/lib/libvirt/vr1/dc1 /var/lib/libvirt/vr1/office1 /var/lib/libvirt/vr1/dc2 +``` +DC2's pool dir + the `dc2_storage` module ARE wired now (operator ruling: a +storage pool has no address dependency); only DC2's PLANES stay deferred pending +NetBox's supernet. Record the real paths -- they become Step 7's +`dc1_pool_path`/`office1_pool_path`/`dc2_pool_path`. First run used +`/var/lib/libvirt/vr1/{dc1,office1,dc2}`. --- @@ -267,10 +323,17 @@ ```bash tofu version ``` -If absent, install per OpenTofu's own official install instructions for -this host's OS (not prescribed here -- an OS-specific package/binary install -is a one-time host-prep action the operator runs directly, matching this -repo's practice of not hardcoding install mechanics that vary by distro). +If absent (or older than 1.6.0), install it with the repo's own idempotent +installer -- this is now a documented prerequisite (see the **Prerequisites** +section at the top of this runbook), not an out-of-band step: +```bash +bash scripts/prereqs/install-opentofu.sh # deb method -> /usr/bin/tofu; needs sudo +``` +Ideally the whole prereq set is already satisfied before Step 1 +(`bash scripts/prereqs/check-prereqs.sh`); this Step is the in-sequence +confirmation that `tofu` specifically is ready. For a non-Debian/non-Ubuntu +control point, `install-opentofu.sh` fails loud with a pointer to OpenTofu's +own portable install options rather than guessing a package manager. **CHECK -- registry network access** (needed for `tofu init` in Step 8 to fetch `dmacvicar/libvirt` 0.9.8 and `canonical/maas` 2.7.2) @@ -331,11 +394,16 @@ tofu validate ``` Or equivalently, from the repo root: `bash scripts/opentofu-validate.sh`. -This is the FIRST real run of this script against a real `tofu` binary -- -`opentofu/README.md` has carried a **SCAFFOLD, UNVALIDATED** banner since -authoring; a clean run here is the first evidence closing that banner (do -not remove the banner from the README until this has actually run clean -- -update it as part of this stage's completion, not preemptively). + +**As-executed (2026-07-10, DOCFIX-179): the first real `tofu init` FAILED** +until two fixes landed, now both in the repo (a fresh clone inits clean): +1. Each child module using a provider resource needs its OWN `required_providers` + (`modules/*/versions.tf`). Child modules do NOT inherit provider SOURCE + mapping, so OpenTofu inferred the `libvirt_` prefix as a nonexistent + `hashicorp/libvirt` and init failed. Fixed for all libvirt + maas modules. +2. `tofu fmt -recursive` on the never-`fmt`'d tree (11 files). +The `opentofu/README.md` banner is now updated from SCAFFOLD/UNVALIDATED to +STAGE-1-VALIDATED (the Stage 2/3 modules remain unexercised). If `validate` surfaces a schema mismatch against the flagged UNVERIFIED notes in `opentofu/README.md` (the `node-vm` boot-order attribute shape is @@ -352,16 +420,23 @@ ```bash cd opentofu -tofu plan -out=phase0.tfplan +tofu plan -input=false -out=phase0.tfplan ``` -Review the plan output line by line against what Step 7's tfvars specify: -expect creates for `module.dc1_planes` (six `libvirt_network` resources), -`module.dc1_storage` + `module.office1_storage` (two `libvirt_pool` -resources), and `module.mesh_dc1_dc2` / `module.mesh_dc1_office1` / -`module.mesh_dc2_office1` (three more `libvirt_network` resources for the -D-100 dark-fiber triangle legs). Confirm nothing else is planned (no DC2 -plane resources -- that module block is commented out in `main.tf` and -should stay that way until gap #3's DATA half closes). +(`-input=false` so a missing required var ERRORS instead of hanging on an +interactive prompt in a non-interactive/agent shell -- DOCFIX-180.) + +Review the plan line by line. **As-executed (2026-07-10, DOCFIX-180): the plan +is 13 resources, not the 11 an earlier draft of this step implied:** +- `module.dc1_planes` -- six `libvirt_network` (the six planes) +- `module.dc1_storage` + `module.office1_storage` + **`module.dc2_storage`** -- + THREE `libvirt_pool` (DC2's storage pool is wired; only its planes are deferred) +- **`module.office1_network`** -- one `libvirt_network` (`office1-local`; + DOCFIX-163 added this, and an earlier draft's expected-list omitted it) +- `module.mesh_dc1_dc2` / `mesh_dc1_office1` / `mesh_dc2_office1` -- three + `libvirt_network` (the D-100 dark-fiber legs) + +Confirm NO DC2 *plane* resources (that block stays commented in `main.tf` until +NetBox assigns the supernet). Expect `Plan: 13 to add, 0 to change, 0 to destroy`. **GATE:** the plan matches this expectation exactly. If it doesn't (extra resources, missing resources, or an unexpected diff), STOP and reconcile diff --git a/scripts/prereqs/README.md b/scripts/prereqs/README.md new file mode 100644 index 0000000..3a3900e --- /dev/null +++ b/scripts/prereqs/README.md @@ -0,0 +1,43 @@ +# scripts/prereqs/ -- workstation/workspace prerequisite installers + +New operators start here. These scripts get a Debian/Ubuntu workstation (or the +vcloud host / Office1 control point) ready to run the VR1 DC-DC workflow, so a +runbook never assumes a runtime is already present (DOCFIX-178). Each installer +is **idempotent** (a satisfied prereq is a no-op) and supports `--check` +(read-only report) and `--dry-run` (print actions, mutate nothing). + +## Quick start + +```bash +bash scripts/prereqs/check-prereqs.sh # read-only: what's missing? +bash scripts/prereqs/install-all.sh --dry-run # preview +bash scripts/prereqs/install-all.sh # install everything missing (uses sudo) +``` + +Then re-login (or `newgrp libvirt`) if the virtualization installer just added +you to the `libvirt` group, and re-run `check-prereqs.sh` to confirm all green. + +## What each script covers + +| Script | Prereq | Needed by | +|---|---|---| +| `install-opentofu.sh` | OpenTofu `tofu` >= 1.6.0 (deb repo -> `/usr/bin/tofu`) | dc-dc Stage 1-6 `tofu` steps + `scripts/opentofu-validate.sh` | +| `install-virtualization.sh` | libvirt + qemu-kvm + virsh + `libvirt` group | dc-dc Stage 1+ (pools, networks, node VMs) | +| `install-jq.sh` | jq | cloud-assert + many ops scripts (and the test gauntlet) | +| `install-image-tools.sh` | qemu-utils (`qemu-img`) + `xorriso` | Stage 2/3 OPNsense build (`opnsense-prep-image.sh`, `opnsense-build-config-iso.sh`) | +| `check-prereqs.sh` | (read-only report of all of the above + curl/python3/sudo) | run first, on any new workspace | +| `install-all.sh` | orchestrator over the four installers + a final check | one-pass setup | +| `lib-prereq.sh` | shared helpers (sourced, not run) | the installers | + +## Scope + conventions + +- **Debian/Ubuntu (apt) only.** On another OS each installer fails loud (exit 3) + and tells you to install manually -- it does not guess a package manager. + `tofu` and the ISO builder have portable upstream options; see each script. +- **`sudo`** is used only for the actual package install / group change. The + `--check` and `--dry-run` modes never touch the system and never call sudo. +- **`tofu`, not `opentofu`.** The OpenTofu installer uses the deb method on + purpose: every runbook step and `scripts/opentofu-validate.sh` call the `tofu` + command, which the `opentofu` snap may not expose. +- Run via `bash scripts/prereqs/.sh` (these are not marked executable, per + this repo's convention). Harness: `tests/prereqs/run-tests.sh`. diff --git a/scripts/prereqs/check-prereqs.sh b/scripts/prereqs/check-prereqs.sh new file mode 100644 index 0000000..f58e32b --- /dev/null +++ b/scripts/prereqs/check-prereqs.sh @@ -0,0 +1,57 @@ +#!/usr/bin/env bash +# scripts/prereqs/check-prereqs.sh +# +# Read-only prereq report for the VR1 dc-dc workflow -- run this first on any new +# workstation/workspace to see what is missing before starting a stage. Mutates +# NOTHING. Each REQUIRED miss has a matching scripts/prereqs/install-*.sh. +# +# Exit: 0 all REQUIRED present | 1 one or more REQUIRED missing. ASCII + LF. +set -uo pipefail +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +. "$HERE/lib-prereq.sh" + +miss=0 +line() { #