diff --git a/docs/changelog-20260713-config-xml-danger-sweep.md b/docs/changelog-20260713-config-xml-danger-sweep.md new file mode 100644 index 0000000..d26bb2e --- /dev/null +++ b/docs/changelog-20260713-config-xml-danger-sweep.md @@ -0,0 +1,57 @@ +# 2026-07-13 -- safety sweep: the config.xml push path is now a LIVE HAZARD + +After D-113(a2) was proven (edge config is API-managed), every instruction in the repo that +still said "render a config.xml and push it to the edge" became **actively dangerous**. This +sweep finds them and marks them. **No behaviour changes; no live system touched.** Warnings and +headers only. + +## Why this was urgent + +Office1's DHCP is now **API-managed**. A full `config.xml` push REPLACES `/conf/config.xml` +wholesale and drops ~667 migration-populated elements (measured 2026-07-13), including the only +two firewall pass rules on the box. Following the old runbook steps against the live edge would +clobber it. An instruction that destroys a working router is worse than no instruction. + +## What the sweep also caught (a PRE-EXISTING landmine, unrelated to today) + +`runbooks/dc-dc-phase2` Step 4 still instructs building a **config ISO** for DC1's edge -- but +**D-112 established that ISO can never be read**. `opnsense-importer -b` probes for a read-only +root; on a pre-installed nano the root is writable and a factory `/conf/config.xml` already +exists, so it `bootstrap_and_exit 0`s without enumerating a single device. That runbook has been +telling anyone who follows it to build an inert artifact and then wonder why the edge came up on +factory defaults -- which is EXACTLY the day we lost on 2026-07-12. It was never corrected at the +source. + +Its `WAN_IF`/`LAN_IF` "chicken-and-egg" discussion is likewise **moot**: that problem only exists +if you try to seed a full config before first boot. D-112(c) measures the real `vtnetN` mapping +*after* boot, where it is knowable. + +## Marked (5 files) + +| file | what was done | +|---|---| +| `runbooks/dc-dc-phase1-office1-standup.md` | DANGER banner; the config.xml render + ISO sub-steps REMOVED as runnable instructions (they would clobber the live Office1 edge). Image-prep retained. History preserved in git + the build changelog. | +| `runbooks/dc-dc-phase2-tofu-dc-substrate.md` | STOP banner on Step 4: the ISO is inert (D-112), full-config rendering is superseded (D-113(a2)), and the WAN_IF/LAN_IF problem is moot. Points at the proven Office1 path. | +| `scripts/opnsense-render-config.sh` | DANGER header at point-of-use. NOT dead -- under D-113(a2) it is to be REDUCED to a minimal bootstrap render (sshd + key + console). Until then, safe only for a brand-new, not-yet-booted edge. | +| `scripts/opnsense-build-config-iso.sh` | RETIRED header quoting the upstream source that proves the importer can never fire on nano. Kept (not deleted) because D-112 says retire it in the same change that reduces the template -- which has not landed. | +| `docs/dc-dc-deployment-workflow.md` | STALE-CONTENT warning at the top of the tooling gap register (the doc a fresh session reads first). | + +Changelogs were deliberately NOT touched: they are history, not instructions. Marking history is +noise; marking instructions is safety. + +## Verification + +Harnesses still green (headers only): `opnsense-render-config` 24 PASS, `opnsense-build-config-iso` +2 PASS, `opnsense-api` 21 PASS. `repo-lint` 0 fail. + +## Still OPEN (the actual D-113(a2) migration -- NOT done) + +Reduce the template to a minimal bootstrap, retire the config-ISO path and the +`opnsense-edge` module's `config_seed`/cdrom wiring, and rewrite Stage 3's edge steps around the +API. This sweep makes the repo SAFE in the meantime; it does not make it FINISHED. + +## Revert + + git revert + +Restores the previous (dangerous) instructions verbatim. Nothing else changes. diff --git a/docs/dc-dc-deployment-workflow.md b/docs/dc-dc-deployment-workflow.md index 05de60f..8797930 100644 --- a/docs/dc-dc-deployment-workflow.md +++ b/docs/dc-dc-deployment-workflow.md @@ -178,6 +178,32 @@ ## Tooling gap register (audited 2026-07-09; update as items close) +> ### !!! STALE-CONTENT WARNING (2026-07-13) -- READ BEFORE ACTING ON ANY OPNSENSE ITEM BELOW !!! +> +> Everything in this document describing OPNsense edge config delivery via a rendered +> `config.xml` and/or a **config ISO** is **OBSOLETE and, against the live Office1 edge, +> DANGEROUS**: +> +> - **The config ISO is INERT (D-112).** The OPNsense Configuration Importer can NEVER fire on a +> pre-installed nano image -- it probes for a read-only root, finds a writable one with a +> factory `/conf/config.xml`, and exits without enumerating any device. Nothing ever reads it. +> - **Full-`config.xml` delivery is SUPERSEDED (D-113(a2), ruled + PROVEN 2026-07-13).** Edge +> config is done over the **REST API** (`scripts/opnsense-api.sh`), proven end to end -- read +> AND write -- against the live Office1 edge. +> - **Office1's DHCP is now API-MANAGED.** A full rendered `config.xml` push to `10.10.0.1` would +> CLOBBER live state (it replaces `/conf/config.xml` wholesale, dropping ~667 +> migration-populated elements including the only 2 firewall pass rules). +> +> The as-built path that actually worked: prep image -> boot on factory defaults -> **D-112(c)** +> console bootstrap -> enable SSH + install key -> configure over SSH and the REST API. See +> `docs/changelog-20260713-opnsense-api-proven.md` and +> `docs/changelog-20260713-opnsense-api-write-proven.md`. +> +> **OPEN (D-113(a2), not yet done):** reduce `opentofu/templates/opnsense-config.xml.tmpl` to a +> MINIMAL bootstrap (sshd + root key + console), retire the config-ISO path and the module's +> `config_seed`/cdrom wiring, and rewrite Stage 3's edge steps around the API. Until that lands, +> treat every OPNsense step in this doc and in `runbooks/dc-dc-phase1/phase2` as reference-only. + Confirmed by grepping the actual repo (scripts/, tests/, netbox/, runbooks/), not inferred from the design doc alone. Ordered roughly by "you'll hit this first," not by size -- #1 is cheap and unblocks a reuse claim made elsewhere in this doc; diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 8b496f0..697cfb4 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -2011,6 +2011,19 @@ scp/install/reboot path against this edge. Template reduction to a minimal bootstrap (sshd + key + console) is the next step and is NOT yet done. +## SAFETY SWEEP 2026-07-13 (done after the checkpoint, operator away, repo-only) + +- Marked every instruction that still said "render config.xml and push it to the edge" as + DANGEROUS -- it would now CLOBBER Office1's API-managed DHCP. 5 files: both dc-dc runbooks, + both opnsense config scripts (point-of-use headers), and the workflow doc's gap register. +- **Caught a PRE-EXISTING landmine:** `dc-dc-phase2` Step 4 still told you to build a **config + ISO** -- which D-112 proved can NEVER be read on a nano image. That runbook has been directing + people straight into the failure that cost 2026-07-12. Now marked STOP at the source. Its + WAN_IF/LAN_IF "chicken-and-egg" note is moot for the same reason. +- Changelogs deliberately NOT touched (history, not instructions). +- Harnesses green (24/2/21 PASS), lint 0 fail. No live system touched. +- `docs/changelog-20260713-config-xml-danger-sweep.md`. + ## CHECKPOINT 2026-07-13 (session close; operator away, resumes in the morning) **LIVE STATE (measured, end of session):** the Office1 edge `office1-opnsense` (10.10.0.1 LAN / diff --git a/runbooks/dc-dc-phase1-office1-standup.md b/runbooks/dc-dc-phase1-office1-standup.md index ea94f14..8b97fd3 100644 --- a/runbooks/dc-dc-phase1-office1-standup.md +++ b/runbooks/dc-dc-phase1-office1-standup.md @@ -346,20 +346,42 @@ Expect no match (fresh standup) or a clear, understood match (re-run/repair case). -**MUTATION -- prepare Office1's OPNsense base image + config.xml + config ISO -(host-local files, gated)**, same mechanism as Stage 3's Step 4: +> ## !!! DANGER -- DO NOT RUN THIS STEP AGAINST THE LIVE OFFICE1 EDGE (2026-07-13) !!! +> +> **The Office1 edge is BUILT, ROUTING, and SERVING DHCP.** Its DHCP config is now +> **API-MANAGED** (D-113(a2), proven live 2026-07-13). Rendering a full `config.xml` and +> pushing it to `10.10.0.1` **WOULD CLOBBER THAT LIVE STATE** -- a full-config push replaces +> `/conf/config.xml` wholesale and drops ~667 migration-populated elements. +> +> **Two things below are now WRONG and must not be followed as written:** +> +> 1. **The config ISO cannot work AT ALL.** Per **D-112**, the OPNsense Configuration Importer +> can NEVER fire on a pre-installed nano image -- `opnsense-importer -b` probes for a +> read-only root, finds a writable one with a factory `/conf/config.xml` already present, and +> `bootstrap_and_exit 0`s without enumerating a single device. The ISO is INERT. Nothing was +> ever going to read it. +> 2. **Full-`config.xml` delivery is SUPERSEDED** by D-113(a2): edge configuration is done over +> the **REST API** (`scripts/opnsense-api.sh`), not by hand-authoring the appliance's +> GUI-owned XML. Hand-authoring that XML was the single root cause of DOCFIX-191 (lockout), +> DOCFIX-192 (dead console) and DOCFIX-193 (no DHCP). +> +> **What is actually true today:** Office1 was brought up by the D-112(c) console bootstrap, and +> is now managed over SSH + the REST API. This step is retained ONLY as the historical record of +> how the image is prepped. +> +> **If you are re-standing-up Office1 from scratch:** the template has NOT yet been reduced to a +> minimal bootstrap (sshd + root key + console) -- that is the open D-113(a2) work. Until it is, +> STOP and re-derive this step; do not render a full config and push it. + +**MUTATION -- prepare Office1's OPNsense base image (host-local files, gated)**: ```bash bash scripts/opnsense-prep-image.sh # see the script's own header for its exact args/output path ``` -```bash -export OPNSENSE_HOSTNAME=... DOMAIN=... ROOT_PASSWORD_HASH=... # plus every other token templates/README.md's legend marks REQUIRED for this site -bash scripts/opnsense-render-config.sh office1-opnsense-config.xml -``` -```bash -bash scripts/opnsense-build-config-iso.sh office1-opnsense-config.xml office1-opnsense-config.iso -``` -Every token filled above must be a real, measured/decided value for Office1 -specifically -- no copy-paste of DC1/DC2's config.xml content. + +The `config.xml` render + config-ISO sub-steps that used to live here are RETIRED -- see the +DANGER banner above. They are preserved verbatim in git history (and in +`docs/changelog-20260712-office1-opnsense-edge-build.md`) rather than kept here as runnable +instructions, because an instruction that clobbers a live router is worse than no instruction. **Repo change -- wire `module "office1_opnsense"` in `opentofu/main.tf`** diff --git a/runbooks/dc-dc-phase2-tofu-dc-substrate.md b/runbooks/dc-dc-phase2-tofu-dc-substrate.md index 61e43a4..ad7e089 100644 --- a/runbooks/dc-dc-phase2-tofu-dc-substrate.md +++ b/runbooks/dc-dc-phase2-tofu-dc-substrate.md @@ -209,6 +209,40 @@ ## Step 4 -- Prepare DC1's OPNsense base image + config.xml + config ISO [MUTATION: host-local files, gated] +> ## !!! STOP -- THIS STEP IS OBSOLETE AS WRITTEN (2026-07-13) !!! +> +> Nothing here clobbers a live system (DC1's edge does not exist yet), but **two of the three +> sub-steps below cannot achieve what they claim**, and following them will waste a session and +> produce a dead edge: +> +> 1. **The config ISO is INERT -- it can never be read.** Per **D-112** (root-caused from +> upstream source): `opnsense-importer -b` probes for a read-only root. On INSTALLER media the +> probe fails and the importer scans attached media. On our PRE-INSTALLED NANO image the root +> is writable AND a factory `/conf/config.xml` already exists, so BOTH conditions hold and it +> `bootstrap_and_exit 0`s **without enumerating a single device**. The Configuration Importer +> can NEVER fire on a nano image, by design. Building the ISO is pure waste. +> 2. **Full-`config.xml` rendering is SUPERSEDED** by **D-113(a2)** (ruled 2026-07-13): edge +> config is done over the **REST API** (`scripts/opnsense-api.sh`), which was proven end to +> end -- read AND write -- against the live Office1 edge on 2026-07-13. Hand-authoring the +> appliance's GUI-owned XML caused DOCFIX-191 (management lockout), DOCFIX-192 (dead console) +> and DOCFIX-193 (no DHCP, and an ISC `` block that would have been inert against a Kea +> backend). None of those bugs is expressible through the API. +> +> **The `WAN_IF`/`LAN_IF` chicken-and-egg problem discussed below is therefore MOOT** -- it was an +> artifact of trying to seed a full config before first boot. The D-112(c) bootstrap (boot on +> factory defaults -> reach the console -> enable SSH + install the key -> configure over the +> network) measures the real `vtnetN` mapping *after* boot, where it is knowable. +> +> **What DC1's edge should actually do (the proven Office1 path):** prep the image -> boot it -> +> D-112(c) console bootstrap -> configure over SSH + the REST API. See +> `docs/changelog-20260712-office1-opnsense-edge-build.md`, +> `docs/changelog-20260713-opnsense-api-proven.md`, and +> `docs/changelog-20260713-opnsense-api-write-proven.md`. +> +> **This step is NOT yet rewritten.** Reducing the template to a minimal bootstrap and rewriting +> Stage 3's edge steps around the API is the open D-113(a2) work. Until that lands, STOP here and +> re-derive rather than following the sub-steps below. + This step produces plain files on the vcloud host filesystem -- no libvirt or MAAS object is created yet. Run from the vcloud host (or wherever these scripts execute with reach to write there). diff --git a/scripts/opnsense-build-config-iso.sh b/scripts/opnsense-build-config-iso.sh index 27877bd..37ffc73 100644 --- a/scripts/opnsense-build-config-iso.sh +++ b/scripts/opnsense-build-config-iso.sh @@ -1,6 +1,37 @@ #!/usr/bin/env bash # scripts/opnsense-build-config-iso.sh # +# ############################################################################ +# # RETIRED (D-112) -- THIS PATH CANNOT WORK. DO NOT USE IT. # +# # # +# # The ISO this builds is INERT. It is never read. Root-caused from # +# # upstream source (opnsense/core, src/sbin/opnsense-importer): # +# # # +# # INSTALL="/.probe.for.readonly" # +# # touch ${INSTALL} 2> /dev/null # +# # if [ -f ${INSTALL} -a -f /conf/config.xml ]; then # +# # bootstrap_and_exit 0 # +# # # +# # The touch is a PROBE FOR A READ-ONLY ROOT. On INSTALLER media the root # +# # is read-only, the probe fails, and the importer goes on to scan attached # +# # media (cd9660 included). On our PRE-INSTALLED NANO image the root is # +# # WRITABLE and a factory /conf/config.xml ALREADY EXISTS -- so both # +# # conditions hold and it bootstrap_and_exit 0's IMMEDIATELY, without ever # +# # enumerating a single device. # +# # # +# # The Configuration Importer can NEVER fire on a nano image, BY DESIGN. # +# # Nothing was ever going to read this ISO. # +# # # +# # The working path is D-112(c): boot on factory defaults -> console # +# # bootstrap -> enable SSH + install the key -> configure over the network # +# # (now the REST API, per D-113(a2): scripts/opnsense-api.sh). # +# # # +# # Kept (not deleted) only because D-112 says to retire it in the same # +# # change that reduces the template -- which has NOT yet landed. Its # +# # harness still passes; that proves the ISO is well-formed, NOT that # +# # anything reads it. # +# ############################################################################ +# # Builds a plain ISO9660 image containing /conf/config.xml, for OPNsense's # Configuration Importer (see opentofu/README.md's "OPNsense deployment # research" section) -- ISO9660 support was added to the Importer diff --git a/scripts/opnsense-render-config.sh b/scripts/opnsense-render-config.sh index 1d90404..852f1d8 100644 --- a/scripts/opnsense-render-config.sh +++ b/scripts/opnsense-render-config.sh @@ -1,6 +1,27 @@ #!/usr/bin/env bash # scripts/opnsense-render-config.sh # +# ############################################################################ +# # DANGER (2026-07-13): DO NOT PUSH THIS OUTPUT TO THE LIVE OFFICE1 EDGE. # +# # # +# # This renders a FULL config.xml. Installing a full config.xml REPLACES # +# # /conf/config.xml wholesale and DROPS ~667 migration-populated elements # +# # (measured) -- including the only 2 firewall pass rules on the box. # +# # # +# # Office1's DHCP is now API-MANAGED (D-113(a2), proven live 2026-07-13). # +# # Pushing a rendered config.xml to 10.10.0.1 WOULD CLOBBER THAT STATE. # +# # Configure the edge with scripts/opnsense-api.sh instead. # +# # # +# # This script is NOT dead: under D-113(a2) it is to be REDUCED to render a # +# # MINIMAL BOOTSTRAP config (sshd + root key + console, nothing else), for # +# # first-boot only. That reduction is NOT YET DONE. Until it lands, the # +# # only safe use of this script is rendering for a BRAND-NEW, NOT-YET-BOOTED # +# # edge -- never against one that is already up. # +# # # +# # Related: D-112 (the config-ISO delivery path is INERT -- the OPNsense # +# # Configuration Importer can never fire on a pre-installed nano image). # +# ############################################################################ +# # Renders opentofu/templates/opnsense-config.xml.tmpl into a real config.xml # by substituting {{TOKEN}} markers (this repo's existing clientdocs # convention, reused here) from required environment variables -- no