diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 7fd97ee..fcce2a0 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -4014,9 +4014,29 @@ ## D-129: OPNsense edge plugin / add-on base profile (VR1 -> Roosevelt) -**Status:** PROPOSED 2026-07-18 (operator asked for a plugin/add-on review of the 26.7 edge for -additions that benefit the stack going forward). Options presented below; NOTHING installed. The -operator rules which buckets are adopted; only then does implementation proceed. +**Status:** OPEN (PARTIALLY RULED 2026-07-18) -- profile adopted, FOUR sub-decisions still OPEN. +Operator directive: "install the OPNsense recommendations and pin the full engineering for the rest for +the appropriate steps in the upcoming deployment(s)". The +plugin PROFILE is adopted: `os-qemu-guest-agent` + `os-iperf` directed for the VR1 edge via the D-113 +firmware API (`core/firmware/install/`); the live install on `office1-opnsense` is operator-gated +(auto-mode classifier blocked the agent's POST -- cloud mutations stay gated), verification pending. The +rest is PINNED to its deployment step (mapping below). **FOUR sub-decisions remain OPEN** (a one-line +directive does not rule these -- they were flagged as decisions, not adds): (i) COS-scrapes-the-edge +scope, (ii) `os-frr` dynamic routing, (iii) `os-tailscale` subnet-router consolidation, (iv) +edge-as-site-NTP. + +**Pinned at deployment steps (the "engineering for the rest").** +- **VR1 edge base profile = {`os-qemu-guest-agent`, `os-iperf`}** -> applied at DC-edge bring-up. + PINNED in `runbooks/dc-dc-phase2-tofu-dc-substrate.md` (Model B inner-apply / edge section). The + reusable per-profile installer (thin wrapper over `scripts/opnsense-api.sh` firmware install, + idempotent, `--check`) is itself engineering to BUILD at that step -- not built now (avoids + premature tooling; `opnsense-api.sh` already carries the install this session). +- **Roosevelt metal edge profile = {`os-smart`, `os-nut`|`os-apcupsd`, `os-cpu-microcode-intel`/`-amd`, + `os-lldpd`}** -> recorded here + in the review doc; enable at the Roosevelt edge build (inert on VR1 VMs). +- **Observability (`os-node_exporter`)** -> PINNED in `runbooks/dc-dc-phase6-designate-cos-magnum.md` + Step 10, gated on the COS-scrapes-edge scope sub-decision (i). +- **Security hardening** (Suricata content, crowdsec, Zenarmor, acme, SIEM) -> DEFERRED per closed-test + posture to post-teardown / Roosevelt; recorded in the review doc, not pinned to a VR1 step. **Problem.** `office1-opnsense` was updated to OPNsense 26.7 and is a near-stock install (only `os-git-backup`, itself unconfigured). The edge is the reusable per-site boundary profile that will diff --git a/docs/opnsense-edge-addon-review-20260718.md b/docs/opnsense-edge-addon-review-20260718.md index 0eb38e7..b8b9e7b 100644 --- a/docs/opnsense-edge-addon-review-20260718.md +++ b/docs/opnsense-edge-addon-review-20260718.md @@ -2,7 +2,10 @@ **Subject.** `office1-opnsense`, **OPNsense 26.7** (amd64), reviewed for plugin/add-on additions that benefit the VR1 build and transfer to the Roosevelt edge. Read-only review; -**nothing was installed.** Adopting any item is a design decision (see D-129, PROPOSED). +**nothing was installed by the review.** **Adoption ruling: D-129 (PARTIALLY RULED 2026-07-18)** -- +operator directive adopted the VR1 edge profile `{os-qemu-guest-agent, os-iperf}` (live install +operator-gated); the rest is pinned to its deployment step; four sub-decisions (COS-edge scope, os-frr, +os-tailscale, edge-NTP) remain OPEN. See D-129 for the mapping. ## Method + the lens that sorts the catalog diff --git a/docs/session-ledger.md b/docs/session-ledger.md index 6d1f4be..d4905fe 100644 --- a/docs/session-ledger.md +++ b/docs/session-ledger.md @@ -36,7 +36,8 @@ per amendment; the off-EOL-1.8.8 path remains OPEN), D-071 (routine update cadence + Juju controller patch policy -- AMENDED, still PROPOSED, awaiting operator ratification; gated on the pre-DC-DC controller HA/backup session), D-129 (OPNsense edge plugin/add-on base profile -- - PROPOSED 2026-07-18; review in `docs/opnsense-edge-addon-review-20260718.md`, operator rules buckets). + OPEN / partially ruled 2026-07-18: VR1 profile adopted, FOUR sub-decisions still OPEN; review in + `docs/opnsense-edge-addon-review-20260718.md`). - **OPEN security rows:** SEC-001, SEC-003, SEC-004, SEC-005, SEC-006, SEC-007, SEC-008, **SEC-010**. (New since last re-seed: **SEC-010** -- the rack transit-leg FORWARD-drop artifact -- is COMMITTED but stays OPEN until applied + verified on vvr1-dc0. SEC-008 -- the vcloud re-enrolment key -- @@ -750,9 +751,17 @@ record for Roosevelt, inert in VR1 VMs); sim scaffolding (`os-qemu-guest-agent`, VR1 VM edges only, aids D-126/D-127 reboot work); diagnostics (`os-iperf` for transit validation); observability (`os-node_exporter` CONDITIONAL on a COS-scope ruling -- D-105 scoped COS per-DC inside the cloud, edge is -outside); DEFER all security hardening per closed-test posture. D-129 OPEN decision points: which buckets; -COS-edge scope; `os-frr` (static transit today); `os-tailscale` (consolidate the D-107 subnet-router VM); -edge-as-site-NTP. **NOTHING installed** -- review only. Docs committed on branch +outside); DEFER all security hardening per closed-test posture. **UPDATE 2026-07-18 -- operator directive "install the recommendations + pin the rest".** D-129 now +PARTIALLY RULED: VR1 edge profile **{`os-qemu-guest-agent`, `os-iperf`}** adopted. Live install on +`office1-opnsense` via the D-113 firmware API is **operator-GATED** -- the auto-mode classifier blocked the +agent's `core/firmware/install` POST (cloud mutations stay gated even during batch), so the operator runs it +via `!` (or approves); **verification pending** (confirm the `qemu-guest-agent` DAEMON runs, not just the +pkg, + edge DHCP/NAT unbounced). The "rest" is PINNED (not built now, per advisor -- pin = record at the +step): metal profile {`os-smart`/`os-nut`/`os-cpu-microcode`/`os-lldpd`} -> Roosevelt edge build (inert on +VR1 VMs); `os-node_exporter` -> `dc-dc-phase6` Step 10 gated on COS-edge scope; VR1 profile -> `dc-dc-phase2` +edge section (+ build the reusable profile-installer there); security -> deferred (posture). **FOUR +sub-decisions STILL OPEN** (COS-edge scope, `os-frr`, `os-tailscale`, edge-NTP) -- a one-line directive does +not rule them. As-built row updated (FreeBSD base 14.3->15.1 too). Docs committed on branch `dc-dc-stage3-phase2-dc-substrate` (not pushed; operator away from workstations). ## OPEN WORK -- VR0 / DC0 (the running testcloud) diff --git a/docs/vr1-office1-as-built.md b/docs/vr1-office1-as-built.md index cd4baee..6ef31e9 100644 --- a/docs/vr1-office1-as-built.md +++ b/docs/vr1-office1-as-built.md @@ -39,7 +39,7 @@ | Host | Address | What runs on it | Level | |---|---|---|---| | **vcloud** | `10.17.11.248` (lab), `10.10.0.10` on `virbr2`, `172.30.1.1` on `virbr11` | libvirt/KVM hypervisor; OpenTofu runs from here | L1 (itself a KVM guest) | -| **office1-opnsense** | LAN `10.10.0.1` / WAN `172.30.1.2` | **OPNsense 26.7** edge router (updated from 26.1.11 on 2026-07-18; 0 outdated pkgs): routing, NAT, **Kea DHCP**, firewall. Plugins: `os-git-backup` only (UNCONFIGURED). See `docs/opnsense-edge-addon-review-20260718.md` + D-129. | L2 | +| **office1-opnsense** | LAN `10.10.0.1` / WAN `172.30.1.2` | **OPNsense 26.7** (FreeBSD base **15.1**-RELEASE-p1; both bumped from 26.1.11 / FreeBSD 14.3 on 2026-07-18; 0 outdated pkgs) edge router: routing, NAT, **Kea DHCP**, firewall. Plugins: `os-git-backup` (UNCONFIGURED); `os-qemu-guest-agent` + `os-iperf` directed 2026-07-18 (D-129 VR1 profile, firmware-API install operator-gated -- confirm live). See `docs/opnsense-edge-addon-review-20260718.md` + D-129. | L2 | | **voffice1** | `10.10.0.20` (Kea reservation) | **MAAS 3.7.2** region+rack + PostgreSQL 16.14; **LXD 5.21.5** (registered to MAAS as VM host `office1-lxd`) | L2 | | **office1-netbox** | `10.10.1.10` | **NetBox 4.6** (netbox-docker; `/opt/netbox-docker`, `docker compose up -d`) on `:8000`; **`restart: unless-stopped`** on all 5 services (2026-07-17 -- survives instance reboot) | L3 (LXD VM) | | **office1-tailscale** | `10.10.1.11` / tailnet **`100.64.0.53`** (`fd7a:115c:a1e0::35`) | **Tailscale 1.98.8**, subnet router advertising `10.10.0.0/22`. Joined the **SELF-HOSTED** control plane `https://tailscale.baldurkeep.com:443` | L3 (LXD VM) | diff --git a/runbooks/dc-dc-phase2-tofu-dc-substrate.md b/runbooks/dc-dc-phase2-tofu-dc-substrate.md index 23d397d..0e58970 100644 --- a/runbooks/dc-dc-phase2-tofu-dc-substrate.md +++ b/runbooks/dc-dc-phase2-tofu-dc-substrate.md @@ -163,6 +163,15 @@ Office1 REGION; MAAS discovers the inner node VMs (Step 10). Still DEFERRED (DOCFIX-179). - **E. netem** (Step 11) -- UNCHANGED: runs on the vcloud-level mesh bridges (virbr7), outer. +- **Edge plugin profile (D-129, pinned 2026-07-18).** After the DC edge boots and its REST API is up + (inner apply C), apply the VR1 edge base profile **{`os-qemu-guest-agent`, `os-iperf`}** via the + D-113 firmware API (`scripts/opnsense-api.sh POST core/firmware/install/`) -- `qemu-guest-agent` + gives the inner libvirt host clean guest coordination (graceful shutdown/fsfreeze/IP report; pairs + with D-127 autostart), `iperf` validates the transit path. **Verify the `qemu-guest-agent` daemon is + RUNNING, not just the package present.** BUILD a small idempotent profile-installer here (thin wrapper + over `opnsense-api.sh`, `--check`) so every DC edge is provisioned identically. NOT the Roosevelt-metal + profile (that set is baremetal-only) and NOT `os-node_exporter` (gated on the COS-edge scope, D-129). + Node count is R-3 = 3 control + 2 compute + **4 storage = 9/DC** (Step 6's "8" is superseded). The Model A single-root layout is preserved as a revert (git tag `model-a-fallback` + `docs/model-a-fallback-plan.md`). Sizing derived by `scripts/dc-dc-whole-host-budget.py` (870/1024 GiB, FIT). SEC-010 DC-LOCAL is enforced diff --git a/runbooks/dc-dc-phase6-designate-cos-magnum.md b/runbooks/dc-dc-phase6-designate-cos-magnum.md index b63e6cb..8a347e0 100644 --- a/runbooks/dc-dc-phase6-designate-cos-magnum.md +++ b/runbooks/dc-dc-phase6-designate-cos-magnum.md @@ -479,6 +479,13 @@ only its scope (D-105: per-DC, in scope, no Office1 roll-up). Do not pick one silently. Present the standard options to the operator at execution time: +!!! **Edge scrape is a SEPARATE scope decision (D-129, pinned 2026-07-18).** D-105 scopes COS to the + cloud (per-DC, no Office1 roll-up). The DC OPNsense **edge sits OUTSIDE** that failure domain, so + "should COS scrape the boundary edge?" is its own sub-decision -- NOT ruled by D-105. IF the operator + rules the edge in-scope, install **`os-node_exporter`** on the DC edge (D-113 firmware API) and add + it as a Prometheus scrape target; the edge LAN faces provider-public (D-100), so reachability from + the in-cloud COS is a segmentation call to make explicitly. If out-of-scope, do nothing here. + - **Option A -- `cos-lite` bundle in a dedicated per-DC `cos` Juju model** (Canonical's standard COS Lite: Prometheus, Grafana, Loki, Alertmanager, Traefik ingress), related to this DC's `openstack` model applications via