diff --git a/docs/changelog-20260807-dc0-tailscale-install.md b/docs/changelog-20260807-dc0-tailscale-install.md new file mode 100644 index 0000000..b9246d1 --- /dev/null +++ b/docs/changelog-20260807-dc0-tailscale-install.md @@ -0,0 +1,66 @@ +# changelog 2026-08-07 -- dc0 .7 Tailscale install: office1-mirrored (untagged), site-tailscale.sh prep+forwarding + +Session goal: finish the dc0 `.7` Tailscale subnet-router install so the operator can log into +Headscale and approve the route -> unblocks dc0 Step 3.3 (Horizon-over-tailnet). Operator +directive: mirror the working Office1 router; defer tags/autoApprovers/ACL to the bare-metal build. + +Branch `dc-dc-stage5-preconditions`. Status authority is CURRENT-STATE.md (this file is history). + +## Measured ground truth (read-only, this session) + +- **dc0 `.7` (`10.12.8.7`, MAAS name `known-marten`, user `ubuntu`, sudo ok):** tailscale package + **ABSENT**; snap present; default route `via 10.12.4.1 dev enp2s0`; apt proxy + `http://10.12.8.6:8000/`; **IP forwarding OFF** (`ip_forward=0`, `all.forwarding=0`), no + `/etc/sysctl.d/99-tailscale.conf`. iptables 1.8.7 + iproute2 5.15 present (tailscale deps). +- **office1-tailscale (working reference):** deb `tailscale 1.98.9`, tailscaled active, forwarding + ON via `/etc/sysctl.d/99-tailscale.conf`; `tailscale debug prefs`: `Tags: None` (UNTAGGED), + `AdvertiseRoutes: ['10.10.0.0/22']`, `RouteAll: True` (accept-routes), `NoSNAT: False` (SNAT on), + control `https://tailscale.baldurkeep.com:443`. +- **Egress for the package:** `.7` cannot resolve/reach `pkgs.tailscale.com` (external DNS blocked; + apt-proxy will not forward it). The **rack and vcloud both have egress**; vcloud pulls office1's + exact `pool/tailscale_1.98.9_amd64.deb` (SHA256 `c739c29ec2342cc7df1a24cd102a3dfb51b359f79338d433ce8f052aeebc62ff`, + Depends: `iptables`). Method = staged `.deb` via a host with egress -> scp to `.7` -> `dpkg -i`. +- **Custody:** operator placed the plain Headscale pre-auth key on vcloud at + `~/vr1-tailnet-creds/headscale-preauth-vr1-dc0.key` (0600; sha256 `d2e74720...`); never read into + session context. Used only via `$TS_AUTHKEY_FILE`. + +## Changes (repo delivery; ALL GREEN before any live step) + +1. **`scripts/site-tailscale.sh` -- three verbs (`prep|install|check`) + untagged VR1 + forwarding.** + - New **`prep `**: installs the tailscale package from a rack-staged `.deb` (`$TS_DEB`, + since `.7` has no egress) and enables IP forwarding (writes `/etc/sysctl.d/99-tailscale.conf`, + `sysctl --system`, then ASSERTS v4+v6 forwarding took). + - **Tag now OPTIONAL**: `TS_TAG` defaults EMPTY = untagged (office1-mirrored, VR1); `install` + omits `--advertise-tags` and adds `--accept-routes` (office1 `RouteAll`); `check` skips the tag + assertion when untagged. `TS_TAG=tag:subnet-router` restores the D-129(iii) tagged design. + - **`check` now ASSERTS IP forwarding** -- the load-bearing subnet-router property. Root cause it + closes (advisor 2026-08-07): `tailscale up --advertise-routes` WARNS-and-succeeds with + forwarding off, so the join completes and Headscale approves the route while NOTHING forwards + to Horizon -- a green gate blind to the one property that makes the router work. + - Why (revert): `git checkout -- scripts/site-tailscale.sh` restores the tagged-only, + no-prep, no-forwarding form. Evidence: harness below. +2. **`tests/site-tailscale/run-tests.sh` -- 23 cases, up from ~15.** New failing-direction fixtures + for every new path: `prep-noforward`->4, `prep-nopkg-nodeb`->4, `prep-fromdeb`->0, `check-noforward` + ->1, `check-tag-happy`->0, `check-tag-notag`->1, `install-tag-happy`->0. Fakes `sysctl`/`dpkg` added. + Result: **23 pass / 0 fail**. Revert: restore the file with the script. +3. **`docs/design-decisions.md` -- D-129(iii) AMENDMENT (2026-08-07 b), [OPS], no new D-number.** + Records the operator's untagged/office1-mirror deferral (verbatim utterance quoted) and the + accepted VR1 exposure (key-expiry clock + no ACL star boundary yet); the Roosevelt tagged/ACL + design in ruling (b)/notes 1-2 is UNCHANGED, only deferred. Revert: delete the `### D-129(iii) + -- AMENDMENT (2026-08-07 b)` subsection. + +Gates: `repo-lint` 0 fail / 1 pre-existing legacy L1 warn; harness 23/23. + +## NOT YET DONE (live, gated -- pending this delivery's commit) + +Stage the `.deb` (vcloud dl + sha256 verify -> scp to `.7`) -> `dpkg -i` -> `prep vr1-dc0` -> +`install vr1-dc0` (untagged, key via `$TS_AUTHKEY_FILE`, removed after) -> **operator approves the +`10.12.8.0/22` route in Headscale** -> `check vr1-dc0` green -> confirm Horizon over the tailnet +(closes Step 3.3). dc1 `.7` is downstream of the dc1 node rebuild -- not prepped this session. + +## Findings queued for the close sweep + +- F1: `.7` MAAS machine name is still the auto name (`known-marten`), not `vr1-dc0-tailscale-01` + (naming convention owed; cosmetic, does not affect the join -- tailnet name set by `--hostname`). +- F2: raw `grep DOCFIX-` returns a stray `DOCFIX-1004` string somewhere (ledger-scan's next-free is + 213); verify it is not a real assignment collision. diff --git a/docs/design-decisions.md b/docs/design-decisions.md index bd91a19..900c801 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -5564,6 +5564,40 @@ assumed. **SEC row for key custody opens at key-mint time in the build.** Roosevelt analog: per-DC operator-access VPN, dedicated router, star ACL, per-DC key custody. +### D-129(iii) -- AMENDMENT (2026-08-07 b): VR1 install is UNTAGGED (office1-mirrored); TAGGED identity + autoApprovers + star ACL DEFERRED to bare-metal + +The IMPLEMENTATION NOTES above lean on a TAGGED-identity Headscale design (note 1) plus +autoApprovers (note 2) and the star ACL (ruling (b)). The operator DEFERRED all three for the +VR1 rehearsal and directed the per-DC `.7` routers be brought up exactly as the working Office1 +router was. **Operator directive, verbatim (2026-08-07):** *"For now, I don't care about the +other autoapprovers or acl policy. We can pin those for the bare metal install. For now, just +follow the steps we used to bring office1 online."* + +**CONSEQUENCE for the VR1 build (Roosevelt design in ruling (b)/notes 1-2 UNCHANGED -- only +deferred):** +- The VR1 `.7` routers join **UNTAGGED**, mirroring `office1-tailscale` (MEASURED 2026-08-07: + `Tags: None`, `AdvertiseRoutes: ['10.10.0.0/22']`, `RouteAll: True`, SNAT on, control + `https://tailscale.baldurkeep.com:443`). A plain (non-tag-scoped) pre-auth key suffices; no + `tagOwners`/`autoApprovers`/star-ACL policy is a prerequisite for VR1. +- Route approval is **MANUAL on Headscale** (operator approves the advertised metal-admin `/22` + after the router first advertises) -- there is no autoApprovers window to burn, which removes + the plain-key first-advertise hazard recorded on 2026-08-07 (a). +- **ACCEPTED cost, stated not glossed:** an untagged node carries Headscale's key-expiry clock + (the note-1 defect) and the star boundary is NOT yet enforced by ACL (an unpoliced Headscale is + allow-all). Both are the deferred bare-metal work; they are a KNOWN VR1 exposure, not an + oversight. The per-DC key-custody SEC row still opens at key-mint (notes above). +- **Tool encoding:** `scripts/site-tailscale.sh` defaults `TS_TAG` EMPTY (untagged) and its + `install` omits `--advertise-tags` while adding `--accept-routes` (office1's `RouteAll`); its + `check` skips the tag assertion when untagged. `TS_TAG=tag:subnet-router` restores the tagged + design for the bare-metal/Roosevelt build, at which point autoApprovers + the star ACL become + prerequisites again. A new `prep` verb installs the package (from a rack-staged `.deb`, since + the `.7` has no external egress) and enables IP forwarding -- the load-bearing subnet-router + property that `tailscale up --advertise-routes` warns-and-succeeds without, and that `check` now + asserts. + +**[OPS]** -- this is a phase-scoped deferral of an already-ruled design; no new D-number (GA-R3). +Roosevelt sessions grep D-129(iii): the tagged/ACL work lands THERE, and this note says so. + ## D-130: cloudinit seed-volume durable fix -- ignore_changes on the staging-derived create [ARCH] **Status:** ADOPTED 2026-07-19 (operator; GA-R5 -- question + utterance below; sweep Batch 1 item 3). diff --git a/scripts/site-tailscale.sh b/scripts/site-tailscale.sh index 1c9a2db..85c5340 100755 --- a/scripts/site-tailscale.sh +++ b/scripts/site-tailscale.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# scripts/site-tailscale.sh -- per-DC Tailscale subnet router. +# scripts/site-tailscale.sh -- per-DC Tailscale subnet router. # # The STANDING per-DC operator-access path (D-129(iii) + its 2026-08-07 amendment, rulings # a-d): a DEDICATED subnet-router VM in the utility band at .7 advertises THIS DC's metal-admin @@ -10,35 +10,46 @@ # # RUNS ON THE .7 SUBNET-ROUTER VM (10.12.8.7 dc0 / 10.12.68.7 dc1), not on vcloud/voffice1. # Invoke from a repo host over ssh, piping the script (no repo clone on the .7 VM needed): -# check: ssh -J voffice1, @10.12.8.7 'sudo bash -s' -- check vr1-dc0 < scripts/site-tailscale.sh -# install: same with 'install vr1-dc0'; the tagged pre-auth key is passed via $TS_AUTHKEY_FILE -# (a path on the .7 VM) or $TS_AUTHKEY -- NEVER on the command line, never printed. +# prep: ssh -J voffice1, @10.12.8.7 'sudo TS_DEB=/tmp/tailscale.deb bash -s' -- prep vr1-dc0 < scripts/site-tailscale.sh +# install: ssh ... 'sudo TS_AUTHKEY_FILE=/tmp/ts.key bash -s' -- install vr1-dc0 < scripts/site-tailscale.sh +# check: ssh ... 'sudo bash -s' -- check vr1-dc0 < scripts/site-tailscale.sh +# The pre-auth key is passed via $TS_AUTHKEY_FILE (a path on the .7 VM) or $TS_AUTHKEY -- +# NEVER on the command line, never printed. # -# HEADSCALE-SIDE PREREQUISITES (control-plane work, NOT this script -- done on -# tailscale.baldurkeep.com; deferred until control-plane access, D-129(iii) amdt note 4): -# - a TAGGED pre-auth key for tag:subnet-router (tag identity removes user-auth + key expiry); -# - the autoApprovers policy for tag:subnet-router + each DC metal-admin route, written BEFORE -# the router first advertises (Headscale does NOT approve retroactively); -# - the STAR ACL: tag:operators -> each DC's metal-admin CIDR; router<->router DENY (an -# unpoliced Headscale is allow-all, so this ACL IS the star boundary). -# This script is the NODE-SIDE half; it fails closed if the key/policy are absent. +# THREE VERBS, mapping onto the two-phase blocker: +# prep -- NODE-SIDE, runs any time: installs the tailscale .deb (from $TS_DEB, since the .7 +# has no external egress -- stage pool/tailscale__amd64.deb from the rack, which +# DOES have egress) and enables IP forwarding (the load-bearing subnet-router +# property: 'tailscale up --advertise-routes' WARNS-and-succeeds with forwarding off, +# so nothing reaches Horizon while every other check reads green). Idempotent. +# install -- NODE-SIDE join: 'tailscale up' advertising this DC's metal-admin /22 + --accept-routes. +# check -- NODE-SIDE verify: Running + route approved + forwarding + (optional) tag + .7 leg. # -# EXIT: 0 ok | 1 check failed | 2 bad args/unknown site | 4 install failed. -# Harness: tests/site-tailscale/run-tests.sh (offline; fakes tailscale). ASCII + LF only. +# TAG / ACL POSTURE -- VR1 mirrors the office1 router, which is UNTAGGED (MEASURED 2026-08-07: +# office1-tailscale Tags=None, AdvertiseRoutes=[10.10.0.0/22], RouteAll=True). D-129(iii)'s TAGGED +# identity + autoApprovers + star ACL are DEFERRED to the bare-metal/Roosevelt install (operator +# 2026-08-07: "We can pin those for the bare metal install ... just follow the steps we used to +# bring office1 online" -- D-129(iii) amendment). So TS_TAG defaults EMPTY (untagged) and route +# approval is MANUAL on Headscale. Set TS_TAG=tag:subnet-router to restore the tagged design (and +# then the Headscale autoApprovers/star ACL become prerequisites again). +# +# EXIT: 0 ok | 1 check failed | 2 bad args/unknown site | 4 prep/install failed. +# Harness: tests/site-tailscale/run-tests.sh (offline; fakes tailscale/ip/sysctl/dpkg). ASCII + LF. set -uo pipefail MODE="${1:-}"; SITE="${2:-}" -case "$MODE" in check|install) ;; *) - echo "usage: site-tailscale.sh " >&2; exit 2 ;; esac +case "$MODE" in prep|install|check) ;; *) + echo "usage: site-tailscale.sh " >&2; exit 2 ;; esac # --------------------------------------------------------------------------- # Constants (measured / ruled, one place). # --------------------------------------------------------------------------- # Control plane, MEASURED 2026-08-07 from office1-tailscale's ControlURL. LOGIN_SERVER="${TS_LOGIN_SERVER:-https://tailscale.baldurkeep.com:443}" -# Tag identity (D-129(iii) amdt note 1). The exact tag string is finalised WITH the Headscale -# autoApprovers/ACL policy (deferred); env-overridable so the policy and this stay in one value. -TAG="${TS_TAG:-tag:subnet-router}" +# Tag identity: DEFAULT EMPTY = untagged (VR1 office1-mirrored per the operator 2026-08-07 +# deferral above). ${TS_TAG-} not :- so an explicit TS_TAG= is honoured as untagged. Set +# TS_TAG=tag:subnet-router to opt into the D-129(iii) tagged design (Roosevelt/bare-metal). +TAG="${TS_TAG-}" # SNAT: ruling (d) = ON = tailscale's default, so NO --snat-subnet-routes flag is passed. # --------------------------------------------------------------------------- @@ -53,6 +64,8 @@ esac TS="${TAILSCALE:-tailscale}" # overridable so the harness injects a fake +SYSCTL="${SYSCTL:-sysctl}" # overridable so the harness fakes forwarding +SYSCTL_D="${TS_SYSCTL_D:-/etc/sysctl.d}" # overridable so the harness writes to a tmp dir say(){ printf '%s\n' "$*"; } # jget [...] -- navigate nested keys of `tailscale status --json`; no eval (a # bracket-key path in an eval string collides its own quotes). Prints Python repr (lists incl.). @@ -65,6 +78,15 @@ if d is None: break print("" if d is None else d)' "$@" 2>/dev/null; } +# fwd_ok -- both v4 and v6 forwarding on. Prints "v4=<> v6=<>" to stdout, returns 0/1. +fwd_ok(){ + local v4 v6 + v4="$("$SYSCTL" -n net.ipv4.ip_forward 2>/dev/null)" + v6="$("$SYSCTL" -n net.ipv6.conf.all.forwarding 2>/dev/null)" + printf 'v4=%s v6=%s' "$v4" "$v6" + [ "$v4" = 1 ] && [ "$v6" = 1 ] +} + # derive the router's OWN metal-admin leg CIDR at runtime and cross-check the table (hard rule # 3: prefer a runtime identity over a bare literal; a mismatch means wrong site or wrong VM). own_leg_matches(){ @@ -74,37 +96,73 @@ ip -o -4 addr show 2>/dev/null | grep -qE "inet ${base%.*}\.[0-9]+\.7/|inet ${base}\.7/" } +prep(){ + # 1. package present -- the .7 has NO external egress, so we do NOT apt from here. If tailscale + # is absent, install from a STAGED .deb ($TS_DEB, pulled on a host that has egress -- the + # rack). This mirrors office1's package (deb from pkgs.tailscale.com), not a snap. + if ! command -v "$TS" >/dev/null 2>&1; then + if [ -n "${TS_DEB:-}" ] && [ -s "${TS_DEB:-}" ]; then + "${DPKG:-dpkg}" -i "$TS_DEB" >/dev/null 2>&1 \ + || "${APTGET:-apt-get}" -y -f install >/dev/null 2>&1 \ + || { say "prep: install of $TS_DEB failed (deps? -- needs iptables, iproute2)"; return 4; } + else + say "prep: tailscale absent and no \$TS_DEB staged -- stage pool/tailscale__amd64.deb from the rack (it has egress) and pass TS_DEB="; return 4 + fi + fi + command -v "$TS" >/dev/null 2>&1 || { say "prep: tailscale still absent after install attempt"; return 4; } + # 2. IP forwarding -- mirror office1 (/etc/sysctl.d/99-tailscale.conf, v4+v6). Idempotent write. + local f="$SYSCTL_D/99-tailscale.conf" + printf '%s\n' '# Managed by site-tailscale.sh -- subnet-router forwarding (mirrors office1).' \ + 'net.ipv4.ip_forward = 1' 'net.ipv6.conf.all.forwarding = 1' > "$f" 2>/dev/null \ + || { say "prep: cannot write $f (need root)"; return 4; } + "$SYSCTL" --system >/dev/null 2>&1 + # 3. assert forwarding actually took (a written file that never applied is the silent trap). + local fw; fw="$(fwd_ok)" && say "prep: tailscale present ($("$TS" version 2>/dev/null | head -1)); IP forwarding $fw" \ + || { say "prep: IP forwarding not enabled ($fw) -- subnet router would not forward to Horizon"; return 4; } +} + +install(){ + command -v "$TS" >/dev/null 2>&1 || { say "install: tailscale not present -- run 'prep $SITE' first"; return 4; } + local key="${TS_AUTHKEY:-}" + [ -z "$key" ] && [ -n "${TS_AUTHKEY_FILE:-}" ] && [ -s "${TS_AUTHKEY_FILE:-}" ] && key="$(cat "$TS_AUTHKEY_FILE")" + [ -n "$key" ] || { say "install: no pre-auth key (\$TS_AUTHKEY / \$TS_AUTHKEY_FILE)"; return 4; } + # SNAT default (on) per ruling (d): no --snat-subnet-routes flag. --accept-routes mirrors + # office1's RouteAll=True. --advertise-tags ONLY when TS_TAG is set (untagged VR1 by default). + local tagflag=() + [ -n "$TAG" ] && tagflag=(--advertise-tags="$TAG") + "$TS" up --login-server="$LOGIN_SERVER" --authkey="$key" \ + --advertise-routes="$METAL_ADMIN_CIDR" --accept-routes "${tagflag[@]}" \ + --hostname="${SITE}-tailscale" >/dev/null 2>&1 \ + || { say "install: 'tailscale up' failed (key rejected, tag not authorised, or control-plane unreachable)"; return 4; } + say "install: tailscale up issued for $SITE advertising $METAL_ADMIN_CIDR${TAG:+ as $TAG}" + say "NOTE: Headscale must APPROVE the route (manual, or autoApprovers if tagged) -- run 'check' to confirm." +} + check(){ - local rc=0 st routes tags + local rc=0 st routes tags fw st="$(jget BackendState)" [ "$st" = "Running" ] || { say "FAIL: tailscale BackendState='$st' (not Running)"; rc=1; } routes="$(jget Self PrimaryRoutes | tr -d "[]' " )" case ",$routes," in *,"$METAL_ADMIN_CIDR",*) say "OK: advertising+approved $METAL_ADMIN_CIDR";; *) say "FAIL: $METAL_ADMIN_CIDR not in approved PrimaryRoutes ('$routes') -- route unadvertised or not approved by Headscale"; rc=1;; esac - tags="$(jget Self Tags | tr -d "[]' ")" - case ",$tags," in *,"$TAG",*) say "OK: tagged identity $TAG";; - *) say "FAIL: node not tagged $TAG (tags='$tags') -- untagged carries a key-expiry clock (note 1)"; rc=1;; esac + # IP forwarding -- the load-bearing property. A subnet router with forwarding off joins, + # advertises and gets approved while forwarding NOTHING (advisor 2026-08-07). Assert it here. + fw="$(fwd_ok)" && say "OK: IP forwarding enabled ($fw)" \ + || { say "FAIL: IP forwarding off ($fw) -- subnet router will not forward to Horizon"; rc=1; } + if [ -n "$TAG" ]; then + tags="$(jget Self Tags | tr -d "[]' ")" + case ",$tags," in *,"$TAG",*) say "OK: tagged identity $TAG";; + *) say "FAIL: node not tagged $TAG (tags='$tags') -- untagged carries a key-expiry clock (note 1)"; rc=1;; esac + else + say "OK: untagged join (VR1 office1-mirrored; TS_TAG empty -- tagged design deferred to bare-metal, D-129(iii) amdt)" + fi if own_leg_matches; then say "OK: router holds a .7 metal-admin leg in $METAL_ADMIN_CIDR" else say "FAIL: no .7 metal-admin leg in $METAL_ADMIN_CIDR on this host -- wrong VM or unconfigured"; rc=1; fi return $rc } -install(){ - command -v "$TS" >/dev/null 2>&1 || { say "install: tailscale not present -- install the package first"; return 4; } - local key="${TS_AUTHKEY:-}" - [ -z "$key" ] && [ -n "${TS_AUTHKEY_FILE:-}" ] && [ -s "$TS_AUTHKEY_FILE" ] && key="$(cat "$TS_AUTHKEY_FILE")" - [ -n "$key" ] || { say "install: no tagged pre-auth key (\$TS_AUTHKEY / \$TS_AUTHKEY_FILE) -- mint it on Headscale first (deferred)"; return 4; } - # SNAT default (on) per ruling (d): no --snat-subnet-routes flag. Idempotent: tailscale up is - # a state assertion. --advertise-tags requires the key to be authorised for the tag. - "$TS" up --login-server="$LOGIN_SERVER" --authkey="$key" \ - --advertise-routes="$METAL_ADMIN_CIDR" --advertise-tags="$TAG" \ - --hostname="${SITE}-tailscale" >/dev/null 2>&1 \ - || { say "install: 'tailscale up' failed (key rejected, tag not authorised, or control-plane unreachable)"; return 4; } - say "install: tailscale up issued for $SITE advertising $METAL_ADMIN_CIDR as $TAG" - say "NOTE: Headscale must APPROVE the route (autoApprovers, or manual) -- run 'check' to confirm." -} - case "$MODE" in - check) check || exit 1 ;; + prep) prep || exit 4 ;; install) install || exit 4 ; check || { say "install ran but check not yet green (route approval pending on Headscale)"; exit 1; } ;; + check) check || exit 1 ;; esac diff --git a/tests/site-tailscale/run-tests.sh b/tests/site-tailscale/run-tests.sh index f4a7822..2d328a5 100755 --- a/tests/site-tailscale/run-tests.sh +++ b/tests/site-tailscale/run-tests.sh @@ -1,20 +1,27 @@ #!/usr/bin/env bash # tests/site-tailscale/run-tests.sh -- offline harness for scripts/site-tailscale.sh. -# Fakes `tailscale` (status --json / up / version) and `ip`; real python3 for jget. NO network, -# mutates nothing outside mktemp. Proves each exit path can FIRE (a new assertion is decoration -# until it has a failing-direction fixture): -# check-happy -> 0 (Running + route approved + tagged + .7 leg) -# check-notrun -> 1 (BackendState != Running) -# check-noroute -> 1 (metal-admin CIDR not in approved PrimaryRoutes) -# check-notag -> 1 (node not tagged) -# check-noleg -> 1 (no .7 metal-admin leg on the host) -# install-nokey -> 4 (no tagged pre-auth key) -# install-noTS -> 4 (tailscale binary absent) -# install-upfail -> 4 (tailscale up rejected) -# install-happy -> 0 (up ok + check green) +# Fakes `tailscale` (status --json / up / version), `ip`, `sysctl` (forwarding), `dpkg` (prep +# install); real python3 for jget. NO network, mutates nothing outside mktemp. Proves each exit +# path can FIRE (a new assertion is decoration until it has a failing-direction fixture): +# prep-happy -> 0 (pkg present + forwarding on) +# prep-fromdeb -> 0 (pkg absent, $TS_DEB staged, dpkg installs it, forwarding on) +# prep-noforward -> 4 (forwarding sysctl reads 0 -- the load-bearing subnet-router property) +# prep-nopkg-nodeb-> 4 (pkg absent and no $TS_DEB staged) +# check-happy -> 0 (Running + route approved + forwarding + untagged + .7 leg) +# check-notrun -> 1 (BackendState != Running) +# check-noroute -> 1 (metal-admin CIDR not in approved PrimaryRoutes) +# check-noforward -> 1 (IP forwarding off -- green everywhere else, forwards nothing) +# check-noleg -> 1 (no .7 metal-admin leg on the host) +# check-tag-happy -> 0 (TS_TAG set + node tagged) +# check-tag-notag -> 1 (TS_TAG set but node NOT tagged) +# install-happy -> 0 (untagged up ok + check green) +# install-tag-happy-> 0 (tagged up ok + check green) +# install-nokey -> 4 (no pre-auth key) +# install-noTS -> 4 (tailscale binary absent -> "run prep first") +# install-upfail -> 4 (tailscale up rejected) # badmode/badsite/noargs -> 2 +# dc1 resolves its own CIDR (10.12.68.0/22), not dc0's # site-table rows carry MEASURED tags (hard rule 2) -# dc1 site resolves its own CIDR (10.12.68.0/22), not dc0's # Exit: 0 all pass | 1 any case failed. ASCII + LF. set -uo pipefail HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -22,7 +29,8 @@ SUT="$REPO/scripts/site-tailscale.sh" TMP="$(mktemp -d)"; trap 'rm -rf "$TMP"' EXIT PASS=0; FAIL=0 -BIN="$TMP/bin"; mkdir -p "$BIN" +BIN="$TMP/bin"; mkdir -p "$BIN" "$TMP/sysctl.d" +printf 'fake-deb-content\n' > "$TMP/fake.deb" # ---- fake tailscale -------------------------------------------------------- cat > "$BIN/tailscale" <<'FB' @@ -33,7 +41,8 @@ up) [ "$mode" = upfail ] && exit 1; exit 0;; status) # $2 == --json - st='"Running"'; routes="[\"$cidr\"]"; tags='["tag:subnet-router"]' + st='"Running"'; routes="[\"$cidr\"]" + tags='[]'; [ -n "${TS_TAG:-}" ] && tags="[\"$TS_TAG\"]" case "$mode" in notrun) st='"Stopped"';; noroute) routes='[]';; @@ -56,27 +65,66 @@ FB chmod +x "$BIN/ip" +# ---- fake sysctl (forwarding) --------------------------------------------- +cat > "$BIN/sysctl" <<'FB' +#!/usr/bin/env bash +fwd="${TS_FAKE_FWD:-1}" +case "$*" in + "--system") exit 0;; + "-n net.ipv4.ip_forward") echo "$fwd"; exit 0;; + "-n net.ipv6.conf.all.forwarding") echo "$fwd"; exit 0;; + *) exit 0;; +esac +FB +chmod +x "$BIN/sysctl" + +# ---- fake dpkg (prep install-from-deb) ------------------------------------ +# "installs" by copying the fake tailscale to $TS_INSTALL_TARGET so command -v succeeds after. +cat > "$BIN/dpkg" < local name="$1" exp="$2" mode="$3"; shift 3 local out code - out="$(PATH="$BIN:$PATH" TAILSCALE="$BIN/tailscale" TS_FAKE_MODE="$mode" \ - TS_AUTHKEY="${CASE_AUTHKEY:-}" TAILSCALE="${CASE_TS:-$BIN/tailscale}" \ + out="$(PATH="$BIN:$PATH" \ + TAILSCALE="${CASE_TS:-$BIN/tailscale}" \ + SYSCTL="$BIN/sysctl" DPKG="$BIN/dpkg" APTGET="/bin/false" \ + TS_SYSCTL_D="$TMP/sysctl.d" \ + TS_FAKE_MODE="$mode" TS_FAKE_FWD="${CASE_FWD:-1}" \ + TS_FAKE_CIDR="${CASE_CIDR:-10.12.8.0/22}" TS_FAKE_LEG="${CASE_LEG:-10.12.8.7}" \ + TS_TAG="${CASE_TAG-}" TS_AUTHKEY="${CASE_AUTHKEY:-}" \ + TS_DEB="${CASE_DEB:-}" TS_INSTALL_TARGET="${CASE_TS:-}" \ bash "$SUT" "$@" 2>&1)"; code=$? if [ "$code" -eq "$exp" ]; then PASS=$((PASS+1)); echo "PASS: $name (exit $code)" else FAIL=$((FAIL+1)); echo "FAIL: $name (got $code want $exp)"; echo "$out" | sed 's/^/ /'; fi } echo "=== site-tailscale harness ===" -# check cases (dc0) -run check-happy 0 happy check vr1-dc0 -run check-notrun 1 notrun check vr1-dc0 -run check-noroute 1 noroute check vr1-dc0 -run check-notag 1 notag check vr1-dc0 -run check-noleg 1 noleg check vr1-dc0 +# prep cases +run prep-happy 0 happy prep vr1-dc0 +CASE_TS="$BIN/staged-ts" CASE_DEB="$TMP/fake.deb" run prep-fromdeb 0 happy prep vr1-dc0 +CASE_FWD=0 run prep-noforward 4 happy prep vr1-dc0 +CASE_TS="$BIN/nope" run prep-nopkg-nodeb 4 happy prep vr1-dc0 +# check cases (default untagged) +run check-happy 0 happy check vr1-dc0 +run check-notrun 1 notrun check vr1-dc0 +run check-noroute 1 noroute check vr1-dc0 +CASE_FWD=0 run check-noforward 1 happy check vr1-dc0 +run check-noleg 1 noleg check vr1-dc0 +# tagged mode (opt-in via TS_TAG) +CASE_TAG="tag:subnet-router" run check-tag-happy 0 happy check vr1-dc0 +CASE_TAG="tag:subnet-router" run check-tag-notag 1 notag check vr1-dc0 # install cases -CASE_AUTHKEY="tskey-fake" run install-happy 0 happy install vr1-dc0 - run install-nokey 4 happy install vr1-dc0 -CASE_AUTHKEY="tskey-fake" run install-upfail 4 upfail install vr1-dc0 +CASE_AUTHKEY="tskey-fake" run install-happy 0 happy install vr1-dc0 +CASE_TAG="tag:subnet-router" CASE_AUTHKEY="tskey-fake" run install-tag-happy 0 happy install vr1-dc0 + run install-nokey 4 happy install vr1-dc0 +CASE_AUTHKEY="tskey-fake" run install-upfail 4 upfail install vr1-dc0 CASE_TS="$BIN/nope" CASE_AUTHKEY="tskey-fake" run install-noTS 4 happy install vr1-dc0 # arg/site validation run badmode 2 happy frobnicate vr1-dc0 @@ -84,11 +132,12 @@ run noargs 2 happy # dc1 resolves its OWN cidr (not dc0's) -- run check-happy with dc1 fixture -PASS_BEFORE=$PASS -out="$(PATH="$BIN:$PATH" TAILSCALE="$BIN/tailscale" TS_FAKE_MODE=happy TS_FAKE_CIDR='10.12.68.0/22' TS_FAKE_LEG='10.12.68.7' \ - bash "$SUT" check vr1-dc1 2>&1)"; code=$? -if [ "$code" -eq 0 ] && printf '%s' "$out" | grep -q '10.12.68.0/22'; then PASS=$((PASS+1)); echo "PASS: dc1 resolves 10.12.68.0/22 (exit 0)" -else FAIL=$((FAIL+1)); echo "FAIL: dc1 site (got $code)"; echo "$out" | sed 's/^/ /'; fi +CASE_CIDR='10.12.68.0/22' CASE_LEG='10.12.68.7' run dc1-cidr 0 happy check vr1-dc1 +out="$(PATH="$BIN:$PATH" TAILSCALE="$BIN/tailscale" SYSCTL="$BIN/sysctl" TS_SYSCTL_D="$TMP/sysctl.d" \ + TS_FAKE_MODE=happy TS_FAKE_CIDR='10.12.68.0/22' TS_FAKE_LEG='10.12.68.7' \ + bash "$SUT" check vr1-dc1 2>&1)" +if printf '%s' "$out" | grep -q '10.12.68.0/22'; then PASS=$((PASS+1)); echo "PASS: dc1 emits 10.12.68.0/22" +else FAIL=$((FAIL+1)); echo "FAIL: dc1 site cidr"; echo "$out" | sed 's/^/ /'; fi # site-table rows must carry MEASURED tags (hard rule 2) for s in 'vr1-dc0.*MEASURED' 'vr1-dc1.*MEASURED'; do