# Session changelog 2026-07-31 (part 2) -- snap proxy apply + IPv6 scoping GA-R2: ONE changelog per session. This is a SEPARATE session from `docs/changelog-20260731-prefer-ipv6-research.md` (same date, different session); that file is closed and is not appended to here. Branch `dc-dc-stage5-preconditions`. Status claims live ONLY in `docs/CURRENT-STATE.md`; nothing here is a status claim. --- ## Item 1 -- FINDING: the `dc-snap-proxy.sh` review's SIMPLIFICATION PLAN was never ## landed on any surface and is LOST **What.** No artifact change. A finding, recorded because the next action in the prior session's close-out cannot be executed as written. `docs/session-ledger.md:296` gives the next step as "apply the review cuts, re-run the mutation pass (S8 invalidates it)", and `docs/audit/queued-findings-20260731-stage5-deploy.txt:145-146` (N2) says "The dc-snap-proxy simplification plan is NOT applied. Applying it REQUIRES re-running the mutation pass -- S8 invalidates M32-M39, M42-M47, M49-M52 as evidence." **MEASURED: the plan those `S`-numbers refer to exists nowhere in the repo.** A repo-wide grep for `\bS8\b` over `*.md` / `*.txt` / `*.sh` returns exactly two live hits -- the ledger line and the N2 line above, both of which CITE the plan without containing it -- plus three unrelated 2026-07-07 archive hits (`docs/archive/script-quality-findings-20260707.md`, an S1-S8 consolidation proposal set about `KEYSTONE_VIP` and D-003 FIP bounds, a different subject entirely). The mutation IDs it invalidates ARE recorded (`docs/audit/stage5-snap-proxy-measurements-20260731.txt:505,516,523`); the simplification items themselves are not. **So N2's wording understates it.** "NOT applied" reads as deferred work that is waiting somewhere. It is not deferred -- it lived only in the review agent's transcript and was lost at the session boundary. This is the repo's own top-tier failure mode (a finding that exists only in a transcript), landing on a finding ABOUT losing findings. **DECISION, and the reasoning stated so it can be overturned:** the simplification is NOT re-derived. It was quality-only work on a tool that is about to be applied for the first time; re-deriving a lost plan and then re-running a 53-mutation pass to restore evidence for it is precisely the record-churn the 2026-07-30 standing operator directive shuts down ("Further hardening, auditing, or record repair is OUT OF SCOPE next session unless it directly blocks the deploy"), and it is the GA-F06 circuit-breaker case. **Consequence that follows and is NOT a loss:** because S8 is never applied, its invalidation of M32-M39/M42-M47/M49-M52 never fires, so the prior session's 53/53 mutation evidence STANDS unchanged for every assertion this session did not touch. **Revert.** Nothing to revert (no artifact changed). To reverse the decision, re-derive the plan from scratch against the current file and re-run the full mutation pass -- the old S-numbers are not recoverable and must not be cited. --- ## Item 2 -- FIX BUG-4: `dpkg -s squid` gates replaced by a BINARY assertion, at ## BOTH sites (`check` AND `install`) **What.** `scripts/dc-snap-proxy.sh` (the `check` package gate and `do_install`'s prereq) + `tests/dc-snap-proxy/run-tests.sh` (the `dpkg` PATH fake removed, a real binary seam added, T41 re-pointed, T41b and T20b NEW). **Why.** BUG-4 from the independent review (`docs/audit/queued-findings-20260731-stage5-deploy.txt:64-66`): `dpkg -s squid` exits 0 for `Status: deinstall ok config-files`, so a PURGED squid reports as present. The existence-vs-content class this repo already has a scar from. `[ -x /usr/sbin/squid ]` is shorter and strictly stronger. **The review named ONE site; there are TWO.** The same gate sat at `do_check` AND at `do_install`'s prereq. Fixing only `check` would have left `install` false-passing its prereq on a purged package and then dying confusingly at `squid -k parse` or at unit start -- a worse failure than the one being fixed, because it surfaces further from its cause. Both are fixed. **Written through the EXISTING test seam, not around it.** The assertion is `[ -x "${SYSROOT}/usr/sbin/squid" ]`. `SYSROOT` is empty in every production invocation, so the live path is byte-identical to `/usr/sbin/squid`; the harness points it at its mktemp tree. This is the script's own documented idiom (`DC_SNAP_PROXY_SYSROOT`, with in-repo precedent in `maas-region-power-key.sh`, `creds-audit.sh`, `cloud-snapshot.sh`) and it keeps the assertion drivable in BOTH directions, which the `dpkg` fake also was -- the fix does not cost a failure direction. **Test cases, and why each exists.** - **T41** (re-pointed): binary absent -> FAIL. The old case drove `dpkg.mode`. - **T41b** (NEW): the path is present but NOT executable -- the exact `deinstall ok config-files` residue shape. **This is the case the OLD assertion could not fail on**, i.e. the one that makes the fix mean something. Without it the fix would be untested in its own direction. - **T20b** (NEW): `do_install` is never EXECUTED by this harness (it needs root and a real rack), so its half of the fix would otherwise carry NO assertion at all. Asserted structurally, following T18's established precedent of scoping a grep to a function body: zero `dpkg -s` gates anywhere in the file, AND `INSTALL_BODY` carries the `[ ! -x ... ]` form. **Mutation-proven (this repo's rule: an assertion nobody can turn red is decoration).** Three mutations, each neutering a PREDICATE, script restored and sha256-verified byte-identical after each: - `check`'s binary predicate -> `if true` : **T41 AND T41b both red** (53/2). - `install`'s prereq -> `if ! dpkg -s squid ...` : **T20b red** (53/1). - `install`'s prereq -> `if false` : **T20b red** on its second branch (53/1). **Revert.** `git checkout -- scripts/dc-snap-proxy.sh tests/dc-snap-proxy/run-tests.sh` (reverts items 2-4 together; they are edits to the same two files). --- ## Item 3 -- FIX BUG-1: the probe ACL was DEAD, and it is now asserted **What.** `gen_squid_conf`'s `acl snap_probe src 127.0.0.1/32` -> `src ${LISTEN}/32`, its comment block rewritten to describe the real mechanism, + T23b NEW. **Why.** BUG-1: squid binds `${LISTEN}:${PROXY_PORT}` only, so nothing can ever arrive on loopback -- the rule matched nothing. The probes were actually admitted by `snap_clients` (because `LISTEN` lies inside `CLIENT_CIDR`), and the comment misdescribed its own mechanism. **The hazard was live, not cosmetic:** anyone narrowing `CLIENT_CIDR` would break `check`, and the rule they would reach for to fix it would not have helped. **Why the stronger fix was taken rather than a comment-only fix.** The obvious minimal action was to correct the comment and leave the dead rule, on the grounds that changing generated content pulls fixtures with it. **Measured, it does not:** the harness renders its fixtures by calling the script's OWN generators (`render gen_squid_conf`, `run-tests.sh:312-318`), never by copying config text into the test file, so generated-content changes cost nothing in fixture maintenance. That removed the only argument for the weaker fix. **Safe in both directions by construction** -- stated because the source-selection step is REASONED, not yet measured on the rack (hard rule 2). A host connecting to one of its own local addresses is expected to select that address as the source, so `${LISTEN}/32` is the expected match; if that is wrong, the probe still falls through to `snap_clients`, because `LISTEN` is inside `CLIENT_CIDR`. So the change cannot regress `check`, and the first live run's access log will say which rule actually admitted the probe. **T23b (NEW)** asserts the RENDERED artifact -- the probe ACL is the LISTEN address and is NOT a loopback rule -- never the comment that describes it. BUG-1's real defect was that a comment and its code disagreed, so an assertion on prose would repeat the defect. **Mutation-proven.** Reverting the ACL to `127.0.0.1/32` turns **T23b red** (54/1); script restored sha256-identical. --- ## Item 4 -- BUG-3 DOCUMENTED IN THE SCRIPT, deliberately NOT "fixed" **What.** A new block in `scripts/dc-snap-proxy.sh`'s header, under the existing "READ THIS BEFORE TRIAGING A REFUSE" section. No behavior change. **Why not fixed.** BUG-3 is that the DENY probe's expected string `CONNECT tunnel failed, response 403` was MEASURED from a **port** deny (`deny CONNECT !SSL_ports`), never from the **dstdomain** deny (`deny !snap_store`) that the probe actually exercises. It cannot be fixed in the repo, because the correct value is a fact about a live squid that nobody has yet observed -- inventing one would be exactly the fabricated-value failure this repo calls the cardinal sin. Guessing a second plausible string would be worse than the bug. **Why it is not merely cosmetic.** If squid answers a dstdomain denial differently, that branch falls to `refuse` -> exit 3, and `check` can then NEVER PASS. Under GA-R6 a gate that cannot PASS blocks a stage close exactly as hard as one that cannot fail, so this is a real gate defect and not a probe nicety. **The protocol is now IN THE FILE** rather than in a queued-findings entry the next reader will not open: a REFUSE on the deny probe at first run is the EXPECTED, HANDLED path; read the actual output; re-point the assertion to the MEASURED shape; keep the three-outcome structure so an unrecognised result still REFUSES; prove the new assertion can fail. This is also the answer to the close-out's "confirm BUG-3's deny shape once" -- it is a byproduct of the first real `check`, not separate work. **BUG-2 NOT FIXED, unchanged from N4.** `| grep -q` under `set -o pipefail` is INHERITED and present identically in `dc-mirror.sh:278,281` and `dc-cache-proxy.sh:234,252,255`. A three-script sweep is out of scope under hard rule 1 and under the standing directive. Its direction is fail-closed (a false MISS, never a false OK), which is why it can wait. --- ## Item 5 -- OPERATOR APPROVAL RECORDED: the agent's rack permissions **What.** No artifact change. Recorded because it relaxes the scope of hard rule 3 for one task and must not be inferred later from silence. **Presented to the operator, verbatim as put:** `.claude/settings.local.json` carries `Bash(ssh vr1-dc0-rack *)` in **allow** -- an unrestricted wildcard onto the dc0 rack, auto-added by an approval on 2026-07-30 and flagged in that session's sweep (F1) as "worth review". Its consequence, stated at the time of asking: a background agent can mutate the dc0 rack (including `sudo bash -s -- install dc0`) with NO permission prompt reaching the operator, so hard rule 3's individual gating does not fire for rack commands. The operator was offered a narrowing. **Operator answer, exact utterance: "The agent permissions are approved".** **CONSEQUENCE, scoped deliberately narrowly.** This authorises the snap-proxy install agent to operate on the dc0 rack under the existing allow rule. It is NOT a general standing relaxation of hard rule 3, it does not extend to dc1, and it does not cover the bundle deploy -- the agent's mandate explicitly stops before `add-model` / `deploy` / any `juju model-config`. A future session must not read this as blanket approval for unsupervised cloud mutation. **Not a GA-R5 ruling and deliberately not recorded as one:** it is a harness permission decision, OPS under GA-R3 (no architectural consequence beyond this stage, no Roosevelt delta), so it takes no D-number. The gitignored `settings.local.json` is unchanged by this session. **Revert.** Remove `Bash(ssh vr1-dc0-rack *)` from `allow` in `.claude/settings.local.json` and replace it with a targeted `ask` rule pinned to the invocation shapes actually needed. Note the 2026-07-30 lesson that applies directly here: quoting style and the profile token are BOTH part of the match, so a narrowing rule that pins the wrong quote form silently fails to match and falls through to the classifier. --- ## Gate results at time of writing - `bash tests/dc-snap-proxy/run-tests.sh` -- **55 passed, 0 failed** (52 -> 55: T20b, T23b, T41b added; T41 re-pointed). - `bash scripts/run-tests-all.sh` -- **GAUNTLET: ALL GREEN (94 harnesses)**. HARNESS-MANIFEST NOT re-recorded and deliberately so: no harness was added or removed, only cases within an existing one, so the manifest is unchanged and the R15(2) re-record reason does not apply. - `bash scripts/repo-lint.sh` -- **0 fail, 1 warn** (the expected L1 legacy non-ASCII carve-out in `docs/design-decisions.md`). - Four mutations run across items 2-3; **all four killed**; script restored and sha256-verified byte-identical after each. --- ## Item 6 -- THE dc0 SNAP PROXY IS INSTALLED, RUNNING AND GATE-VERIFIED **What.** No repo artifact changed. A LIVE MUTATION on the dc0 rack, executed by a delegated agent under the operator approval recorded in item 5. Capture: `docs/audit/dc0-snap-proxy-install-20260731.txt` (739 lines, ASCII/LF). **NAMED GATE, GA-R6:** `dc-snap-proxy.sh check dc0` = **PASS, EXIT 0, 16 assertions**. **RE-RUN INDEPENDENTLY** from the main session after the agent reported, rather than taken on its word: squid 6.14 bound to `10.12.8.4:3129` SPECIFICALLY (not wildcard), both units active AND enabled, packaged `squid.service` neither active nor enabled, all five owned artifacts matching their generators BY CONTENT, and all three behavioral probes green (store API with `channel-map` + `sha3-384`, CDN tunnel allowed, non-allowlisted destination DENIED). **The before-state was captured and it FAILED (exit 1) on a real rack** -- so the gate is proven able to fail somewhere other than a fixture. It was NOT uniformly failed: three OK lines (virsh-resolved bridge, rack route table) prove the instrument was on the RIGHT HOST. That distinction is the recorded wrong-host trap for these per-DC scripts, where a jumphost run reports every item MISS. **MAAS's squid was untouched throughout** -- same pid (1871633) before and after, still holding `*:3128` / `*:8000`. Net-layer idempotence with `dc0-mirror-net.service` verified: virbr2 addresses and the default route are byte-identical before and after, which is the D-135 coexistence property this script was built to honour. **A REAL SNAP PAYLOAD WAS FETCHED THROUGH THE PROXY** -- HTTP 206, and the first bytes are `hsqs`, the squashfs magic of the genuine `core` snap. 4 bytes of ~110 MB read and discarded; nothing installed. **This is what moves the tool from FIXTURE-green to end-to-end proven**, and it is the specific claim the prior session was careful to say it could not make. **STILL NOT PROVEN, stated because the deploy depends on it:** no charm LXD container was tested. All nine dc0 nodes are shut off (measured via virsh), so the "containers source from metal-admin" premise underpinning `CLIENT_CIDR` was NOT re-proven and is carried forward from the prior session's measurement. snapd's own consumption is unproven and out of scope (the juju keys are unset, so `node`'s `/etc/environment` assertion FAILS as expected). dc1 untouched. **Revert.** `ssh vr1-dc0-rack 'sudo systemctl disable --now dc0-snap-proxy.service dc0-snap-proxy-net.service'`, then remove the five generated artifacts named in the script header. The packaged `squid` remains installed and disabled; MAAS's squid is unaffected either way. --- ## Item 7 -- BUG-3 RESOLVED BY MEASUREMENT: the deny shape is CONFIRMED, no ## assertion changed **What.** No change. The confirmation BUG-3 was waiting for. The dstdomain deny (`deny !snap_store`) produces a shape BYTE-IDENTICAL to the port deny the assertion was originally written from: **curl exit 56, `CONNECT tunnel failed, response 403`, http_code 000**, with squid logging `TCP_DENIED/403 ... HIER_NONE`. Measured by hand with stderr both merged and unmerged, not inferred from `check`'s own say-line -- which matters, because reading the gate's verdict to confirm the gate's assumption is circular. **MECHANISM, so this is not re-opened later:** curl reports the CONNECT **response status**, and squid renders 403 for ANY `http_access deny` regardless of which ACL fired. The two shapes coincide by construction, not by luck. **CONSEQUENCE:** BUG-3's protocol steps (b) and (c) were correctly NOT performed, and `scripts/dc-snap-proxy.sh` + its harness are sha256-identical to their pre-install state. The header block added in item 4 stays as the record of why the question existed and how it was settled. `check dc0` can now be cited as a gate. --- ## Item 8 -- MY BUG-1 FIX WAS WRONG, AND THE MEASUREMENT REFUTES IT (FINDING 9a) **What.** `tests/dc-snap-proxy/run-tests.sh` T23b gains a prominent in-file annotation. `scripts/dc-snap-proxy.sh` deliberately NOT changed. **Item 3 changed `acl snap_probe src 127.0.0.1/32` to `src ${LISTEN}/32` and stated the source-selection step as REASONED, NOT MEASURED. The measurement says it is wrong.** Measured on the dc0 rack, and re-run independently from the main session: $ ip route get 10.12.8.4 local 10.12.8.4 dev lo src 10.12.8.2 `.4` is a **SECONDARY** on virbr2 (primary `.2`), and **Linux does not auto-select a secondary address as a source**. Squid's own access log attributes every rack-side probe to `10.12.8.2`. So the rule is STILL DEAD -- BUG-1's fix corrected the mechanism DESCRIPTION and landed on the WRONG ADDRESS. **Nothing is broken.** The probes are admitted by `snap_clients` exactly as before, and `check dc0` passes. What remains unconnected is the rule's stated PURPOSE: that narrowing `snap_clients` must not silently break `check`. **NOT FIXED NOW, and the reason is not laziness.** `check` diffs the generated config against the rack's LIVE file, so any edit to `gen_squid_conf` turns the live dc0 gate RED until `install` is re-run. That makes it a LIVE MUTATION on a just-verified surface, out of scope under hard rule 1 and against the standing directive. The correct value is the rack's primary metal-admin leg (dc0 `10.12.8.2`; dc1's `10.12.68.2` is already recorded MEASURED in `lib-hosts.sh`), and it wants a site-keyed row. **Fix the script and T23b TOGETHER at the next install -- dc1 is the natural point, so both DCs land in one pass.** **T23b is annotated rather than deleted or re-pointed**, per the standing rule that a test asserting a finding must be REPLACED with the new invariant and never deleted to go green. Editing its expected string to `10.12.8.2` today would turn the harness red against the live config; leaving it unannotated would let a future session "fix" the test to match a value that is itself wrong. The annotation states the measurement, the correct value, and why the change is deferred. **GENERALISABLE, and it is the reusable half:** a secondary IPv4 alias is never the kernel's chosen source address. Any ACL, firewall rule or allowlist keyed to a service's ALIAS address -- which is exactly what the D-134 utility `.4` band is on both racks -- will not match that host's own outbound traffic. This applies equally to `dc-mirror.sh` and `dc-cache-proxy.sh`, which share the `.4` alias pattern. **Revert.** `git checkout -- tests/dc-snap-proxy/run-tests.sh`. --- ## Item 9 -- FOUR FURTHER FINDINGS FROM THE INSTALL, logged not fixed 1. **The script header's CDN aside is WRONG (finding 8a).** It records that `core`'s download URL "came back on api.snapcraft.io, not a CDN host -- so the CDN entry may be unused." That measured only the FIRST HOP: the URL **302-redirects to `canonical-bos01.cdn.snapcraftcontent.com`**. The `.snapcraftcontent.com` entry is **LOAD-BEARING, not speculative**, and the same run proves squid's dot-prefix wildcard admits a subdomain appearing in NO probe and NO config line. DOCFIX material against the header. 2. **`apt-get install squid` leaves a FAILED but still ENABLED unit (9c)** that would fight for 3128 at the next boot -- the header's predicted `FATAL: Unable to open HTTP Socket ... listening port: 3128` was observed VERBATIM. `install` closes it; an operator who installs the package and stops there does not. 3. **The packaged unit rests `failed`+`disabled`, not `inactive` (9e).** `check` is right to pass, but `systemctl --failed` shows red on a healthy rack, which is quiet triage-poisoning. `systemctl reset-failed squid` is the remedy; NOT applied (a live mutation outside the mandate). 4. **>>> NO JUJU CLIENT EXISTS ON THE JUMPHOST, AND THE dc0 JUJU VM REJECTS THE CONFIGURED KEY (9d). <<<** `ssh vr1-dc0-juju` -> `Permission denied (publickey)`. Not a permission-harness refusal and NOT retried in an altered shape. **This blocks the next phase**, which is `add-model` + spaces + `deploy` under D-138 (cloud-facing clients run from INSIDE the DC). Owed before that phase starts. --- ## Item 10 -- D-139 RULING B's DECIDING REASON IS REFUTED. Ruling FLAGGED, not amended. **What.** `docs/audit/gai-conf-rfc6724-verification-20260801.txt` (NEW) + `docs/CURRENT-STATE.md`. No script, no cloud change, and **no edit to D-139 itself**. **Why it was checked at all.** D-139 ruling B was recorded WITH its own unverified premise named in the entry: "UNVERIFIED and owed before the carve is executed: the deployed jammy image's own `/etc/gai.conf`, which can override the RFC 6724 default table that ruling B's deciding reason rests on." Closing that owed item is what produced this. **The discipline of naming an unverified premise at ruling time is the only reason this was caught before the carve rather than after it.** **MEASURED, in two steps, neither inferred.** 1. jammy's `libc-bin` **2.35-0ubuntu3** ships `/etc/gai.conf` with **ZERO active lines** -- every line commented. So the node image applies NO override and glibc's compiled-in table governs. (noble/vcloud measured identically.) 2. glibc 2.35's `default_precedence[]`, fetched in full from sourceware at tag `glibc-2.35` and read rather than recalled, carries glibc's OWN comment `/* See RFC 3484 for the details. */` and is: `::1/128 -> 50`, `2002::/16 -> 30`, `::/96 -> 20`, `::ffff:0:0/96 -> 10`, `::/0 -> 40`. **There is NO `fc00::/7` entry.** **THE INVERSION.** ULA has no precedence entry, so it falls through to `::/0` = **40** -- identical to GUA. IPv4-mapped is **10**. So on the node image **ULA = GUA = 40, both outranking IPv4 = 10**. Ruling B was declared on "ULA loses to IPv4 (3 vs 35), GUA wins (40 vs 35)" -- the **RFC 6724** table. RFC 6724 obsoletes RFC 3484 and does define those values; **glibc 2.35 does not implement it.** The claim is correct about the RFC and wrong about the software this cloud runs. **The `fc00::/7` entry that DOES exist is in `default_labels[]` (label 6)**, beside `fec0::/10` (5) and `2001::/32` (7), under glibc's comment "The next two entries differ from RFC 3484". Labels drive SOURCE-selection rules 5/6 -- a different mechanism from destination precedence, and not what the ruling cited. Recorded so the two are not later conflated into "glibc does handle ULA specially, so the argument stands". **This is the citation class again, third instance.** A citation is an EXISTENCE claim; only its CONTENT is evidence -- and here the citation (RFC 6724) was real, current and correctly quoted, yet still wrong for this deployment because **nobody had checked whether the implementation follows the RFC**. That is a NEW variant worth naming: it is not enough to open the source and check its status and dates; for a standards citation you must also check that the software actually implements it. **WHAT SURVIVES, and why the ruling is not simply withdrawn.** Three of ruling B's four reasons are untouched: conformance with Willamette (a REAL site) and VR0 DC0, both full GUA; MINIMIZE DELTA TO ROOSEVELT, whose `2602:f3e2:103::/48` is GUA with no ULA and no plane carve yet; and isolation being unaffected either way because containment lives at the FORWARDING layer. Those are the governing-constraint arguments and are independent of the refuted one. **FLAGGED, NOT AMENDED, and this is deliberate.** GA-R5 makes a ruling the operator's. They answered a question whose stated DECIDING reason no longer holds, so the honest action is to put the refutation back to them -- not to silently keep the ruling because I still like the answer, and not to silently reverse it. D-139's text is UNCHANGED; CURRENT-STATE carries the refutation and the flag. **Nothing of D-139 is executed, so nothing is half-built in either direction and the reconsideration costs nothing but the exchange.** **Revert.** `git rm docs/audit/gai-conf-rfc6724-verification-20260801.txt` and drop the CURRENT-STATE paragraph. Note that reverting does not un-refute the measurement.