Changelog 2026-07-31 -- the prefer-ipv6 / v6 VIP charm research

Session changelog (GA-R2/D1: ONE per session). Branch dc-dc-stage5-preconditions. Status claims live ONLY in docs/CURRENT-STATE.md.

Context: bundle deploy attempt 1 failed 2026-07-31 with unknown option "prefer-ipv6" on barbican, and the operator ruled "Research what those 6 charms do with v6 VIPs first (Recommended)". Stage 5 is blocked on that research. This session executes it, the operator RULES on the result, and the ruling is then BUILT and verified.

Scope correction to this header, made when it stopped being true. It first read "Read-only throughout: no cloud mutation, no overlay edit, no ruling adopted", which was accurate for items 1-5. After the ruling landed the session edited the renderer, the gate, both overlays and both values files, and refreshed the deploy input staged on the dc0 rack. No cloud state changed at any point -- the vr1-dc0 model is still empty and no juju deploy was run.

Item 1 -- docs/audit/stage5-prefer-ipv6-charm-research-20260731.txt (NEW)

What. The capture answering the ruled question, plus two findings that came out of the same measurement.

Method, and why it is not the Charmhub config-yaml API. The attempt-1 root cause was measured against config-yaml, which answers only is the knob declared. It cannot answer what the charm does. This research downloads the charm each app actually deploys -- juju download <charm> --channel <pinned> --base ubuntu@22.04 --arch amd64 from voffice1 -- and reads the templates and code inside it. Revisions are recorded in the capture because 2024.1/stable moves. Two declaring charms (keystone 857, cinder 820) were downloaded as CONTROLS: without one, "binds v4 only" and "binds v6 by another mechanism" cannot be told apart.

The answer. prefer-ipv6 is not what makes HAProxy bind :::port, and never was, for any of the thirteen. In both template families the v6 frontend bind is gated on ipv6_enabled = not is_ipv6_disabled() -- a read of the kernel net.ipv6.conf.all.disable_ipv6 sysctl. Measured 0 on a live MAAS-deployed jammy node in this DC. The v6 VIP is assigned by pacemaker on per-address family detection, also independent of the option. Vault ships no haproxy template at all and its listener is the hardcoded literal [::]:8200.

What the option really does: it is a unit address-family switch -- get_relation_ip() returns early with the unit's IPv6 address for EVERY relation, ignoring the network-space binding; plus bind_host ::, ip6-localhost stats, and keystone's sync_db_with_multi_ipv6_addresses. The value that sounds like a bind address, haproxy_host, is consumed by NO template in any charm downloaded.

Consequence recorded, not acted on: provider-bundle-check invariant 9's mechanism claim is factually wrong; when a ruling lands the invariant must be REPLACED with a charm-schema-aware one and proven able to fail in both directions, never deleted to go green.

Revert. git rm docs/audit/stage5-prefer-ipv6-charm-research-20260731.txt. The file is a read-only capture; nothing depends on it but the CURRENT-STATE entry in item 2.

Item 2 -- docs/CURRENT-STATE.md section 1: the research result

What. The research outcome appended under the 2026-07-31 ruling block that records the question -- the answer, its evidence, the invariant-9 consequence, the new question about the seven declaring charms, the option-name sweep result, and the runbook defect. Required in the same commit by GA-R1 rule 8 (C1); repo-lint L10 enforces it.

Revert. git revert this commit, or delete the appended block; the preceding ruling text is unchanged and still reads correctly on its own.

Item 3 -- MEASUREMENT ONLY (no artifact): full option-name sweep of the dc0 deploy input

What. Attempt 1 proved juju deploy --dry-run does not validate config option NAMES, and juju aborts the whole bundle atomically on the first bad one -- so a second unknown option elsewhere costs another full attempt. Every option assignment in attempt 1's exact input (bundle.yaml + vr1-dc0-vips + vr1-dc0-machines + vr1-dc0-octavia-pki) was compared against each charm's own config.yaml downloaded at its pinned channel.

Result: 56 applications, 81 option assignments, 20 charm schemas, 0 unresolvable, and the only findings are prefer-ipv6 on the same six. No further option-name failure is waiting for attempt 2. An unreadable or unrecognised schema REFUSES rather than passing.

Not built as a gate, deliberately. Hard rule 1 plus the standing operator directive: hardening is out of scope unless it blocks the deploy, and this one-shot measurement already de-risks attempt 2. The method is reproduced verbatim in the capture so it can be re-run or promoted later. CURRENT-STATE already records that no gate in this repo reads a charm config schema.

Both owed extensions were then RUN, after the ruling landed -- option VALUE TYPES (every downloaded config.yaml carries a type: field the first pass discarded) and the dc1 input. dc0 and dc1 each: 75 assignments / 20 schemas / 0 unknown name / 0 type mismatch / 0 note / 0 unresolvable, EXIT 0. dc1 introduces no new charm@channel pair, checked rather than assumed.

PROOF OF TEETH. A clean sweep across two inputs is exactly the implausibly-uniform result this repo has been burned by, so three defects were planted in a THROWAWAY copy: a restored prefer-ipv6 on barbican ([FAIL name]), a quoted "true" on keystone's boolean ([FAIL type]), and an unquoted float where a string is declared ([note]). All three caught, EXIT 1. The EXIT-0 readings are a measurement, not a silent no-op.

Type judgement, stated because it decides what the sweep is worth. A mismatch is reported only where juju's own coercion cannot save it (a boolean given a non-boolean, an int/float given a non-number). A scalar where a STRING is declared is a NOTE -- juju coerces it -- but it is still surfaced, because an unquoted value is how a version string silently becomes a float. An UNRECOGNISED declared type REFUSES rather than passing.

Revert. Nothing to revert -- no repo artifact was created by this item.

Item 5 -- MEASUREMENT ONLY: what option (b) would actually cost

What. Before putting the fork to the operator, the implementation cost of (b) was measured rather than assumed. prefer-ipv6: true is INJECTED UNCONDITIONALLY by scripts/render-dc-overlays.py:243-246 for every app whenever fam == "dual", and family: dual is a SINGLE TOP-LEVEL key at render/values/vr1-dc{0,1}-vips.yaml:3 -- not per-app data.

Consequence. There is no value to edit. (b) means teaching the renderer which charms declare the option, re-pointing tests/render-drift, and replacing provider-bundle-check invariant 9 -- a code change with a harness, not a one-line data fix. The injection's own inline comment repeats the mechanism claim this session's research refutes.

Revert. Nothing to revert.

Item 4 -- LOGGED NOT FIXED: the dc0 deploy block omits a non-no-op overlay

What. runbooks/dc-dc-phase4-juju-bundle-per-dc.md:553-557 gives the dc0 deploy without overlays/vr1-dc0-machines.yaml, while the dc1 block three lines below includes its machines overlay. Measured, the overlay's ONLY delta to the merged input is ovn-chassis.options.bridge-interface-mappings = 'br-ex:52:54:00:8c:2a:8c br-ex:52:54:00:50:48:88' -- the two dc0 compute provider MACs. Deploying dc0 exactly as the runbook reads would leave ovn-chassis with no provider bridge mapping, which surfaces later as tenant networks with no external path, not as a deploy error. Attempt 1 as executed correctly included the overlay.

Why not fixed here. Hard rule 1 -- this session's step is the research. It is DOCFIX material and is recorded in CURRENT-STATE and in the capture so it cannot be lost.

Revert. Nothing to revert.

Item 6 -- THE RULING, and the three-part build that implements it

Ruling. Recorded under GA-R5 in docs/design-decisions.md as a D-101 RULING NOTE dated 2026-07-31 (OPS under GA-R3; D-101's matrix UNAMENDED), and in docs/CURRENT-STATE.md. Committed and pushed BEFORE any dependent work. Operator utterance, verbatim: "I want it to use IPv6, if there is spam mechanisms being applied then that is bad. Even if it doesn't cause an issue now, it might in the future. A clean IPv6 network is better than one with unusable and possibly future breaking configurations." -- and, because that stated a principle rather than selecting an option, a confirming exchange: "Yes -- keep the v6 legs, remove only the option".

(i) scripts/provider-bundle-check.py -- PREFER_IPV6_CHARMS (NEW) + invariant 9 REPLACED. The constant is the measured authority, keyed by CHARM name (an application name is the deployer's choice; only a charm has a schema) and carrying the revisions and a re-measure-if-a-pin-moves warning. Invariant 9's old text -- "prefer-ipv6 makes HAProxy bind :::port in ADDITION to *:port, so the two must travel together" -- is refuted by this session's measurement and is REPLACED, never deleted: 9a the option is only legal on a charm that declares it (asserted on PRESENCE, since prefer-ipv6: false is the same fatal unknown option to juju as true); 9b on a declaring charm it still travels with the v6 legs, keeping the L3-9 protection intact for the seven; 9c arity.

(ii) scripts/render-dc-overlays.py + both overlays re-rendered. The renderer emits the option only for a declaring charm, reading the set from provider-bundle-check with ast rather than restating it (the read-don't-restate rule APP_OCTET already uses). render() keeps its purity property -- the set is a PARAMETER, and the default REFUSES rather than defaulting to empty, because an empty set renders a plausible-looking artifact with the option nowhere. The values-file headers were updated too, since they carried the refuted mechanism claim. Measured delta: exactly six lines removed per DC and nothing else -- every vip string byte-identical, so all v6 legs are retained.

(iii) tests/provider-bundle-check/ 44 -> 48. T43 reproduces the attempt-1 defect; T44 proves presence-not-truthiness; T45 is the positive control (without it, 9a could be satisfied by rejecting everything); T46 asserts the DIAGNOSIS. T21's comment was re-pointed to its real rationale -- the assertion is unchanged and still correct, for a different reason than the one originally written.

Mutation pass -- six mutations, every one killed tests. 9a deleted (T43+T44 die); 9a keyed on value (T44 dies); 9b restored to all charms (T19+T45 die, 10 failures); the charm list widened with barbican+vault (T43-T45 die); renderer emitting for every dual app, and prefer6_charms() returning empty instead of refusing (both kill render-drift). Source restored byte-identical after each.

ONE NEW ASSERTION WAS DECORATION AND WAS REPLACED. T46's first form mutated only a vip; a non-declaring charm with no option has prefer6 == dual == False and reaches neither branch, so it could not fail under ANY mutation of the charm list. Re-written to assert that an input matching BOTH rules is diagnosed by 9a rather than 9b -- a misleading diagnosis would send the next session to add v6 legs when the fix is to remove the option -- and re-proven against the fold-9a-into-9b mutation. Third time in a week that "prove each new assertion can FAIL" has caught a real hole.

Verification. Gauntlet ALL GREEN (93); repo-lint 0 fail; the fixed dc0 deploy input now sweeps CLEAN on option names (75 assignments, 20 schemas, 0 unknown, 0 unresolvable).

Item 7 -- A DEFECT IN ITEM 6, found by a repo-wide grep and corrected the same session

What was wrong. Two faults with one cause: item 6 measured and gated only the THIRTEEN VIP charms.

1. PREFER_IPV6_CHARMS was WRONG. Re-measured across ALL 33 charms in bundle.yaml: twelve declare the option, not seven -- the seven VIP ones plus ceph-mon 491, ceph-osd 953 (squid/stable), nova-compute 894, hacluster 166 (2.4/stable) and mysql-innodb-cluster 164 (8.0/stable).
2. Invariant 9a was written INSIDE the VIP loop, so an application with no vip never reached it.

Why it mattered, and why no test would have caught it. The repo ALREADY has the missed case: overlays/dc-dc-ipv6-family-matrix.yaml sets prefer-ipv6 on ceph-mon, which carries no VIP. That overlay is a LATER deploy step (runbooks/dc-dc-phase4-juju-bundle-per-dc.md:728/737), not part of the swept attempt-2 input -- so the miss would have passed attempt 2 cleanly and surfaced at the step after, looking like a new fault. ceph-mon happens to declare the option, so nothing would have broken today; the gate was simply blind there.

Fixed. 9a now runs over EVERY application, before the VIP loop, and the VIP loop skips an app 9a already flagged so the diagnosis is not doubled. ceph-mon's in-file claim ("CONFIRMED real option, charm-ceph-mon config.yaml") is now INDEPENDENTLY VERIFIED from the artifact rather than taken from the comment. Harness 48 -> 50/50: T47 (a non-VIP app whose charm lacks the option FAILS) and T48 (ceph-mon PASSES), both mutation-proven -- re-scoping 9a back to the VIP loop kills T47; dropping ceph-mon from the list kills T48. Gauntlet ALL GREEN (93); repo-lint 0 fail.

The generalisable lesson, and it is one this repo keeps paying for: I measured the population the QUESTION named (the 13 VIP charms) rather than the population the INVARIANT covers (every application juju validates). The grep that found it took two minutes and was not prompted by any gate.

Revert. Folded into item 6's revert.

Revert (item 6). git revert this commit. It restores the previous renderer behaviour, the previous invariant 9 and the previous overlays together -- they must move as one, since a re-rendered overlay without the matching renderer fails tests/render-drift.

What was NOT done, stated so it is not assumed

• R2 is unamended and D-101's matrix is unchanged. The dual-stack posture does not move; every v6 VIP leg is retained.
• Nothing changed for the SEVEN charms that DO declare the option -- that is a separate, newly-raised question and was deliberately not bundled into the ruling (GA-R5: one decision per exchange).
• No cloud state changed. The vr1-dc0 model is still empty; no juju deploy was run this session.
• No charm-schema GATE was built. The one-shot sweep is a measurement; a gate is logged as owed.

Item 8 -- pre-deploy sequence: rack input refreshed, preflight run, dry-run green

Rack deploy input refreshed and hash-verified. ~/repo-stage on the D-138 client host is a COPY, not a git clone, so nothing updates it automatically and it was still carrying the pre-ruling vips overlay. Exactly one of four files differed; it was copied and all four then compared against repo HEAD. The gitignored vr1-dc0-octavia-pki.yaml was deliberately NOT touched -- a wholesale directory refresh would have risked clobbering or re-permissioning it (SEC-029) -- and it verifies byte-identical to voffice1's at 5fc117f1, still 0600.

Preflight captured: docs/audit/stage5-preflight-dc0-20260731.txt (242 lines, exit 1). Run with MAAS_PROFILE=vr1-dc0-region, because preflight is REGION-BLIND and without it emits 19 false negatives. The instrument was proven current first -- maas-profile-assert.sh vr1-dc0-region hot-kid exits 0. P1-P4 and P7 PASS; P5 FAIL with 11 findings.

The P5 delta is enumerated, not waved through. The 2026-07-30 acceptance covered SIX findings and says it covers "these six, enumerated, and nothing else". Five NEW ones -- all vr1-dc1 S2 rows from SEC-027/-028 -- are the D-137 forward register correctly reporting that dc1's region has not been built. The diff is appended to the capture and put to the operator as its own GA-R5 exchange.

Step 4.2 --dry-run against the fixed input: exit 0, 56 apps / 108 relations / 33 unit placements. Its green is not evidence on option names -- that is attempt 1's lesson -- but it confirms the re-rendered overlay resolves and plans. The ceph-osd tags=openstack exposure is now MEASURED rather than reasoned: the plan places ceph-osd/0..3 on machines 5-8 by explicit id, so the absent tag never has to match and the initial deploy is unaffected. The residual (a later UNPLACED juju add-unit) is unchanged and still logged.

Revert. Nothing to revert in the repo beyond the capture. The rack-side file copy is reversed by copying the previous overlay back; its pre-change hash was 3ae79e82.

Item 9 -- SECOND RULING: the five new P5 findings accepted

Question as presented. Preflight for dc0 is RED on P5 only. It reports 11 findings; the 2026-07-30 acceptance covered SIX, enumerated, and says it covers "these six, enumerated, and nothing else". All six are still present. Five are NEW, all vr1-dc1, all opened after that ruling -- maas-region-db-password, maas-region-admin-password, maas-region-api-key.txt (SEC-027) and maas-juju-api-key.txt, maas-juju-user-password (SEC-028) -- and every one is the D-137 forward register correctly reporting that dc1's MAAS region is authored but not applied. Accept and proceed, or stop and remediate?

Operator answer, exact utterance: "Accept the five and proceed to the dc0 deploy".

Consequence. The five are carried as known, by-design absences on their existing SEC-027 / SEC-028 rows. They describe dc1, which is not built; nothing about them affects the dc0 deploy. preflight.sh keeps exiting FAIL on P5 for the rest of this stage, that RED is ruled-accepted, and it must NOT be made green by deleting or weakening a matrix row. Like its predecessor this covers these FIVE, enumerated, and nothing else. Total accepted at P5 is now ELEVEN across the two rulings.

Revert. A ruling is a record, not a change; there is nothing to revert. Withdrawing it would require a new GA-R5 exchange.

Item 10 -- BUNDLE DEPLOY ATTEMPT 2: the prefer-ipv6 defect is gone; two new ones found

What happened. The deploy was run from the dc0 rack against the fixed, hash-verified input. The prefer-ipv6 failure did not recur. Two new defects did, and for the first time the model is PARTIALLY POPULATED: 23 applications, 0 machines, 0 units -- measured immediately, not assumed. Attempt 1 aborted during VALIDATION and left the model empty; this one got past validation into EXECUTION, which is why it left residue. Nothing is provisioned: no MAAS machine left Ready, no disk was written.

Defect 1 -- the deploy input on the client host is a PARTIAL COPY of the repo. ~/repo-stage has no policies/, and bundle.yaml:214 is the only local-file reference in the whole deploy input (policyd-override: ./policies/overrides.zip), resolved relative to the bundle and uploaded as a charm resource -- so it must exist ON THE HOST THAT DEPLOYS. Nothing caught it: --dry-run does not upload resources (the SECOND --dry-run blind spot in two days), attempt 1 aborted before resource upload, and preflight.sh runs on voffice1 where policies/ DOES exist. Same wrong-host instrument class as the dc-mirror.sh and egress-probe errors, and the second instance this session. Fixed forward under gating: both policy files copied and sha256-verified.

Defect 2 -- a bundle that relies on default-base is deployable exactly once. The re-run failed with barbican: downgrades are not currently supported: deployed revision 265 is newer than requested revision 261. Measured: barbican 2024.1/stable is rev 265 at --base ubuntu@22.04 and 261 at --base ubuntu@24.04 or with no base. The 23 existing apps are all on 22.04, so run 2a honoured bundle.yaml:85 -- but on the re-run, for an application that ALREADY EXISTS, juju resolves without that default.

Two fixes tested READ-ONLY (--dry-run reproduces the error against the populated model, making it a safe harness): a model-level default-base=ubuntu@22.04 has NO EFFECT (set, read back, re-run identical, then RESET -- model config is back to its documented Step-3.5 state); an explicit per-application base: WORKS (added to barbican alone, the error moved to barbican-vault).

OWNED -- I walked into a trap this repo has already recorded twice. The first two attempts at the second test staged a throwaway bundle under /tmp and returned ERROR no charm was found at "./bundle.yaml". That is the juju SNAP's PRIVATE /tmp, documented in CURRENT-STATE and in the phase-4 runbook. Under $HOME it worked immediately.

The rollback decision tree does not reach this case. runbooks/dc-dc-teardown-rollback.md:586 is written for tofu apply and defaults to fix-forward, which is measured not to work here. A partial juju deploy is a failure mode the repo does not cover -- logged as a finding, not fixed here.

Revert. The repo change is the capture and the CURRENT-STATE entry. The rack-side policy copy is reversed with rm -rf ~/repo-stage/policies. The model residue is the subject of the operator recovery decision and is deliberately untouched.

Item 11 -- THIRD RULING: option D -- fix the bundle AND clear the model

Question as presented. Recovery from the partial deploy. Options put: (A) clear the 23 definitions and deploy once cleanly from the unchanged bundle, logging the re-run defect; (B) add an explicit per-app base: to all 56 and continue incrementally.

Operator answer, exact utterance: "I want to do option D: Fix the bundle by defining the base 22.04 on each app and then clearing the modeling and deploying from a clear model. It is very cheap to clear and redeploy."

The operator composed a fourth option, and it is strictly stronger than any offered. (B)'s durable fix lands, AND the deploy runs the path that is MEASURED to work -- a clean deploy onto an empty model, which is exactly what run 2a did before it hit the missing resource file -- rather than juju's incremental re-run path, which is the one carrying the defect. Their rationale settles the cost question I had left open: clearing is cheap because nothing is provisioned.

Classification. OPS under GA-R3 -- a bundle defect fix; doubt resolves DOWN, no D-number. No ruled surface changes: an explicit base: per application states what default-base: ubuntu@22.04/stable at bundle.yaml:85 already means, it does not alter it. The invariant it establishes -- a bundle whose applications rely solely on default-base is deployable exactly once -- is skill material at stage close.

Revert. A ruling is a record; withdrawing it needs a new GA-R5 exchange.

Item 12 -- BUILD option D's first half: explicit base on all 56, gated by invariant 12

bundle.yaml -- all 56 applications now carry base: ubuntu@22.04/stable, plus a note at the default-base line recording WHY the redundancy is deliberate and telling a future reader not to simplify it away.

Proven against the real model, not inferred. The earlier one-app test showed only that the error MOVED to the next app. The fully-based bundle was staged as a throwaway on the rack and --dry-run against the LIVE 23-application model -- the same command that errored on barbican 265-vs-261 -- exits 0.

Subordinates are in scope, measured: mysql-router is rev 1154 at 22.04 and 1178 at 24.04, so exempting subordinates would have left the trap open. hacluster shares rev 166 across both bases, which is not a property to depend on.

Two silent under-matches, both caught by cross-checking the edit against the parsed file rather than by review.

1. A first pass keyed on the block form ^ <app>:$ inserted 44 of 56 -- the twelve -hacluster applications are single-line FLOW MAPPINGS and matched nothing. The inserter asserts its app list equals what yaml.safe_load sees and refuses otherwise, which is what caught it. The final edit also asserts the ARTIFACT: every app carries the base, and the parsed structures with base stripped are IDENTICAL to the original.
2. overlays/dc-ha-scaleup.yaml DEFINES vault-hacluster, an application that exists nowhere in bundle.yaml, and it had no base. That overlay is a LATER deploy step, so this would have re-opened the trap after the deploy succeeded. Found by the new gate on its FIRST run, in a file the fix was not looking at -- the same later-step-overlay class as the ceph-mon prefer-ipv6 miss in item 7. Fixed.

provider-bundle-check invariant 12 (NEW). Every application carries an explicit base equal to the bundle's OWN default-base -- keyed on that value, not a literal, so a future base change moves one line. A bundle with no default-base REFUSES rather than passing vacuously.

Five cases, each mutation-proven individually (tests/provider-bundle-check 50 -> 55/55): T49 stripping every base -- a byte-identical PASS before this landed; T50 a single app; T51 a base disagreeing with default-base; T52 a SUBORDINATE, not exempted; T53 the refusal. Neutering nobase kills T49/T50/T52; neutering wrongbase kills T51; neutering the refuse branch kills T53. Source restored byte-identical after each.

Gauntlet ALL GREEN (93); repo-lint 0 fail.

Revert. git revert this commit -- bundle, overlay, gate and harness must move together, since the gate fails on a bundle without the base lines.

Item 13 -- option D half 2: model cleared, and BUNDLE DEPLOY ATTEMPT 3 SUCCEEDED

Deploy of bundle completed. EXIT 0. Capture docs/audit/stage5-dc0-deploy-attempt3-20260731.txt.

The clear. All 23 application definitions removed INDIVIDUALLY, never batched (hard rule 3), each read back. The precondition was re-verified immediately before the first removal -- 0 units, 0 machines, nothing provisioned -- and the model read back as Model "vr1-dc0" is empty afterwards. Step-3.5 state survived, checked not assumed: apt-mirror still set, all six spaces still bound with both address families.

Measured immediately after the deploy: juju 56 applications / 9 machines / 33 units, machines pending/allocating; MAAS 9 Deploying + 1 Deployed (the controller).

Operational note, recorded because the first removal read as a failure: juju remove-application PROMPTS by default and aborts on non-interactive stdin (ERROR application removal: aborted, exit 1). --no-prompt is required from a non-interactive session.

What is NOT claimed. Deploy of bundle completed. means juju ACCEPTED and QUEUED the bundle. It does not mean the cloud is up. The settle takes hours and nothing here asserts unit health, relation settling or any service verdict.

G17's dc0 half is now genuinely armable -- the existing capture was taken on the CONTROLLER VM and says so. The nine role nodes are booting for the first time.

Revert. Not revertible as a repo change; the deploy is live cloud state. Reversing it means juju remove-application per app or destroying the model, which is an operator decision.

Item 14 -- DEPLOY BLOCKER FOUND, LOGGED NOT FIXED: the dc0 mirror lacks jammy-backports

Measured end to end. 22 of 33 units in error, every one hook failed: "install". Hook output: E: The repository 'http://10.12.8.4/ubuntu jammy-backports Release' does not have a Release file. Per-suite probe against the mirror: jammy / jammy-security / jammy-updates 200, jammy-backports 404 -- exactly D-135's "jammy triple" scope. The node's /etc/apt/sources.list carries deb http://10.12.8.4/ubuntu jammy-backports ..., because the Step-3.5 apt-mirror model-config rewrites the stock Ubuntu sources to point EVERY suite at the DC mirror, including one it was never built to hold. All 9 machines are started/running -- purely the artifact layer, and units retry.

Not fixed: D-135 scope is a ruled surface (hard rule 1). Add the suite and re-sync, or drop backports from the nodes' sources -- operator's call.

dc-mirror.sh check dc0 PASSES while this is true. It verifies sync STATUS, not suite COVERAGE against what the deployed image asks for. Logged as the gap, not fixed.

Revert. Nothing to revert.

Item 15 -- QUEUED BY OPERATOR DIRECTION: per-DC Tailscale as a standing standup requirement

Operator direction, verbatim (2026-07-31): "A closer real work analog would be to install tailscale in DC0, DC1, and any future DCs as they would be running their own tailscale on, or beside, the utility node for that particular DC. Queue up the tailscale installation in DC0 and add it as a requirement for DC1 and any future installations."

QUEUED, NOT EXECUTED -- the current step is the Stage-5 deploy. Home of record: docs/dc-dc-deployment-workflow.md tooling gap register item 21 (new), carrying the requirement, a per-DC definition-of-done, the vendor-documented build constraints, and the four sub-decisions needing GA-R5 rulings before any build: the utility-band OCTET (D-134's map is a standing cross-DC standard, so it cannot be chosen per-DC), the admin-reachability model (star vs mesh -- the real architecture question at region-region scale), HA count per site, and SNAT on/off.

This is EXECUTION of the already-ruled D-129(iii) shape, not a new decision.

Measured while scoping it, all read-only:

• SEC-010 does not need relaxing. Tailscale reaches each router as ordinary WireGuard UDP to the VM's own address; the forwarding to permit is on that VM between tailscale0 and metal-admin -- a different decision from the transit-leg DROP rule.
• voffice1 holds no route to 10.12.x at all (ip route get 10.12.8.1 falls to the default; only the two transit /30s), so the tailnet stops at Office1 by construction. A subnet router can only forward to nets its own host can reach.
• The existing Office1 node is UNTAGGED (AdvertiseTags: None) -- user-owned, with a 180-day key-expiry clock on the operator's only tailnet path. A defect to fix alongside.
• D-129(iii) cites D-107 as governing the Office1 installation; D-107 rules nothing about Tailscale (read in full). Second instance of the F7 miscitation class, this time inside a ruling.

Research basis. Tailscale KB (subnet routers, HA, site-to-site, ACL tags, autoApprovers, key expiry, 4via6, exit nodes) and Headscale's own docs, fetched and quoted rather than recalled. Two findings the build must honour: autoApprovers does not apply retroactively, so the policy must exist BEFORE a router first advertises; and Headscale does not document 4via6 support, which makes globally-unique per-DC management CIDRs a hard constraint at region scale rather than a nicety.

Revert. git revert -- register item 21 and the CURRENT-STATE entry move together.

Item 16 -- OPERATOR DIRECTION: all Tailscale decisions deferred to the end of the step

Operator direction, verbatim (2026-07-31): "To make sure we don't muddle the in process deployment, queue all the tailscale decisions to the end of the step. When you bring them back up give me decisions choices with your recommendation for each."

All four sub-decisions in gap register item 21 (utility-band octet; admin-reachability model star-vs-mesh; HA count per site; SNAT on/off) are DEFERRED to the end of the current step. When re-raised they are to be presented as decision CHOICES, each with an explicit recommendation. Nothing about Tailscale is to be built or ruled before then.

Revert. Nothing to revert.

Item 17 -- MEASUREMENT: the size of jammy-backports if it is added to the dc0 mirror

Measured 2026-07-31 against archive.ubuntu.com from the dc0 rack, summing the Size: field of every entry in each component's binary-amd64/Packages.gz, at the EXACT scope dc-mirror.sh:177-181 uses (--section=main,restricted,universe,multiverse --arch=amd64 --nosource --diff=none):

Context measured alongside: the existing dc0 mirror is 951 GB (/var/lib/dc-mirror/ubuntu), with 1.8 T free on the rack. Backports is therefore ~0.1% of the mirror and a non-issue for capacity.

Sanity checks run, because a clean total is exactly the shape this repo has been burned by: the five largest entries were listed to confirm the Size: field was being read (top entry libreoffice-dev-doc at 190 MB -- plausible); a separate binary-all index was probed and returns 404, so arch:all packages are already inside binary-amd64 (301 of main's 338 entries) and are not double-counted or missed.

Worth knowing before choosing: ~462 MB of the 859 MB in main is LibreOffice (two libreoffice-dev-doc versions at 190 MB + 184 MB, plus -core and -common) -- i.e. roughly 46% of the suite is a desktop package set no OpenStack control-plane node will ever install.

Transfer estimate, stated as the weak measurement it is: a single-stream fetch of one Packages.gz measured 932 KB/s egress from the rack, which puts 1.07 GB at roughly 19 minutes. That is one file on one stream; debmirror parallelises, so treat it as an upper bound rather than a figure.

Revert. Measurement only; nothing changed.