| 2026-08-07 |
NetBox-pending: queue this session's new IPAM objects for office1-netbox (operator-flagged)
...
office1-netbox (10.10.1.10) is the live VR1 IPAM apex (DOCFIX-195); this
session's new objects are not yet in it. Enumerated in changelog Item 11
(dc0 .7 + dc1 .6 region VM + new vr1-dc1-region + dc1 .7-when-live +
D-134 utility .4-.9 assignments + dc0 renames) and flagged as a
do-not-miss NetBox item in the next-session NEXT (CURRENT-STATE + ledger)
so it isn't lost before the end-of-deployment write-back.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
vr1-dc1-region profile registered + verified; session close consolidated
...
Registered the vr1-dc1-region profile on voffice1 (dc0-pattern tunnel
-L 5243:10.12.68.6:5240 via the rack; apikey piped .6->`maas login -`
stdin, never exposed). Verified: rack vr1-dc1-maas-01 (qtw8pm), 0 machines
(empty -- the 9 nodes+juju+.7 get rebuilt in). changelog Item 10 + a
handoff block enumerating the remaining config/rebuild chain.
Consolidated this session's ledger block to a bounded GA-R4 summary
(dc0 .7 carved-and-ready + D-134 naming amendment + vr1-dc1-region LIVE);
ledger 296/300, repo-lint 0 fail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
vr1-dc1-region MAAS is LIVE -- region init done on the .6 (operator-authorised one-shot)
...
Located the shared vr1-office1-svc key ON VCLOUD (~/vr1-office1-creds/
office1_svc_ed25519, fingerprint matches the injected key -- operator's
"what was used on DC0"); reached the .6 from vcloud (key stays local,
ProxyCommand via voffice1->rack). Snap egress works via the snapd proxy
10.12.68.2:8000. Installed maas 3.7.2 + postgresql 16.14. Operator
authorised ("You run it"); ran the file-staged credential one-shot (creds
generated + stored 0600 on the .6, NEVER in my context): DB role+db,
maas init region+rack, createadmin. http://10.12.68.6:5240/MAAS/ -> 301.
OWED: consolidate the .6 creds to ~/vr1-dc1-creds/ (SEC-020/D-137).
NEXT: register vr1-dc1-region profile + config via tested tools + rebuild
9 nodes+juju+.7 fresh into it. changelog Item 9 + CURRENT-STATE + ledger.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
dc1 .6 region VM DONE to Deployed jammy (carved, both legs live) -- ready for MAAS install one-shot
...
Extended aux-carve for -maas-01 (9fea7c1); carved the .6 in Office1
(--profile admin, all 3 racks; metal-admin 10.12.68.6 + provider-public
10.12.64.6, pass=6/0); MAAS-deployed jammy -> Deployed, power on, both
legs ping 0% from the rack (transient power/virsh flakes cleared on
re-query).
NEXT is the MAAS region install/init on the .6 -- OPERATOR ONE-SHOT
(createadmin=SEC-020 + reaching the .6 needs the operator's
vr1-office1-svc key). Then register vr1-dc1-region + config via tested
tools + rebuild 9 nodes+juju+.7 fresh into it. changelog Item 8 +
CURRENT-STATE + ledger NEXT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
dc1-region workstream START: .6 region VM bootstrapped to Ready + MAAS 3.7 procedure researched
...
Operator ruled dc1 node-handling "Rebuild fresh into dc1-region". Measured:
dc1's full set is in the Office1 region (9 nodes+juju Ready; .6/.7 Failed
commissioning on unset-power). Bootstrapped the .6 region VM (Office1-side,
like dc0 hot-kid): power set, renamed normal-piglet->vr1-dc1-maas-01
(convention), recommissioned -> Ready (a transient virsh-login error
cleared on retry).
Researched + recorded the MAAS 3.7 region+rack install/init (changelog
Item 7; matches Office1 3.7.2 + PostgreSQL 16; external DB required;
createadmin=SEC-020 operator one-shot). Remaining next-session: extend
aux-carve for -maas-01 + carve .6 (10.12.68.6 + 10.12.64.6) before deploy
(hot-kid under-carve lesson); deploy jammy; MAAS init one-shot; register
region + config via tested tools; rebuild 9 nodes+juju+.7 fresh into it.
CURRENT-STATE dc1 block + ledger NEXT updated. repo-lint 0 fail.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
D-134 amendment: MAAS hostname naming convention (standing) + dc1 rebuild-fresh ruling
...
Operator: "Record that as the preferred naming convention going forward."
Recorded the VR1 MAAS hostname convention (set vr1-<dc>-<role>-NN ==
libvirt domain == power_id after commission/deploy; random-by-default is
not leave-it-random; standup DoD = 0 non-vr1-<dc>-* names) as a D-134
AMENDMENT (per-DC identity standard; ARCH, no new number) + lib-hosts
comment.
dc1 node-handling RULED "Rebuild fresh into dc1-region" (build region,
then power/enlist/commission/deploy the 9 nodes+juju FRESH into it, not
delete+re-enlist) -> CURRENT-STATE dc1 block + ledger NEXT.
repo-lint 0 fail; gauntlet ALL GREEN (101).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
dc0 MAAS hostname convention: rename tailscale + juju controller (operator correction)
...
Operator: "You are not following naming conventions." Renamed the two
random-named vr1-dc0-region machines to convention (hostname == libvirt
domain == power_id, lib-hosts:26): known-marten->vr1-dc0-tailscale-01,
subtle-grouse->vr1-dc0-juju-01. All 11 dc0-region machines now vr1-dc0-*.
Cosmetic to tooling (carve/power resolve by boot MAC); load-bearing for
operability. changelog Item 6 + CURRENT-STATE note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Session bookend 2026-08-07 (dc0 tailscale .7 carved-and-ready): GA-R4 + sweep
...
Bounded ledger summary + rotation (08-04 block -> archive, 296/300); sweep
docs/audit/queued-findings-20260807-dc0-tailscale-provisioning.txt (F1-F4
first-surface); CURRENT-STATE + changelog ping-verification edits (both .7
legs live). Deliverable d36d815..c8ddfb6 already pushed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
dc0 tailscale .7: carve applied+verified + deployed jammy (carved-and-ready)
...
LIVE (gated): carved the .7 router's two legs (metal-admin 10.12.8.7 +
provider-public 10.12.4.7 VLAN 5002, no br-ex) via the new aux-carve;
check pass=8/0. MAAS-deployed jammy -> Deployed, sshd live on 10.12.8.7.
State: dc0 .7 = carved-and-ready. Two JOIN prerequisites remain, both
off-session: (a) tagged pre-auth key + Headscale autoApprovers/ACL
(operator key is PLAIN; join NOT attempted); (b) SSH access via
vr1-office1-svc (region injects only that key; operator holds it).
CURRENT-STATE tailscale block + changelog Items 3-5 updated; dc1 ruling
recorded (build vr1-dc1-region first, no migration).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
dc0 tailscale .7: MAAS power+commission (Ready) + aux-carve tooling
...
Provision the dc0 Tailscale .7 subnet-router toward phase-03 Step 3.3
(Horizon over the tailnet). LIVE (gated): set power_type=virsh on the
already-enlisted known-marten (n4parh) via the DC-region-reachable
10.12.8.2 address, commissioned -> Ready.
Tooling (hard rule 4 -- no artifact carved this 2-leg shape):
- lib-hosts.sh CARVE_AUX_HOSTS isolates the .7 VM from every HOSTS
consumer; inert octet(7)/boot-MAC entries; NODE_CARVE_TAILSCALE_PLANES.
- dc-node-carve.sh is_tailscale_host/is_two_plane_host -> metal-admin +
provider-public RAW, no OVS br-ex (juju-controller idiom, octet 7),
keyed on stable lib-hosts name + boot MAC; --host accepts aux, bare
apply still HOSTS-only.
- harness t01/emit_tailscale + T28..T33 (pass + tsbrex/tscarved failing
directions). 58/0; repo-lint 0 fail; gauntlet ALL GREEN (101).
Implements D-129(iii) amdt (2026-08-07) + D-134 octet map; no new D-num.
Operator rulings: dc1 "No migration. Build region on DC1 correctly.";
join key is plain (not tag-scoped) -> tailscale JOIN stays blocked on a
tagged key + Headscale autoApprovers/ACL (N1). Provisioning to
carved-and-ready is unblocked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Session close 2026-08-06/07 (GA-R4 bookend): phase-03 Step 3.4 G3 + per-DC Tailscale rulings & .7 VMs
...
Bounded ledger summary + sweep (queued-findings-20260807, O1-O5) + CURRENT-STATE
tailscale-build status + node-vm harness reconcile (11/66 -> 12/72, red gate that
my substrate commits caused by not re-running the gauntlet -- the 2026-07-30 lesson
repeated). Rotated the 2026-08-03 close to archive (ledger 298 < 300).
Session delivered: Step 3.4 G3 domain-manager probe (built + live PASS, closes the
last phase-03 exec item); Decision C (Horizon reconciled to VR1, Step 3.3 splits to
its own tailnet-gated row); D-129(iii) amendment rulings a-d (dedicated .7 VM / STAR
/ single-HA-pinned / SNAT-on, both DCs) + D-134 octet .7 + D-107 citation DOCFIX;
site-tailscale.sh tooling; the .7 subnet-router VMs applied + MACs pinned on both
DCs (dc0 tailscale; dc1 region + tailscale). Headscale-side join deferred (no
control-plane access); MAAS commission/deploy/carve + dc1 region setup owed.
Gates: repo-lint 0 fail; gauntlet ALL GREEN (101). Memory: +ipv6-primary-posture
(drift-prevention, operator-directed) + instrument-currency #19.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
substrate dc1: apply full (region + tailscale VMs) + pin MACs
...
Gated full dc1 apply: vr1-dc1-maas-01 (region, authored 07-30 unapplied) +
vr1-dc1-tailscale-01 created (4 added, 0 change/0 destroy), both powered off.
MACs for both captured (virsh domiflist) and pinned pre-enlistment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
substrate dc0: apply tailscale VM (2 added) + pin its MACs pre-enlistment
...
Gated apply on voffice1: vr1-dc0-tailscale-01 created (domain+disk, 0 change/
0 destroy), powered off. MACs captured via virsh domiflist and pinned in
vr1_dc0_node_nics order before enlistment (MAC-regen trap). Post-pin tofu plan
must show No changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
substrate: add the .7 Tailscale subnet-router VM to both DC node maps
...
opentofu/vr1-dc{0,1}-substrate/main.tf gain "vr1-dc{0,1}-tailscale-01"
(2 vCPU / 2 GiB / 25 GiB, macs=[]), the dedicated per-DC Tailscale subnet
router at utility .7 (D-129(iii) amendment 2026-08-07). for_each keyed ->
expect 1 add / 0 change / 0 destroy per DC.
Capacity re-gated FIT (dc-dc-whole-host-budget.py at overhead 34/14 ->
874/1024 = 85% RAM, 150 GiB headroom). tofu validate PASS (both substrate
roots + all modules). repo-lint 0 fail.
NOT YET APPLIED -- the plan/apply run on voffice1 (inner substrate root,
D-128) and each is individually gated (verify 1 add/0/0, then pin MACs
pre-enlistment, MAAS commission/deploy, carve .7 metal-admin+provider-public).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
D-129(iii) amendment: per-DC Tailscale operator-access rulings (a-d), both DCs
...
Record four GA-R5 rulings (2026-08-07) that pull the gap-21 per-DC Tailscale
subnet-router forward to close phase-03 Horizon properly (operator: "pull the
tailscale steps forward"; "plan and push to both DC0 and DC1"):
(a) dedicated VM at utility .7 (10.12.8.7 / 10.12.68.7)
(b) STAR -- operator->DC only (the Headscale ACL / security boundary)
(c) SINGLE router, HA scale-up PINNED
(d) SNAT ON now, source-IP preservation PINNED
These are D-129(iii) implementation sub-decisions (not a new D-number). Also:
correct the D-107 citation defect (D-107 is airgap/mirror/NTP, governs no
Tailscale; D-129(iii) governs); extend D-134's standing octet map to .7;
update the gap-21 register row and CURRENT-STATE (phase-03 does NOT close this
session -- Step 3.3 Horizon splits to its own gate row, gated on the tailnet
build + vault CA on the workstation + a browser login over the tailnet).
Records only -- no code, no cloud change. The build (site-tailscale.sh + the
.7 VM per DC + Headscale star ACL/autoApprovers + SEC row) is next.
repo-lint 0 fail; Decision C reconciliation measured read-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
| 2026-08-06 |
phase-03 Step 3.4 (dc0): G3 domain-manager probe -- named check + live PASS
...
Build scripts/g3-domain-manager-probe.sh + tests/g3-domain-manager-probe (harness
12/12, every exit path proven) as the GA-R6 named executable check for phase-03
Step 3.4 stage 2 (gate G3), filling the hard-rule-4 gap (was a manual runbook walk
only) and giving dc1's Step 7 a reusable probe. Grounded in the real policy
(domain-manager-policy.yaml:103 create_grant managed-role guard).
Ran it LIVE from the dc0 rack (operator-approved): G3 PASS, 7 ok / 0 fail, teardown
verified clean (zero g3-* residue). Stage-1 (PO:) verified read-only: policyd-override
attached, all 3 keystone units 'PO: Unit is ready'. Capture
docs/audit/g3-dc0-probe-20260806.txt.
HARNESS-MANIFEST recorded 99->100; gauntlet ALL GREEN (100), repo-lint 0 fail.
CURRENT-STATE: Step 3.4 -> RESOLVED; phase-03 exit gate now turns on the Horizon
reachable/login-works item (D-044/D-075 per-rebuild + VR0-nginx vs VR1-tailnet access
model = Decision C), to be measured + ruled next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Session close 2026-08-06 (part 3, GA-R4 bookend): phase-03 core-verify F-CV1/F-CV2/F-CV3 RESOLVED + binding conformance
...
GA-R4 bookend: bounded ledger summary + sweep. F-CV2 (openstack CLI on the dc0
rack), F-CV1 (designate binding, BUNDLEFIX-056), F-CV3 (dashboard TLS via D-072
AMENDMENT VR1 / BUNDLEFIX-057) all RESOLVED; BUNDLEFIX-058 designate-stack
conformance -> binding conformance clean cloud-wide. D-134 Roosevelt-delta +
gap-21 access-model context recorded.
Sweep O11 FIRST SURFACE (dc0 MAAS query method); O10 rack repo-stage bundle STALE
(re-stage before redeploy). Memory instrument-currency #18. repo-lint 0-fail,
gauntlet ALL GREEN (99) at ace0e16, ledger 292 lines (no rotation).
NEXT: Step 3.4 (domain-manager policy) -- last phase-03 exit-gate item -> Steps 8-12
-> Stage-5 exit. Body: docs/changelog-20260806-phase03-coreverify.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
BUNDLEFIX-058: designate-stack binding conformance (amqp/cluster -> metal-internal) + ceph-rbd-mirror Stage-6 watch
...
Operator-requested binding-conformance sweep after F-CV1/F-CV3. The only remaining
ACTIVE svc-endpoint deviations were the designate stack: designate amqp (->rabbitmq)
+ cluster (peer), designate-bind cluster (peer) -- all on the metal-admin default,
deviating from the generic svc-to-svc rule (14 apps use metal-internal). No ruled
exception -> conformance repair, no new D-number.
Benign (designate served metal-admin so nothing broke, unlike F-CV1/F-CV3) but
off-plane. Repaired to metal-internal so dc1 inherits a fully-conformant bundle.
Proven live (same method): juju bind designate amqp=metal-internal cluster=metal-internal
+ juju bind designate-bind cluster=metal-internal (individually gated). Verified no
regression: bindings moved; designate still Stage-7-blocked on nameservers ONLY;
designate-bind active/idle; designate haproxy 0 DOWN (F-CV1 intact). No harness asserts
these bindings. gauntlet ALL GREEN, repo-lint 0-fail.
RECORDED: ceph-rbd-mirror certificates/cluster on metal-admin are UNBOUND today
(Stage-6 DR not wired) -> sweep O9 STAGE-6 WATCH + CURRENT-STATE, pre-check before
wiring at dc-dc-phase5. No other active deviations found cloud-wide.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
BUNDLEFIX-057 / D-072 AMENDMENT (VR1): dashboard cluster -> metal-internal (F-CV3 RESOLVED)
...
VR1 split-metal INVERTS the D-072 cluster placement. The openstack-dashboard charm
declares no admin/internal extra-binding and no os-*-network (metadata + charmhub
docs), so apache serves its SSL vhost on metal-internal while haproxy dialed
cluster=metal-admin -> vhost-less -> plaintext (the D-072 trap, inverted). Option A
(serve metal-admin) unavailable in-deployment (no charm lever).
Retire the VR0 exception: openstack-dashboard cluster -> metal-internal (generic rule
+ the served plane). Proven LIVE before ratifying (operator process directive), then
ratified GA-R5 "Ratified, land the config-of-record".
bundle.yaml cluster=metal-internal; design-decisions D-072 AMENDMENT (VR1);
binding-reference matrix + exception RETIRED + cross-ref; CURRENT-STATE/sweep/changelog
F-CV3 RESOLVED. Verified live: provider + operator metal-admin VIPs both TLS 200
CA-verified (were plaintext); cert covers all 3 VIP IPs (resolves AH01909); units
active/idle. No harness asserts this binding. gauntlet ALL GREEN, repo-lint 0-fail.
dc1 inherits via the shared bundle (no rebind).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Gap-21: capture the operator-access model (tailnet -> metal-admin dashboards) as forward context
...
Operator discussion 2026-08-06: operators reach each DC's metal-admin plane over
the tailnet and access routed dashboards (Horizon) from there. Added as dated
CONTEXT (not a ruling) to workflow gap-register item 21, informing the four
deferred Tailscale sub-decisions when re-raised. Captures the implications that
reach outside the Tailscale build: the operator-facing reverse proxy becomes
unnecessary; D-044/D-075 per-rebuild accommodations sunset for the operator path;
cert SANs on the metal-admin VIP (and FQDN certs, D-106/D-008) become load-bearing;
and it reinforces metal-admin as the dashboard's serving plane (the access-model
argument for F-CV3 option A over option B). Cross-referenced from the F-CV3 sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
D-134: Roosevelt-delta annotation -- LXD container addresses are auto-picked, not carved
...
Logs the operator-requested Roosevelt-delta observation from the phase-03
core-verify thread. D-134 pins node statics (per-role octet bands) and the VIP
band (.50-.99 reserved); the OpenStack control plane runs in Juju LXD containers
whose addresses are NOT pinned -- MEASURED 2026-08-06 as MAAS device interfaces in
mode=static but auto-picked from the .100-.200 pool independently per subnet, so
container octets float across planes and are not redeploy-stable. Harmless in VR1
(clients use VIPs; certs reissue to cover whatever a unit holds). Roosevelt-delta:
a bare-metal build wanting predictable container addressing would add a
per-container static reservation (a "container carve"). GA-R3: observation, no new
D-number -- annotated onto D-134.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
BUNDLEFIX-056: designate public/internal bindings -- F-CV1 RESOLVED (fix executed + verified)
...
designate's REST API public+internal endpoints were omitted from the bundle,
defaulting to the '' metal-admin fallback -> orphaned the provider + metal-internal
legs of its ruled .62 VIP triple (D-020 amendment) -> _admin haproxy backend SSL-DOWN
on the unserved metal-internal address. The deployed charm declares public/admin/
internal extra-bindings (metadata verified); dnsaas (D-106) is ADDITIONAL, not a
replacement -- the prior "no public binding" reading was the defect's root.
Config-of-record: bundle.yaml +public:provider-public +internal:metal-internal;
provider-bundle-check EXPECT_PUBLIC_VIP 11->12 (vault stays out, not 13) with T16c/T16d
failing-direction tests (57->59, ALL PASS); network-space-binding-reference row 88 +
note. Gauntlet ALL GREEN (99); repo-lint 0-fail.
Live (operator-approved): juju bind designate public=provider-public
internal=metal-internal (rc=0, dc0 rack). Verified: full haproxy sweep 0 DOWN
cloud-wide; apache https vhosts span all 3 planes; cert reissued for provider-public;
catalog triple correct (public 10.12.4.62 / internal 10.12.12.62 / admin 10.12.8.62).
Governing: D-052 / D-020 amendment. Evidence:
docs/audit/stage5-dc0-phase03-coreverify-20260806.txt. Body:
docs/changelog-20260806-phase03-coreverify.md (Item 6).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0 F-CV1: research resolves :public -- fix BOTH endpoints (D-020 triple + charm supports it)
...
Operator asked to investigate designate's tenant surface + charm docs before
ruling on :public. Conclusion: fix BOTH :public and :internal.
Evidence: (1) D-020 AMENDMENT (2026-07-27, operator GA-R5) gave designate the
established provider/admin/internal triple; that ruling implies the bindings, so
the current metal-admin-fallback bundle is a conformance defect. (2) The DEPLOYED
charm-designate metadata declares extra-bindings public/admin/internal (read in
full on the unit -- authoritative; a WebFetch on master wrongly said none, a
small-model error). Charm description: "Multi-tenant ... REST API" -> tenant-facing
by design. (3) Every sibling binds public->provider-public + internal->metal-internal;
designate is the lone deviation, no ruled exception.
Feasibility confirmed: all 3 units hold provider-public + metal-internal addresses
(juju bind not refused); cert covers metal-internal, will reissue for provider-public.
Proposed gated fix (D-072/BUNDLEFIX-011 precedent): bundle bindings
+public:provider-public +internal:metal-internal, live juju bind, verify by
haproxy readback + cert SAN re-read + haproxy sweep. Awaiting operator go-ahead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0 F-CV1: reconcile after research -- :internal defect CONFIRMED, :public OPEN; D-141->D-052
...
Prior-art + governing-decision research on the designate bind-plane mismatch.
Corrects the earlier commit (aaeee93) which reached "CONFIRMED" and cited D-141
before checking the governing decisions (backwards ordering, memory #17).
D-072/BUNDLEFIX-011 prior art: the dashboard-plaintext class was root-caused in
VR0 (charm renders haproxy 443 backend on cluster-binding addr, apache SSL vhosts
only for default+public). dc0 bundle already carries that fix -> F-CV3 is a NEW
cause, parked for its own triage.
F-CV1 re-scoped: governing surface is D-052 + the generic binding rule, NOT D-141.
`:internal`->metal-internal is a CONFIRMED defect (on the metal-admin fallback;
every sibling binds it to metal-internal; no ruled exception; cert already covers
the metal-internal SANs; all 3 units have metal-internal addrs -> juju bind won't
be refused). `:public`->provider-public is OPEN -- designate uniquely carries
`:dnsaas` on provider-public (D-106 dual-VIP); its REST API may be intentionally
metal-admin-only. The per-app table is generated from bundle.yaml (descriptive of
the defect), not intent.
Fix method = D-072 precedent (bundle + live juju bind + haproxy-readback verify).
designate is Stage-7-blocked -> no urgency. Awaiting operator ruling on :public.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0 Step 7: triage F-CV1 (CONFIRMED designate bind-plane mismatch) + F-CV3; sweep
...
Operator-authorized triage of the two plaintext-vs-TLS findings (read-only; not
fixed, hard rule 1). Both apps have vault certs rendered + the certificates
relation, so NOT the ovn CN-issuance class -- charm apache-TLS-frontend layer.
F-CV1 CONFIRMED (two findings, not one): designate/0 apache https frontend binds
only 10.12.8.198:8991 (metal-admin); haproxy's _admin backend dials
10.12.12.110:8991 (metal-internal) where no SSL vhost exists -> check-ssl hits
plaintext -> DOWN. VR1 dual-metal-plane bind mismatch (D-141), structural (not
Stage-7 collateral). F-CV3 (dashboard) is separate: :433 served by Ubuntu
default-ssl.conf, charm https frontend not effective; root cause not nailed.
Owned instrument caveat: earlier "no SSLEngine in sites-enabled" was a grep -r
false negative (does not follow the symlinks); apache2ctl -S corrected it.
Remediation = a focused, gated session. Sweep:
docs/audit/queued-findings-20260806-phase03-coreverify.txt
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0 Step 7: phase-03 core-API VERIFIED; F-CV2 openstack-client on rack; F-CV1/F-CV3 logged
...
Core-API layer of phase-03 core-verify (adapted for vr1-dc0, run from the dc0
rack per D-138) VERIFIED read-only: settle walk (falsifiable prediction matched),
haproxy 0-DOWN across 12 activated VIP apps, admin-openrc scoped token, IP-only
endpoints, two-sourced keystone VIP, vault CA TLS OK.
F-CV2 RESOLVED (07-30 queued-F1, hit at Step 7): installed python3-openstackclient
6.6.0-0ubuntu2 on the dc0 rack; CURRENT-STATE section 7 row amended (GA-R1/C1).
Exit gate NOT MET (2 open, GA-R6/E3 no conditional close): F-CV3 dashboard VIP
serves plaintext (apache-SSL-inactive despite certs) -> Horizon exit-gate fails;
Step 3.4 domain-manager policy NOT RUN. F-CV1 designate-api plaintext vs haproxy
check-ssl (same shape); "collateral of block" reading retracted. Logged not fixed.
Evidence: docs/audit/stage5-dc0-phase03-coreverify-20260806.txt
Body: docs/changelog-20260806-phase03-coreverify.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Session close 2026-08-06 (part 2, GA-R4 bookend): F4 + dc-ha-scaleup retired + memcached 1->3 live
...
GA-R4 bookend for the F4/retire/memcached session:
- Bounded ledger summary (SESSION CLOSE 2026-08-06 (part 2)); ledger rotated (08-02 x2 ->
docs/archive/session-ledger-rotated-20260806.md), 299 -> 280 lines (under the 300 cap).
- Sweep docs/audit/queued-findings-20260806-postwave-retire-memcached.txt -- 5 FIRST SURFACE,
led by F-1 (operator-flagged): designate coordination points at ONE memcached unit,
Stage-7 activation re-check. Always-sweep-5 incl. the as-executed-log gap; 4 OWNED items.
- Memory instance #17 (instrument-currency-before-negatives) persisted outside the repo.
Gauntlet ALL GREEN (99); repo-lint 0 fail (1 legacy L1 warn); ledger-scan reconciled
(4 open decisions, SEC 29 unchanged; DOCFIX 213 / BUNDLEFIX 056 next-free as assigned).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0: memcached scaled to 3 LIVE (operator-approved) + consumer verify
...
Live scale-up executed on the dc0 rack (operator "Both approved"):
juju add-unit memcached -n 2 --to lxd:1,lxd:2 -m vr1-dc0 (exit 0).
- CONVERGED 3/3 active/idle, one per control node (memcached/0 control-01, /1 control-02,
/2 control-03; all "Unit is ready and clustered", 11211/tcp). Clean install, no F5 apt-wedge.
Capture docs/audit/stage5-dc0-memcached-scaleup-20260806.txt.
- Consumer verify (D-121 verify-at-deploy): nova-cloud-controller sees all 3 servers
(memcache_servers = .115,.166,.165:11211). designate coordination backend_url shows ONE
(memcached/0) -- designate is workload-blocked pre-Stage-7, so re-check at designate
activation; recorded, not a scale-up defect.
- bundle.yaml re-staged to the dc0 rack (42845edb == repo HEAD).
- CURRENT-STATE reconciled: memcached config=3 AND live=3.
Record-only commit (the live mutation itself was the add-unit above). repo-lint 0 fail.
Revert of the LIVE state (if ever needed): juju remove-unit memcached/1 memcached/2 -m vr1-dc0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0: memcached 1->3 config-of-record [BUNDLEFIX-055 + DOCFIX-212]
...
Operator ruling 2026-08-06, exact utterance "3 units (restore intent)" -- restoring the
2026-07-31 memcached->3 direction that BUNDLEFIX-053 never folded and the retired
dc-ha-scaleup.yaml had carried unrealized. Found while reconciling a stale comment:
- Full merge-diff (measured) of the archived overlay vs bundle.yaml showed the ONLY divergence
was memcached (overlay num_units:3 / base 1) -- BUNDLEFIX-053 folded the HA chain but not this.
Corrects two earlier misstatements (this session): "memcached=3 in bundle.yaml" (it was 1) and
the retirement's "wholly redundant / all-keys no-op" claim (it diverged on memcached).
- BUNDLEFIX-055: bundle.yaml memcached num_units 1->3, to:[lxd:0]->[lxd:0,1,2] + the
3-independent-caches rationale (no hacluster/VIP; clients hash across the full server list).
This completes the fold, making the archived overlay genuinely, fully redundant.
provider-bundle-check 57/0 (memcached is orthogonal to the arity/VIP gates).
- DOCFIX-212: design-decisions D-121 "Left single (NOT scaled)" list -- memcached removed,
recorded as =3.
- CURRENT-STATE: records the CONFIG=3 / LIVE=1 divergence -- a live add-unit (operator wants it
deployed live) or a teardown+redeploy reconciles it. The live scale-up is a separate gated step.
Gauntlet ALL GREEN (99 harnesses); repo-lint 0 fail (1 pre-existing legacy L1 warn).
Owed: re-stage the functionally-changed bundle.yaml to the dc0 rack (gated); live add-unit to 3.
Revert: bundle.yaml memcached back to num_units:1 + to:[lxd:0]; revert the DOCFIX-212 D-121 line
and the CURRENT-STATE gap note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
Stage 5 dc0: RETIRE + archive dc-ha-scaleup.yaml (operator-ruled; R6 superseded) [DOCFIX-211]
...
Operator ruling 2026-08-06, exact utterance "Retire the redundancy and archive." The
overlay was wholly redundant -- BUNDLEFIX-053 folded its entire content into bundle.yaml,
so applying it was an all-keys no-op deep-merge (Task-#1 post-wave review, measured).
- Archived: git mv overlays/dc-ha-scaleup.yaml -> docs/archive/dc-ha-scaleup-RETIRED-20260806.yaml
+ a do-not-deploy banner. docs/archive/ chosen explicitly (no prior overlay-archive precedent).
- design-decisions.md: GA-R5 SUPERSESSION note on D-121's R6 (original 2026-07-27 record retained
as history); fixed :503 and :4563 "overlay encodes v-a" refs -> bundle.yaml/BUNDLEFIX-053.
- Harness (tests/provider-bundle-check): T17 REMOVED (== T18 once vault-hacluster is in base);
T17b/T32/T33/T34 RE-POINTED onto good.yaml (base HA chain). T33/T34 now use the deterministic
mutate() helper, not sed-on-overlay (no silent no-op fixture; a wrong key path KeyErrors). Each
FAIL case proven to fire against the real checker message. 58->57 cases; provider-bundle-check 57/0.
- Runbook dc-dc-phase4 Step-4 note + Step-12.3(a) gate: the two-phase "base at cluster_count:1 then
apply the overlay to reach 3" model RETIRED -- base deploys 3-unit HA directly, so cluster_count:3
is the expected post-Step-4 value.
- Rendered vips overlays: edited the render SOURCE (render/values/vr1-dc{0,1}-vips.yaml, D-136) and
re-rendered (not hand-edited); overlay diff = the two comment lines only; render-drift 4/0.
Hand-maintained vr1-dc{0,1}-machines.yaml deploy-command comments + provider-bundle-check.py +
cloud-assert.sh comments + dc-dc-deployment-workflow.md item 22 reconciled.
- CURRENT-STATE: no edit needed -- active status already omits the HA overlay from the deploy input;
remaining mentions are dated historical narrative (retained as history, GA-R1).
- Also finalizes Item 3/5 changelog notes (F9 approved+staged; Task #1 recommendation ruled).
Gauntlet ALL GREEN (99 harnesses); repo-lint 0 fail (1 pre-existing legacy L1 warn).
Staging: dc0 rack vr1-dc0-vips.yaml now stale-by-one-comment vs the re-render (functionally
identical); re-syncs + sha-verifies at the next deploy per D-138 (logged, no rack write for a comment).
Revert: git mv the overlay back (strip banner) + git revert this commit + re-render vips from
reverted values.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fg98z7QyzwYUs8fsWCn728
|
