# Session ledger -- in-flight work continuity record

**Purpose.** Long ops sessions on this cloud routinely exceed a single context window
and get COMPACTED (sometimes several times). Anything living only in the chat scrollback
is lost at compaction. This ledger is the durable, committed record of what is IN FLIGHT,
so any session -- after a compaction, or a fresh one -- can resume without losing pending work.

**How to use it (standing practice).**
1. At session start: read this ledger AND run `bash scripts/ledger-scan.sh`. Reconcile.
2. `ledger-scan.sh` is the DRIFT CHECK -- it derives the open-work it reliably can
   (PROPOSED/OPEN decisions, OPEN security rows, next-free numbers) straight from the repo.
   This narrative must not claim CLOSED anything the scan shows OPEN, nor omit what it surfaces.
3. Update this ledger at every deliverable/commit -- it is a standing deliverable like the
   changelog. The changelog says what CHANGED; this ledger says what is still OPEN.
4. The machine-derived block below is seeded from a scan; re-run the scan and re-seed rather
   than editing it by hand.

**SINGLE STREAM (collapsed 2026-07-13).** This ledger previously carried three parallel,
separately-owned stream sections (`main-chat`, `jumphost`, `shared`) plus ~30 append-only
session narratives. Those streams are CLOSED and reconciled into the one list below. There is
now ONE stream. Do not re-introduce per-stream sections.

**Where the history went.** The 2,189-line session-by-session narrative is NOT lost -- it is in
git history (the parent of the collapse commit) and, in durable form, in the 65 `docs/changelog-*.md`
files, `docs/design-decisions.md`, and the incident reports. This ledger deliberately carries only
what is still OPEN, plus the facts that would otherwise be lost because they live nowhere else.

---

<!-- SECTION: machine-derived | OWNER: ledger-scan (regenerate; never hand-edit) -->
## Machine-derived (re-seed from `scripts/ledger-scan.sh`; do not hand-edit)

_As of the 2026-07-13 scan (the collapse). Re-run the scan to refresh._

- **PROPOSED / OPEN decisions:** D-068 (Vault substrate hardening, Roosevelt -- 1.16 ruled OUT
  per amendment; the off-EOL-1.8.8 path remains OPEN), D-071 (routine update cadence + Juju
  controller patch policy -- AMENDED, still PROPOSED, awaiting operator ratification; gated on
  the pre-DC-DC controller HA/backup session).
- **OPEN security rows:** SEC-001, SEC-003, SEC-004, SEC-005, SEC-006, SEC-007.
  (The prior machine block listed only SEC-001/003/004 -- it was STALE by three rows.)
- **Next-free numbers:** D = 114, DOCFIX = 195, BUNDLEFIX = 013.
  (The prior machine block said D=111 / DOCFIX=162 / BUNDLEFIX=012 -- STALE.)
- **Standing numbering rule:** never write an identifier-shaped token (D-/DOCFIX-/BUNDLEFIX-NNN)
  ABOVE the real high-water mark anywhere in `docs/` or `runbooks/` prose -- `ledger-scan` reads
  prose and a decoy token inflates the next-free counter. This has bitten twice.

<!-- END: machine-derived -->

---

## Live state (measured 2026-07-13; nothing is mid-flight)

- **VR1 / Office1 edge `office1-opnsense` is UP and healthy.** LAN 10.10.0.1/24, WAN 172.30.1.2
  (gw 172.30.1.1). Routing + automatic NAT, egress 0.0% loss, 8 LAN pass rules in pf, serial
  console + getty alive, and `kea-dhcp4` serving DHCP on udp/67 (subnet 10.10.0.0/24, pool
  .100-.199). **DHCP is API-MANAGED (D-113(a2)).**
- **OpenTofu repo/state are IN SYNC** -- `tofu plan` reports "No changes". tofu v1.12.3; the
  tree validates (10/10 modules); `opentofu/.terraform.lock.hcl` is committed.
- **VR0 / DC0 testcloud:** fleet fully current (juju 3.6.25, 91 agents); D-011 CLOSED except
  item 6 (manual unseal, blocked on SEC-003). devteam is a live tenant running their own
  cluster. No beta cluster, no foil tenant, no orphan trustees (see State facts).
- Branch `dc-dc-stage1-phase0-first-exec`, pushed. VR1 Stage 1 COMPLETE; Stage 2 partially done.

---

## OPEN WORK -- VR1 DC-DC (the current mission)

The authoritative per-stage tracker is `docs/dc-dc-deployment-workflow.md` (stage table + tooling
gap register). Read it FIRST. This list is only what that doc does not already carry.

1. **Stage 2 is the critical path and is NOT finished: the three Office1 headend service VMs**
   (MAAS-region, NetBox, GitBucket) do not exist. Blocked on two undesigned things, both repo-side:
   (a) no base image source chosen; (b) no real `user_data`/`meta_data`/`network_config` written
   for the `cloudinit-vm` module. The Stage 2 runbook offers an explicit fork -- Option A (OpenTofu
   `cloudinit-vm`) vs Option B (manual `virt-install`, logged as debt). **UNRULED: the operator has
   not picked.** Everything downstream (Stage 3+) waits on this.
2. **NetBox sandbox loop is undocumented.** Operator-clarified architecture: draft-source
   `netbox.baldurkeep.com` (READ ONLY) -> stand up a NEW NetBox on the Office1 VM (sandbox) ->
   import draft + planned changes -> simulate VR1 there -> feed validated refinements BACK to
   the real NetBox. The sim NEVER `--commit`s upstream. `docs/dc-dc-netbox-buildout-scope.md`
   and the Stage 2 runbook still need a note capturing this loop. PENDING.
3. **`pynetbox` / `python3-venv` are not installed** on the vcloud host -- needed for the NetBox
   import against the Office1 sandbox; needs operator sudo.
4. **The apparmor rule is applied on THIS host but is NOT in the repo.** `/var/lib/libvirt/vr1/** rwk,`
   in `/etc/apparmor.d/local/abstractions/libvirt-qemu` GATES EVERY VR1 VM BOOT (custom pool path is
   outside libvirt's abstraction; without it a domain defines but qemu fails Permission-denied).
   It is a one-time operator sudo that exists only as host state -- a rebuild or a second host hits
   the identical foundational failure. **Fold it into `scripts/prereqs/`.** (Was logged as a PREREQ
   candidate and never actioned.)
5. **DC2 planes are DEFERRED (operator ruling, Option B):** do not invent CIDRs -- wait for NetBox
   to assign the D-101 supernet/ULA/GUA. DC2 mesh legs and `dc2_storage` are already wired.
   Values pending final import: ORG_ULA_48 `fd50:840e:74e2::/48`, GUA dc1 `f02::/48` / dc2 `f03::/48`,
   DC2_V4_SUPERNET `10.13.0.0/19`.
6. **G5 NetBox site rename** (`vr1-dc0`/`vr1-dc1` -> `vr1-dc1`/`vr1-dc2`) needs its own D-number.
   Also open in NetBox: G2 (aggregates empty -- top blocks are container prefixes), G3 (ULA /48
   not assigned).
7. **DOCFIX-185 follow-up (operator to rule):** a D-100/D-107 amendment note so the Stage-3 DC
   edges are built to the per-site-ISP transport model (each site simulates its OWN ISP; dark
   fiber is East-West/replication ONLY, never an internet path).
8. `docs/dc-dc-deployment-workflow.md`'s status table was corrected 2026-07-13, but re-read it
   against reality before trusting any row -- it has drifted twice.
9. **Doc-accuracy finding (measured 2026-07-13, logged not actioned):** `docs/security-ledger.md`
   SEC-007 says "D-112(c) makes SSH the only management path". MEASURED on the live edge: the web
   GUI IS listening on the LAN side (10.10.0.1:443 HTTPS 200; :80 301-redirects to it), and is
   correctly firewalled OFF on the WAN side (172.30.1.2:443/:80 closed). This is OPNsense-default
   and not an exposure -- the LAN is the office network, and D-113 kept OPNsense precisely BECAUSE
   the GUI is an operator requirement. But "the only management path" is imprecise: SSH is the only
   path used by AUTOMATION. Reword SEC-007/D-112(c) so nobody concludes the GUI is disabled.
   Reaching the GUI from a workstation requires an SSH tunnel via vcloud (the LAN is a host-local
   libvirt bridge, `virbr2`, not routable off-host): `ssh -L 8443:10.10.0.1:443 <user>@10.17.11.248`
   -> `https://localhost:8443`.

## OPEN WORK -- VR0 / DC0 (the running testcloud)

1. **d011 batch-3 live-validation queue** -- `scripts/checks/d011-04-octavia-lb.sh` and
   `d011-05-magnum-e2e.sh` are DRAFTED and mock-tested; the live PASS was recorded in the
   2026-07-06 batch-3 window, but these remain the checks to re-confirm on any rebuild:
   d011-04 `amphora_headroom()` field parsing (safe default HOLDs on a parse miss -- confirm the
   OK path live); the OCCM LB-name-contains-service-name assumption; d011-05 needs a FOIL tenant
   (a 2nd tenant) for its P3 isolation case -- foil1 was offboarded, so onboard one or set
   `VR_FOIL_APPCRED`. Sequence: non-disruptive half first, then `--include-disruptive`.
2. **Script backlog item 3 (remainder):** fold `scripts/vault-kv-health.sh` into `cloud-assert.sh`
   as a section (cloud-assert has its own A1 vault check but does NOT absorb the KV health script
   -- VERIFIED still open), then use the combined gate to regenerate the stale restart-procedure
   runbook.
3. **Script backlog item 5:** `scripts/tenant-cluster-create.sh` -- generalize the stage-6 +
   watch + D-066-evidence wrapper. NOT STARTED (verified absent).
   (Items 6/7/8 -- `keystone-policy-drift.sh`, `cloud-snapshot.sh`, `tests/tenant-acceptance/` --
   are DELIVERED. The old main-chat section called them "not started"; that was FALSE. Reconciled.)
4. **S-class script quality (`docs/script-quality-findings-20260707.md`):** S4/S7/S8a/S8b DONE;
   S6 no-action; S8c ruled WON'T-DO. **S1/S2/S3/S5 (lib-validate adoption on tenant-onboard/
   offboard) are HELD** by the operator until the DOCFIX-126/127 live-verify lands.
5. **DOCFIX-126/127 live-verify is still OPEN** -- the offboard-E0 and onboard-cred-file fixes have
   never been exercised end-to-end (foil1's offboard predated them). The next REAL onboard/offboard
   confirms. Related known gap: onboard cred-write stages 1/2 have no unit coverage.
6. **`test1` orphan user -- approved for deletion, staged, NOT run.** Role-less shell in the
   devteam domain, 0 roles, no resources. Exact gated command (RE-VERIFY role-less at execution):
   `openstack user delete 9a62f88e331f41adb5c9e41225ec8698`.
7. **The handoff doc is stale:** `docs/handoff-20260703-open-items.md` still lists R-3 (D-063) as
   PROPOSED/OPEN. D-063 is `RESOLVED / CLOSED` in `design-decisions.md` (verified). Correct the
   handoff doc.
8. **Upstream reports queued (operator):** magnum `can-upgrade-to` anomaly -- Charmhub magnum
   2024.1/stable latest (r101) is s390x-ONLY, no valid amd64 target, so magnum was EXCLUDED from
   the update window; report to OpenStack Charmers. (The dashboard-TLS upstream bug was WITHDRAWN
   -- operator judged it site misconfig; the draft is retained.)
9. **Barbican `host_href`:** the charm renders a literal `https://None:9312`, suppressing the
   upstream request-derived fallback. Benign (catalog unaffected, exercised fine live). A
   charm-level fix would need its own D-NNN. Logged, not actioned.
10. **v1-close checklist is QUEUED** (handoff section 6) -- gated on D-011 item 6, which is gated
    on the SEC-003 unseal rehearsal.

## Operator-gated / PINNED (do NOT re-ask; these are rulings, not open questions)

- **D-071** ratification -- held for the dedicated pre-DC-DC Juju controller HA/backup planning
  session. Backups now exist (a 902MB controller backup was taken; `juju create-backup` /
  `download-backup` DO exist on juju 3.6 -- the original "removed in 3.0" premise was wrong).
  Controller HA (single-controller, no-HA) is the unresolved half.
- **D-068** off-EOL Vault path (Roosevelt-scale). 1.16 is RULED OUT; vault stays 1.8/stable rev 714.
- **SEC-001 / SEC-004 / SEC-005 / SEC-007:** rotate / flip-to-private / revoke at v1 close.
- **SEC-003:** custodian assignment + second-person unseal rehearsal -- DEFERRED by the operator.
  This keeps `d011-06-vault-unseal` MANUAL and gates the FULL close of D-011.
- **SEC-006** (burned NetBox API token) -- **DEFERRED by operator ruling 2026-07-13 to deployment
  completion.** Until then the token is LIVE and EXPOSED.
- **D-076 is NOT this repo's to work.** D-076..D-099 are a RESERVED source-project band, never
  assigned here (D-110). The dashboard-policy-override workstream and its uncommitted draft live
  in the SOURCE project, not this repo. Operator also pinned it: no D-076 work while devteam are
  running their own tests.
- **quota-vs-capacity: CLOSED** -- operator ACCEPTS as-documented (no quota trim).

---

## Standing lessons and traps (load-bearing; these live nowhere else)

1. **THE EDGE TRAP -- DHCP on Office1 is API-MANAGED.** A full rendered `config.xml` push WOULD
   CLOBBER IT. The old scp/install/reboot path is DELETED (template, renderer, ISO builder all
   gone -- `opentofu/templates/README.md` is a tombstone). Configure the edge via the REST API
   (`scripts/opnsense-api.sh`); mint keys with `scripts/opnsense-bootstrap-apikey.sh`.
2. **"updated in-place" DOES NOT mean "no restart".** A `tofu apply` whose plan says
   `libvirt_domain.vm will be updated in-place` **RESTARTED the guest** (measured: uptime 8:36 ->
   6 secs; ~30s with no routing and no DHCP). "In-place" means the RESOURCE is not replaced -- it
   says NOTHING about the guest staying up. The dmacvicar/libvirt provider redefines the domain to
   apply a disk-list change and BOUNCES it. **Assume ANY apply touching a `libvirt_domain`'s
   devices (DC edges, node VMs) is an OUTAGE. Schedule and gate it as one.**
3. **Do not re-implement vendor internals; call the interface.** OPNsense stores API secrets as
   `crypt(secret,'$6$')` (SHA-512, EMPTY salt). We COULD mint keys offline and seed them -- we
   deliberately DO NOT: it would couple provisioning to a vendor hash format we must keep
   byte-compatible forever, and a mismatch FAILS SILENTLY (keys that never authenticate).
4. **A config ISO can NEVER be read on an OPNsense nano image** (D-112). Any instruction to build
   one is a trap; the dc-dc-phase2 runbook directed people into it for days.
5. **Guide changes must sweep the tenant AI skill in the SAME change.** Skill-lag was caught twice
   (DOCFIX-135/136) and is now ENFORCED by repo-lint L8 (guide<->skill coupling; the guide's sha256
   is pinned, and any guide edit FAILS lint until re-recorded).
6. **The controller model is `admin/controller`,** not `<user>/controller`.
7. **`juju status --format=line` omits workload messages on 3.6** -- this caused a cloud-assert
   false negative (fixed by DOCFIX-087). Don't rely on it.
8. **repo-lint's guards are load-bearing; reword rather than weaken.** L3 (docs must not reference
   missing scripts) correctly went RED on the config.xml tombstones -- the tombstones were reworded,
   the guard kept.

## State facts to remember

- **The beta cluster and the foil1 tenant are GONE** (both confirmed absent by the 2026-07-08
  read-only audit). The long-standing "beta cluster left at node_count=2" note was STALE and is
  retired here. `capi-test-1` and the `lbtest` LB leftover are also gone; orphan sweep reads
  ORPHAN=0, zero unattached floating IPs.
- **devteam is a LIVE tenant** running a self-created cluster (built via Horizon, calico,
  keypair `devteam-key`). Their template has `volume_driver` unset but the cluster DOES get a
  default StorageClass from the cinder-CSI magnum default -- persistent storage works out of the
  box; no template change needed.
- The vcloud host was NOT pristine: 9 orphaned `vr0-*` libvirt nets were torn down; **`wan` was
  KEPT** (dual-stack NAT-out-enp1s0, GUA `2602:f3e2:fe:10::/64`) as the working model for the
  per-site ISP-uplink network.
- MTU has TWO domains: the ISP uplink `enp1s0` is 1500; the host-internal plane/dark-fiber fabric
  is **jumbo, underlay_mtu=9000** (operator ruling; verified on all 10 nets).
- The repo is temporarily **PUBLIC** (SEC-004) -- flip private at v1 close.
- Secrets live jumphost-local and are NEVER committed: `~/vr1-office1-creds/` (edge SSH key,
  root password, API key), `~/.vr1-netbox.env` (NetBox token), `~/tenant-devteam/`, `~/vault-init/`.

## Project-completion (execute after D-011 passes)

- Consolidate the 10 per-phase do-documents into `docs/v1-deploy-runbook.md`.
- Set repo visibility PRIVATE (SEC-004); revoke/rotate the SEC-005/006/007 credentials.
- v2-deferred: SSH on GitBucket (port 29418), IPv6 dual-stack, NetBox import bundle.