diff --git a/docs/CURRENT-STATE.md b/docs/CURRENT-STATE.md index 0139f13..1b8e0ef 100644 --- a/docs/CURRENT-STATE.md +++ b/docs/CURRENT-STATE.md @@ -1184,6 +1184,18 @@ rack-resident, so the class is real). The lesson generalises D-137's founding argument one level up: absence of a ROW is invisible to the register, so enumerating what EXISTS is a distinct control from auditing what is declared. + **RE-RUN 2026-07-27 (Stage-5 grounding audit), operator-authorized privileged sweep:** + `python3 scripts/creds-matrix.py --tier2 --remote --privileged` -> exit 1, capture + `docs/audit/stage5-creds-privileged-20260727.txt`. This is a RE-CONFIRMATION of the + 2026-07-26 privileged run against the current 82-row matrix, NOT a newly-closed item. + Result: **still exactly 7 findings, the same set** (S2 dc0-edge-api, S5 x3 power-key + asymmetry, S6 conflation, E1 dc0-edge-api on-disk, E4 x2 uncheckable) -- no drift in a + day. All three root-owned locations read successfully via ESCALATION (`sudo -n`, + metadata only): `/root/maas-secrets/*`, `/var/snap/maas/current/root/.ssh/*`, + `/root/netbox-secrets/*`. **ZERO E3 findings** -- no undeclared file at any declared + location. Note the checker's own honesty on the V1 arm, worth preserving: "V1 + provenance: no identity had two digestible copies -- NOTHING was verified here; this is + a skip, not a pass." **QUEUED FINDINGS CAPTURED 2026-07-26 at session close: `docs/audit/queued-findings-20260726.txt`** -- an end-of-session sweep for content that existed ONLY in the session transcript. Part A: the three secrets-storage items NOT diff --git a/docs/audit/stage5-committee-raw-20260727.md b/docs/audit/stage5-committee-raw-20260727.md index b2c9337..9a4d268 100644 --- a/docs/audit/stage5-committee-raw-20260727.md +++ b/docs/audit/stage5-committee-raw-20260727.md @@ -409,3 +409,104 @@ CLAIM: Juju on the headend is 3.6.27, not the 3.6.25 the record carries -- an in-channel patch refresh that D-071 permits but that leaves CURRENT-STATE's pin stale, and it is the version that will bootstrap the controller. EVIDENCE: `ssh voffice1 'snap list'` -> `juju 3.6.27 rev 35621 3/stable`. Against `docs/CURRENT-STATE.md:161-162` ("Juju 3.6.25 on voffice1") and the section-7 pin row ("Last recorded VR0 value 3.6.25 -- historical, unverified here"). D-071 ADOPTED 2026-07-21 rules patch-only controller jumps and in-channel-only refreshes, so 3.6.25 -> 3.6.27 within `3/stable` is POLICY-COMPLIANT; the defect is only that the record was not updated. GA-R1 amendment C2: the measurement wins and corrects the document in the capture's commit. BLOCKS-STAGE-5: no | DISPOSITION: DOCFIX + +--- + +## LENS 4 -- GATE AND CHECKER INTEGRITY (returned sixth) + +Read-only; all fixtures under `/tmp`; no repo file, git ref or cloud object mutated. No tool call was denied or prompted. + +**ID: L4-1** +CLAIM: `scripts/repo-lint.sh` reports `PASS: repo lint (0 fail, 0 warn)` over ZERO files when given a mistyped flag or a mistyped repo root -- the gate whose "0-fail" every GA-R6 stage-close commit cites cannot distinguish a clean repo from an UNEXAMINED one. +EVIDENCE: `scripts/repo_lint.py:134-135` -- only the two KNOWN flags are stripped, so any other `--flag` survives and becomes `argv[0]`, i.e. the ROOT. No `R.is_dir()` check, no floor check. `scripts/repo-lint.sh:14-19` passes any `--*` token through. MEASURED on this repo: `bash scripts/repo-lint.sh` -> `FAIL (1 fail, 1 warn)`; `bash scripts/repo-lint.sh --record-clientdocs-sweeps` (a ONE-character typo of the documented flag at `repo_lint.py:100`) -> `PASS (0 fail, 0 warn)`; `bash scripts/repo-lint.sh /home/jessea123/openstack-caracal-dc-d` (one-character path typo) -> `PASS (0 fail, 0 warn)`. Resolved root for `--bogus-flag` = `/--bogus-flag`, exists=False, 0 files linted. CONTRAST: `creds-matrix.py:169` has exactly the floor this lacks. +BLOCKS-STAGE-5: yes | DISPOSITION: DOCFIX + +**ID: L4-2** +CLAIM: preflight gate P3 (`channel_assert.py`) verified ZERO of 33 charm-channel pins on the real tree TODAY and still returned WARN, because `juju` is not on this host's PATH and a missing BINARY is folded into the same soft verdict as an unreachable charmhub. +EVIDENCE: `scripts/channel_assert.py:41` appends "juju info failed" to `warns` for ANY exception including `FileNotFoundError`; `:53` sets WARN when `fails` is empty. MEASURED: `[WARN] placement: juju info failed ([Errno 2] No such file or directory: 'juju') -- verify 2024.1/stable manually` ... `WARN: channel assert (33 pins, 0 fail, 33 warn)`. NOT latent -- this is the present state of the Caracal version-pin gate. The harness cannot see it: `tests/preflight/run-tests.sh:57-67` (T7) simulates only charmhub-UNREACHABLE and its comment assumes the binary exists. +BLOCKS-STAGE-5: yes | DISPOSITION: DOCFIX +[CORRECTS THE AUDIT SESSION'S OWN CAPTURE: `stage5-live-measurement-20260727.txt` section 3 listed P3's `33 pins, 0 fail, 33 warn` as part of the expected red set and said "nothing new". The COUNT was expected; the CAUSE was not. Zero pins are verified against charmhub, and the reason is a missing binary on the host preflight runs on -- which ties to S-1 (no `openstack` client either).] + +**ID: L4-3** +CLAIM: `provider-bundle-check.py` -- the single deploy gate -- returns PASS on a Stage-5 input whose HA is DECORATIVE: 13 hacluster subordinates at `cluster_count: 1` behind principals at `num_units: 3`. +EVIDENCE: `grep -rn cluster_count scripts/ tests/` -> **no matches** (measured). The placement block DOES fire and affirms the input (`:269-272` satisfied by `to: ["lxd:0","lxd:1","lxd:2"]`), printing `[ok] placement: role-separated ...`. CONSTRUCTED AND MEASURED: a copy of `overlays/dc-ha-scaleup.yaml` with every `options.cluster_count` rewritten 3 -> 1 (13 subs, incl. `vault-hacluster` at `:148`) yields `PASS`, rc=0 -- byte-identical to the correct overlay. The missed failure is the classic one: N principals each paired with a 1-node pacemaker cluster, each claiming the VIP. +BLOCKS-STAGE-5: yes | DISPOSITION: DOCFIX + +**ID: L4-4** +CLAIM: `provider-bundle-check.py --overlay ` treats an EMPTY, comments-only, or MIS-KEYED overlay as a silent no-op, prints PASS, and never names the overlays in its verdict. +EVIDENCE: `:62` converts a `safe_load` of `None` into `{}`; `:82-84` copies only recognised top-level keys so an `application:` typo lands as an inert key; `:285` prints only `path`. MEASURED against the real bundle: 0-byte overlay -> rc=0 PASS; comments-only -> rc=0 PASS; `application:` typo -> rc=0 PASS; `/nonexistent/overlay.yaml` -> rc=2 ERROR (fails closed); top-level list -> rc=2 ERROR. Fails closed on ABSENT/structurally-wrong, fails OPEN on EMPTY/MIS-KEYED -- and the rc=2 path is itself downgraded to WARN by preflight (L4-6). +BLOCKS-STAGE-5: yes | DISPOSITION: DOCFIX + +**ID: L4-5** +CLAIM: `preflight.sh`'s aggregator ignores every sub-gate exit code that is not exactly 1 or 2, so a gate that VANISHES or CRASHES leaves `PREFLIGHT: PASS -- clear to add-model / deploy`. +EVIDENCE: `scripts/preflight.sh:27` -- `note()` only tests rc -eq 1 and rc -eq 2. 127, 126, 130, 139 and every rc>=3 leave WORST untouched. P1/P2/P3 invoked bare (`:30,:33,:36`). MEASURED with the harness's own fixture shape: `repo-lint.sh` DELETED -> rc=0 `PREFLIGHT: PASS`; `pre-flight-checks.sh` present but exiting 127 -> rc=0 `PREFLIGHT: PASS` (this defeats P4's existence guard entirely -- the guard tests `[ -f ]`, not the outcome). +BLOCKS-STAGE-5: no -- LATENT (no gate is deleted today; the real run returns rc=1 correctly). | DISPOSITION: DOCFIX + +**ID: L4-6** +CLAIM: The T9 class the project fixed for P5 is still OPEN in P1, P2 and P3 -- a checker's own deliberate hard-error exit 2 is remapped to WARN, which is precisely how these checkers signal "I could not evaluate anything". +EVIDENCE: `preflight.sh:27` maps rc=2 -> WARN; `provider-bundle-check.py:32,:108,:113` and `channel_assert.py:19` all return 2 BY DESIGN. MEASURED: P2 with unparseable bundle -> WARN; P2 checker DELETED -> WARN; P3 checker DELETED -> WARN; P1 `repo_lint.py` DELETED -> WARN (the T9 case in the FIRST gate). +BLOCKS-STAGE-5: no -- LATENT | DISPOSITION: DOCFIX + +**ID: L4-7** +CLAIM: Gate G17's named executable check CANNOT FAIL as written: `curl -sI` exits 0 on 404/403/500, and the dc0 URL it probes is an nginx `autoindex` root that answers 200 with zero packages behind it. The dc1 half names no command at all. +EVIDENCE: `docs/CURRENT-STATE.md:852`. (a) MEASURED against a local HTTP server: an EMPTY document root returns `HTTP/1.0 200 OK` for `/`; `curl -sI <404 path>` printed `404 File not found` with **curl exit=0**, while `curl -fsI` exited 22. No exit-code predicate exists in the recorded command. (b) The probed root is created EMPTY before any sync: `dc-mirror.sh:330` + generated vhost `:235-240` (`root /var/lib/dc-mirror; autoindex on;`), so `/` answers 200 whether or not `last-sync.status` says OK -- the very file `:299-318` was rewritten to assert on. (c) The dc1 clause is PROSE; the real named check already exists at `dc-cache-proxy.sh:210-217` and G17 does not name it. +BLOCKS-STAGE-5: no -- the row itself states G17 is not a Stage-5 precondition. | DISPOSITION: DOCFIX + +**ID: L4-8** +CLAIM: `run-tests-all.sh` counts only harnesses it DISCOVERS and nothing pins the count, so a harness that is renamed, moved or deleted is neither run nor failed, and the gauntlet still prints ALL GREEN. +EVIDENCE: `scripts/run-tests-all.sh:21-22` (`shopt -s nullglob`) and `:37` -- `RAN` is COMPUTED, never compared to a floor. 81 exists only as PROSE (`docs/CURRENT-STATE.md:46,:393,:1146`). CONSTRUCTED AND MEASURED in a 3-harness fixture: baseline (gamma red) -> `1/3 FAILED` rc=1; `gamma/run-tests.sh` renamed to `.sh.disabled` -> `ALL GREEN (2 harnesses)` rc=0. Real-tree control: 81 harness lines, `ls tests/*/run-tests.sh | wc -l` = 81. Secondary: a harness executing ZERO cases still prints its own "ALL PASS". +BLOCKS-STAGE-5: no | DISPOSITION: DOCFIX + +**ID: L4-9** +CLAIM: repo-lint L10 and L12 are evaded by doing NOTHING: each inspects exactly ONE changeset, so a violating commit is invisible the moment any unrelated file is touched or any further commit lands. +EVIDENCE: `scripts/repo_lint.py:378-391` and `:464-472`. CONSTRUCTED AND MEASURED in a throwaway git repo: commit adds `docs/audit/capture.md` alone, tree clean -> `[FAIL] L10`; append one line to README.md (tree dirty) -> `PASS`; commit that unrelated change instead -> `PASS`. Second evasion: the TRIGGER SET is only three surfaces (`:394-407`), so a status claim elsewhere is out of scope -- including the workflow-doc gate cells DOCFIX-200 found carrying a false Stage-4 exit condition. +BLOCKS-STAGE-5: no | DISPOSITION: DOCFIX + +**ID: L4-10** +CLAIM: `cloud-assert.sh` -- the Stage-5 completion verifier -- asserts a unit COUNT exactly once (mysql). It catches a SICK unit and cannot catch a MISSING one; A3 reports "Cluster ID uniform across units" over a SINGLE ovn-central unit, and the 12 scaled API services get no count, no hacluster and no VIP-ownership assertion. +EVIDENCE: `scripts/cloud-assert.sh:109-110`. Reproduced the arithmetic directly: units=3 -> `[ok]`; **units=1 -> `[ok]`**. The docstring at `:17` claims "uniform across the 3 units"; the code never asserts 3. `:94-96` is the only count assertion in 218 lines; `:71-73` (A0) selects only `error`/`blocked`. WOULD NOT CATCH: any application at the wrong scale; any hacluster at `cluster_count: 1`; any ABSENT application (`:100`, `:117` downgrade missing ovn-central / chassis principals to `warn ... skipped`). +BLOCKS-STAGE-5: no (exit verifier, not entry gate) -- but it is the second half of L4-3: nothing before OR after the deploy asserts real HA. | DISPOSITION: DOCFIX + +**ID: L4-11** +CLAIM: `pre-flight-checks.sh` (preflight P4) exits 0 -- "Summary: 0 fatal, 0 warning" -- when `jq` is absent, having NEVER contacted MAAS. +EVIDENCE: `:108` -- `need_jq || finish`. `need_jq` (`lib-net.sh:175-177`) RETURNS 1 and prints its own message; it never calls `fail()`, so FATAL stays 0 and `finish()` exits 0. MEASURED with real script+libs+bundle in a temp tree, PATH stripped of only `jq`: VIP checks PASS, `FAIL: jq not found on PATH`, `Summary: 0 fatal, 0 warning`, `EXIT CODE = 0`. Contrast `:109` where a genuine MAAS failure calls `fail` THEN `finish` and correctly exits 1. +BLOCKS-STAGE-5: no -- LATENT (`jq-1.7` present, measured). | DISPOSITION: DOCFIX + +**ID: L4-12** +CLAIM: `pre-flight-checks.sh` never selects a DC, so its VIP gate validates the dc0 base bundle's 11 VIPs and returns PASS while saying nothing about the per-DC overlay a VR1 stage actually deploys. +EVIDENCE: `:30-32` sources both libs and never calls either selector (grep: zero occurrences). CHECK 1 greps `bundle.yaml` by literal filename (`:75-95`) with no `--overlay` notion, so `overlays/vr1-dc1-vips.yaml` and `dc-ha-scaleup.yaml` are structurally INVISIBLE to P4. The overlay-aware gate exists and preflight invokes it WITHOUT either flag (`preflight.sh:33`). +BLOCKS-STAGE-5: no | DISPOSITION: log-only + +**ID: L4-13** +CLAIM: The 2026-07-27 fixes to the two per-DC artifact checkers hold, but both keep one "could not look" hole: their HTTP proof is wrapped in `if command -v curl` with NO else and NO `miss`. +EVIDENCE: `scripts/dc-mirror.sh:280-284` (the only proof the mirror SERVES anything) and `scripts/dc-cache-proxy.sh:210-217` -- in the proxy's case the curl block is the ENTIRE behavioral smoke test, so without curl the checker verifies config files and a listening socket and asserts nothing about whether the proxy serves a package. NOT CLAIMED: whether curl is present on the racks. +BLOCKS-STAGE-5: no | DISPOSITION: DOCFIX + +**ID: L4-14** +CLAIM: `dc-cache-proxy.sh` accepts a unit that would NOT start at boot as "enabled", where `dc-mirror.sh` correctly asserts the word. +EVIDENCE: `dc-cache-proxy.sh:198` uses `systemctl is-enabled --quiet` (exit code only); `dc-mirror.sh:267` compares the state WORD. MEASURED on this host: `is-enabled --quiet` returns rc=0 for units whose state word is `static` -- the state a unit takes when its `[Install]` section is absent, exactly the failure the message at `:200` warns about. +BLOCKS-STAGE-5: no | DISPOSITION: DOCFIX + +**ID: L4-15** +CLAIM (CONFIRMED, with one addition): `tests/creds-matrix` T24's baseline covers TIER 1 ONLY -- and the tier-3 V-classes are not even REPRESENTABLE by the classifier. +EVIDENCE: `tests/creds-matrix/run-tests.sh:261` runs the checker with NO flags; `creds-matrix.py:1063` gates tier 2 behind `--tier2/--tier3/--all`. `expected-findings.txt` contains exactly three tier-1 S-codes. ADDITION: `classify-findings.py:14` matches `re.match(r"([SE]\d)\b")`; the tier-3 codes are `V1` and `V2` -- neither matches, so a V-class FAIL classifies as a generic "schema/row defect". MITIGATION IN PLACE: the real tree's tier-2 state IS checked by the BLOCKING preflight P5. The gap is the gauntlet's regression PIN. +BLOCKS-STAGE-5: no | DISPOSITION: log-only + +### LENS 4 COULD NOT BREAK (positive results -- tried and failed to construct a false green) +1. `dc-mirror.sh check`'s `last-sync.status` handling (`:299-318`) -- all five arms traced: absent -> miss; `OK*` -> say; `RUNNING*` -> miss on BOTH sub-branches; `FAIL*` -> miss; `*` -> miss, "refusing to interpret an unknown status word as success". **The 07-27 fix HOLDS.** +2. `dc-cache-proxy.sh:176-177` -- shape of the classic precedence bug; truth table traced all four cases. Correct as written. +3. `provider-bundle-check.py` item 4 (relations) -- a DELETED application could not be made to pass. +4. `provider-bundle-check.py` item 7 -- the policyd zip is compared by CONTENT; four separate FAIL paths. +5. preflight's P4/P5 existence guards DO fail closed for a genuinely ABSENT file. The hole is the exit-code path (L4-5), not the guard. +6. `creds-matrix.py` tier-2 E0 probe states -- "unreadable" is a FAIL; "unreachable" is an `[ok]` that ALSO marks scope incomplete and downgrades rows to "NOT JUDGED". `:169` refuses an empty matrix; `:1068-1073` refuses a missing or EMPTY location list. **Could not turn "could not look" into a green judgement. The most disciplined checker in the set.** +7. `cloud-assert.sh` -- no admin scope -> WARN/HOLD exit 2, explicitly "a HOLD, not a skip"; mysql absent -> FAIL; missing A9 probe driver -> FAIL. +8. `run-tests-all.sh` -- a harness that crashes DOES exit nonzero and IS counted FAILED; only non-DISCOVERY evades. +9. `repo_lint.py` L11 rglobs the whole tree; no evasion beyond the documented opt-out. +10. Both artifact checkers exit 2 on an unknown site; `check dc1` failing by design is DOCUMENTED, not silently special-cased. +11. `creds-audit.sh` with a typo'd site exits 1 -- it does not treat "no manifest" as "nothing to check". + +### LENS 4 UNMEASURED +- The canonical piped rack-side checker invocations -- not run by this lens; whether `curl` is on the racks' PATH (L4-13) is UNMEASURED. +- `bash scripts/cloud-assert.sh` -- not run; no OpenStack model exists yet. L4-10's A3 claim is from source plus a reproduction of the identical shell arithmetic. +- G17's own check -- not runnable; all 18 nodes powered off. L4-7 is proved against a local HTTP server, not against `10.12.8.4`. +- `creds-matrix.py --tier2 --remote --privileged` -- not run by this lens. diff --git a/docs/audit/stage5-creds-privileged-20260727.txt b/docs/audit/stage5-creds-privileged-20260727.txt new file mode 100644 index 0000000..30abf6e --- /dev/null +++ b/docs/audit/stage5-creds-privileged-20260727.txt @@ -0,0 +1,27 @@ +=== creds-matrix: tier 1 (STATIC) === +=== creds-matrix: tier 2 (EXISTENCE) === + [ok] S1 schema: 82 rows, all enums valid, site-keys region-qualified, no duplicate (id,site,host-role,filename) + [ok] S3 render: 3 source field(s) SKIPPED -- rendering them needs the declared path from creds-manifests/vm-secret-locations (ruling 3); the list now EXISTS but the source-field derivation is not wired + [ok] S3 render drift: rendered row fields (mode, source) match checked-in; header prose and non-jumphost rows are OUT OF SCOPE of this compare + [ok] S4 mint-ref: every script:/runbook: reference resolves to a real location + [ok] S4 provenance debt: 30 row(s) are mint-ref=operator-terminal -- NOT reproducible from the repo (research FINDING 1). Admitted by design; converting them is remediation, not a checker fix. + [ok] S7 notes: 34 note key(s) referenced, all resolve, none orphaned + [ok] E0 jumphost location '~/vault-init/*' does not exist -- SKIPPED (typically a mint stage this deployment has not reached) + [ok] E0 jumphost location '~/octavia-pki/*' does not exist -- SKIPPED (typically a mint stage this deployment has not reached) + [ok] E0 jumphost location '~/octavia-pki/issuing-ca/*' does not exist -- SKIPPED (typically a mint stage this deployment has not reached) + [ok] E0 jumphost location '~/octavia-pki/controller-ca/*' does not exist -- SKIPPED (typically a mint stage this deployment has not reached) + [ok] E0 jumphost location '~/octavia-pki/controller/*' does not exist -- SKIPPED (typically a mint stage this deployment has not reached) + [ok] E0 headend location '/root/maas-secrets/*' read via ESCALATION (sudo -n, metadata only) after an unprivileged probe could not open it + [ok] E0 headend location '/var/snap/maas/current/root/.ssh/*' read via ESCALATION (sudo -n, metadata only) after an unprivileged probe could not open it + [ok] E0 netbox location '/root/netbox-secrets/*' read via ESCALATION (sudo -n, metadata only) after an unprivileged probe could not open it + [ok] E1 18 expected artifact(s) deferred as not-yet-minted per creds-manifests/stages-reached + [ok] tier 3 (VALIDITY) NOT RUN -- pass --tier3 (with --tier2) to compare cross-copy sha256 provenance + [FAIL] S2 vr1-dc0 EXPECTED-BUT-ABSENT: 'opnsense-api.txt' (id dc0-edge-api, SEC-021) is expected by the matrix but NOT declared in the manifest -- the credential is either missing or undeclared + [FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=id_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape + [FAIL] S5 ASYMMETRY: vr1-dc0 declares id=dcN-maas-power-key file=maas-virsh_ed25519 on headend (custody=off-manifest-known) with no counterpart in vr1-dc1 -- a per-DC credential must exist at BOTH DCs in the same shape + [FAIL] S5 ASYMMETRY: vr1-dc1 declares id=dcN-maas-power-key file=id_dcN_power on headend (custody=off-manifest-known) with no counterpart in vr1-dc0 -- a per-DC credential must exist at BOTH DCs in the same shape + [FAIL] S6 IDENTITY CONFLATION: id 'maas-region-admin' serves 2 principal types (human via gui; service via api, cli-profile) -- ruling 5 requires one identity to serve one principal type + [FAIL] E1 EXPECTED-BUT-ABSENT: dc0-edge-api 'opnsense-api.txt' expected at jumphost/vr1-dc0, not found (mint-stage stage3, SEC-021) + [FAIL] E4 UNCHECKABLE: 2 row(s) have no declared location for their (role, site) and can never be verified -- add a location row or correct the matrix: capi-mgmt-kubeconfig 'config' (cloud/-); rbd-mirror-peer-token 'rbd-mirror-bootstrap-token' (unit/-) + +FAIL: creds-matrix tier 1 -- 82 row(s), 16 check group(s) clean, 7 finding(s) diff --git a/docs/audit/stage5-live-measurement-20260727.txt b/docs/audit/stage5-live-measurement-20260727.txt index 7cbafa8..2a20232 100644 --- a/docs/audit/stage5-live-measurement-20260727.txt +++ b/docs/audit/stage5-live-measurement-20260727.txt @@ -68,6 +68,17 @@ P1 repo lint ............ FAIL (this session's uncommitted captures; see 2) P2 bundle invariants .... PASS (Pattern A / D-052-D-053) P3 channel assert ....... WARN (33 pins, 0 fail, 33 warn) + ** AMENDED after lens 4 -- this line was WRONG to wave through. The COUNT + was in the expected set; the CAUSE was not, and it is consequential. + All 33 warns read "juju info failed ([Errno 2] No such file or directory: + 'juju') -- verify manually". `channel_assert.py:41` folds a + MISSING BINARY into the same soft verdict as an unreachable charmhub, and + `:53` sets WARN whenever `fails` is empty. So ZERO of the 33 committed + Caracal channel pins are currently verified against charmhub by any + executable check -- on the eve of the deploy stage. The harness cannot see + it: tests/preflight T7 simulates only charmhub-UNREACHABLE and assumes the + binary exists. Ties directly to S-1 below: neither `juju` nor `openstack` + is installed on the host preflight runs on. ** P4 live pre-flight CHECK 0 octavia-pki .. FAIL overlays/octavia-pki.yaml MISSING (gitignored secret; known, expected)