diff --git a/.claude/skills/openstack-cloud-ops/references/script-authoring.md b/.claude/skills/openstack-cloud-ops/references/script-authoring.md index 1bf6aea..635dcc1 100644 --- a/.claude/skills/openstack-cloud-ops/references/script-authoring.md +++ b/.claude/skills/openstack-cloud-ops/references/script-authoring.md @@ -164,6 +164,13 @@ decompose-detection, substrate-collision aborts). A `--no-prompt` flag on a destructive script exists FOR its harness, nothing else. +**Harnesses are HERMETIC: fakes shadow, deletion un-shadows.** Never simulate +an unavailable tool by DELETING its fake -- on a real host that un-shadows the +real binary and the test falls through to live infrastructure (a jumphost +gauntlet run reached real Charmhub this way). Simulate failure with a FAILING +fake (prints a realistic error, exits nonzero); keep the fail-loud-on-unmatched +arm in every fake so an uncovered call can never silently hit a real CLI. + **A script migration commit MUST carry its harness.** The D-060 revert updated the scripts and left their harnesses testing the retired D-058 world -- red-at- HEAD tests that trained everyone to ignore red. If you change a script's diff --git a/docs/changelog-20260703-process-hardening.md b/docs/changelog-20260703-process-hardening.md index 9eaba71..b2af03d 100644 --- a/docs/changelog-20260703-process-hardening.md +++ b/docs/changelog-20260703-process-hardening.md @@ -355,3 +355,17 @@ the permission+hook layer is the v1 boundary. Revisit at Roosevelt. - disableBypassPermissionsMode: managed-settings scope (system paths, admin) -- worth doing when a second operator joins the jumphost. + +### 32. Block 5.1 -- hermetic-harness fix (caught by the FIRST jumphost gauntlet run) +tests/preflight T7 simulated "charmhub unreachable" by DELETING the fake juju +-- which un-shadowed the REAL juju on the jumphost, fell through to LIVE +Charmhub (a hermeticity violation: a test made a network call), found the pin +valid, and correctly returned 0 against the test's expectation of 2. FIX: T7 +now installs a FAILING fake (realistic connection error, exit 1) -- identical +behavior on any host. Same-class sweep: the carve and standup fakebin maas +unmatched arms returned '{}' exit 0 (silent-tolerate); flipped to fail-loud +(harnesses stay green, proving no uncovered calls exist today and none can +sneak in silently tomorrow). Rule encoded in the skill (script-authoring): +fakes SHADOW and fail loud; never simulate absence by deletion. The .skill +artifact regenerated from .claude/skills/ source. Credit: the gauntlet-on- +real-host practice worked exactly as designed on its first run. diff --git a/skills/openstack-cloud-ops/openstack-cloud-ops.skill b/skills/openstack-cloud-ops/openstack-cloud-ops.skill index 55c1740..b2eb349 100644 --- a/skills/openstack-cloud-ops/openstack-cloud-ops.skill +++ b/skills/openstack-cloud-ops/openstack-cloud-ops.skill Binary files differ diff --git a/tests/carve-host-interfaces/fakebin/maas b/tests/carve-host-interfaces/fakebin/maas index 3a60979..096b744 100644 --- a/tests/carve-host-interfaces/fakebin/maas +++ b/tests/carve-host-interfaces/fakebin/maas @@ -5,4 +5,4 @@ [ "$obj" = machine ] && [ "$act" = read ] && { cat "${FIX_MACHINE:?}"; exit 0; } [ "$obj" = subnets ] && [ "$act" = read ] && { cat "${FIX_SUBNETS:?}"; exit 0; } [ "$obj" = interfaces ] && [ "$act" = read ] && { cat "${FIX_IFACES:?}"; exit 0; } -echo "{}"; exit 0 # unexpected (mutations) -- never hit in dry-run +echo "fake-maas: UNMATCHED call: $*" >&2; exit 1 # fail loud: an uncovered call must never pass silently diff --git a/tests/phase-00-maas-standup/fakebin/maas b/tests/phase-00-maas-standup/fakebin/maas index c57f41e..664557f 100644 --- a/tests/phase-00-maas-standup/fakebin/maas +++ b/tests/phase-00-maas-standup/fakebin/maas @@ -14,4 +14,4 @@ fi echo "Unable to find fabric with id '$fab'." >&2; echo "Not Found"; exit 2 ;; esac -echo "{}"; exit 0 +echo "fake-maas: UNMATCHED call: $*" >&2; exit 1 # fail loud: an uncovered call must never pass silently diff --git a/tests/preflight/run-tests.sh b/tests/preflight/run-tests.sh index 2fde066..dd2e240 100644 --- a/tests/preflight/run-tests.sh +++ b/tests/preflight/run-tests.sh @@ -48,8 +48,16 @@ # T6: channel pinned to a track fakebin does not publish -> FAIL D=$(mkfix badpin 0 0 1); printf 'applications:\n vault: {charm: vault, channel: 9.9/stable}\n' > "$D/bundle.yaml" run 1 'does not publish' "T6 unpublished channel pin FAILS" "$D" -# T7: juju unreachable -> WARN (exit 2), not FAIL -D=$(mkfix offline 0 0 1); rm "$D/fakebin/juju" +# T7: charmhub unreachable -> WARN (exit 2), not FAIL. +# HERMETIC: simulate failure with a FAILING fake, never by deleting the fake -- +# deletion un-shadows the REAL juju on a real host (the jumphost gauntlet run +# fell through to live charmhub and returned 0). Fakes must always shadow. +D=$(mkfix offline 0 0 1) +cat > "$D/fakebin/juju" <<'FB' +#!/usr/bin/env bash +echo "ERROR cannot connect to charmhub.io: connection timed out" >&2; exit 1 +FB +chmod +x "$D/fakebin/juju" run 2 'verify .* manually' "T7 charmhub unreachable -> WARN" "$D" echo; echo "RESULT: PASS=$PASS FAIL=$FAIL"