diff --git a/docs/changelog-20260709-mtu-ceph-budget-calculators.md b/docs/changelog-20260709-mtu-ceph-budget-calculators.md new file mode 100644 index 0000000..e9538b7 --- /dev/null +++ b/docs/changelog-20260709-mtu-ceph-budget-calculators.md @@ -0,0 +1,138 @@ +# Changelog 2026-07-09 -- MTU/geneve budget + Ceph disk-budget calculators (DOCFIX-162) + +No live infrastructure touched -- read-only, arithmetic-only shell scripts +plus test harnesses only. Closes tooling gap register item #7 in +`docs/dc-dc-deployment-workflow.md`, which named the measurement-then- +computation gates in D-101 and buildout-design Section 3 as manual +arithmetic in prose today, not a script -- blocking the Stage 1 Phase-0 +gates in `runbooks/dc-dc-phase0-vcloud-prep.md` Step 3. + +**Numbering note:** several other agents are working the tooling gap +register in parallel this session, all uncommitted -- to avoid a +collision on the next-free DOCFIX number, this delivery uses the literal +placeholder `DOCFIX-162` everywhere a real number would go (this file, +the gap-register update, the session-ledger entry). The orchestrating +session assigns the real sequential number and find-replaces it at +integration/commit time. + +## Item + +### 1. DOCFIX-162 -- `scripts/dc-dc-mtu-geneve-budget.sh` +FILES: `scripts/dc-dc-mtu-geneve-budget.sh`, new +`tests/dc-dc-mtu-geneve-budget/run-tests.sh`. + +WHAT: turns D-101's ("Tenant/MTU sub-policy", folded in from D-102) +geneve-over-v6 MTU arithmetic into a runnable, read-only calculator. +Required `--underlay-mtu ` (no default -- the measured host L2 MTU +per Step 3's own `ip -o link show | awk '{print $2, $0}' | grep -i mtu` +check); optional `--jumbo-threshold` (default 9000, overridable for policy +exploration only). Computes: `>= 9000` -> jumbo, tenant MTU stays 1500; +otherwise `tenant_mtu = underlay_mtu - 56` (56 = 40 IPv6 outer header + 8 +UDP + 8 Geneve base, quoted verbatim from D-101), matching D-101's own +worked example exactly (1500 - 56 = 1444). Added one sanity floor NOT +itself part of D-101's text: if the computed tenant MTU falls below 1280 +(the IPv6 protocol's own minimum link MTU, RFC 8200 Sec 5), the verdict is +FAIL rather than a silent PASS -- clearly labeled in the script's header as +a separate, defensible addition, not D-101 arithmetic. Prints a +human-readable computation, a PASS/FAIL verdict, and a single +`MTU-BUDGET: ...` summary line for pasting into a changelog/as-executed +log, matching what Step 3 asks the operator to record. + +VERIFIED: +- New harness `tests/dc-dc-mtu-geneve-budget/run-tests.sh`: 19/19 PASS. + Covers the exact D-101 worked example (1500 -> 1444, both the + human-readable line and the summary line reproduce verbatim), jumbo + input (9000 and above) correctly leaves tenant MTU at 1500, the jumbo + threshold boundary (exactly-9000 vs 8999), missing `--underlay-mtu` + fails loud (rc 2, clear message), non-numeric/zero/negative + `--underlay-mtu` and bad `--jumbo-threshold` all fail loud (rc 2), the + IPv6-minimum-MTU sanity floor triggers FAIL (rc 1) at 1300 and 1280 + underlay, `--help` exits 0 and cites both D-101 and the 56-byte + overhead, and an unknown flag fails loud (rc 2). +- Manually re-ran the worked example, jumbo case, and floor case directly + (not just through the harness) to confirm output text before writing the + harness's regexes against it. +- `bash scripts/repo-lint.sh`: 0 fail, 1 documented legacy WARN (unchanged + D-001..018 non-ASCII carve-out; nothing this delivery touched). + +REVERT: `rm scripts/dc-dc-mtu-geneve-budget.sh && rm -rf +tests/dc-dc-mtu-geneve-budget` (safe -- no other script calls this one +yet; it is a standalone calculator, not sourced by anything). + +### 2. DOCFIX-162 -- `scripts/dc-dc-ceph-disk-budget.sh` +FILES: `scripts/dc-dc-ceph-disk-budget.sh`, new +`tests/dc-dc-ceph-disk-budget/run-tests.sh`. + +WHAT: turns buildout-design Section 3 / D-101's Ceph size=3-vs-size=2 +disk-budget arithmetic into a runnable, read-only calculator. Required +inputs, ALL with no default (fails loud if any is missing): `--total-disk +` (measured total disk, Step 2's `df`/`lsblk` output), `--dc1-nodes +` / `--dc1-per-node-osd ` and `--dc2-nodes ` / +`--dc2-per-node-osd ` (measured or firmly planned per-DC Ceph OSD +node count and per-node raw footprint -- already reflecting each +cluster's own size=3 replication as it would actually consume disk), and +`--backup-overhead-fraction ` (a decimal fraction of the +combined DC1+DC2 Ceph total representing radosgw backup pools + cross-DC +multisite duplication + Glance rbd-mirror targets + images + per-DC +mirrors -- quoted directly from Section 3's own itemization). This +fraction is NOT specified as a hard number anywhere in the repo (checked +`docs/design-decisions.md` and `docs/dc-dc-buildout-design.md` +directly) -- so it has NO default and the script refuses to run without +it explicitly passed; its header documents that it must be estimated from +real Ceph/radosgw-admin usage measurements once DC1's cluster exists. +Sizes accept plain byte integers or 1024-based `Ki/Mi/Gi/Ti`/`K/M/G/T` +suffixes (matching `df -h` binary units). + +Computes `required = (dc1_total + dc2_total) * (1 + overhead_fraction)` +against the measured total disk. If it fits: PASS, prints the margin. If +not: FAIL, prints the shortfall, and states -- but does NOT compute or +silently apply -- that size=2 is the ONLY documented fallback per D-101 / +Section 3, and that adopting it is a decision the OPERATOR must explicitly +log (its own decision note / changelog entry). Deliberately does NOT +invent a size=2 disk-requirement number: the real per-node footprint under +size=2 depends on a distinct measurement this repo does not have yet, and +fabricating a scaling factor would be exactly the unmeasured-arithmetic +this repo's discipline forbids. Prints a `CEPH-BUDGET: ...` summary line +for the same changelog/as-executed-log use as the MTU script. + +VERIFIED: +- New harness `tests/dc-dc-ceph-disk-budget/run-tests.sh`: 16/16 PASS. + Covers a constructed size=3-fits numeric example (3 nodes x 500G per DC, + 30% overhead, 10TiB measured total -> ~3.80TiB required, PASS with + ~6.19TiB margin, exact summary line asserted) and a constructed + size=3-does-not-fit example (same Ceph footprint against a 2TiB measured + total -> FAIL, exact shortfall asserted, size=2 fallback named, + explicit-logged-decision language asserted, never applied silently); + each of the six required arguments fails loud (rc 2) when missing, + specifically confirming `--backup-overhead-fraction` has no default and + blocks the run alone; bad numeric/size formats (non-integer node count, + non-decimal fraction, bad size suffix) all fail loud (rc 2); a + plain-byte total-disk value produces the same PASS verdict as its + 1024-based-suffix equivalent (10995116277760 bytes == 10Ti); `--help` + exits 0 and cites Section 3 and the no-default-fraction rule; an unknown + flag fails loud (rc 2). +- Manually re-ran both constructed examples directly to confirm output + text before writing the harness's regexes against it. +- `bash scripts/repo-lint.sh`: 0 fail, 1 documented legacy WARN (unchanged). + +REVERT: `rm scripts/dc-dc-ceph-disk-budget.sh && rm -rf +tests/dc-dc-ceph-disk-budget` (safe -- standalone, nothing else calls it +yet). + +## Next actionable step + +Tooling gap register item #7 CLOSED (mechanism). Both calculators are +read-only and take every varying input as a required, explicitly-measured +argument -- no invented defaults anywhere, matching hard rule 2. Neither +has been run against the real vcloud host yet (no live infrastructure this +session, prep-only per the operator's own instruction) -- that first real +run is part of executing `runbooks/dc-dc-phase0-vcloud-prep.md` Step 3 +tomorrow morning: measure the real interface MTU and feed it to +`dc-dc-mtu-geneve-budget.sh`, and once Step 2's `df`/`lsblk` numbers and a +real (or firmly planned) per-node OSD footprint exist, feed those to +`dc-dc-ceph-disk-budget.sh`. The `--backup-overhead-fraction` input +specifically remains a genuine open item until DC1's Ceph cluster exists +and reports real radosgw/rbd-mirror pool usage (Section 3's own point: +"the reserve split ... is set in Phase 0 from the measured DC0 per-node +OSD footprint") -- this delivery gives the arithmetic a home, not a real +number to plug into it yet. diff --git a/scripts/dc-dc-ceph-disk-budget.sh b/scripts/dc-dc-ceph-disk-budget.sh new file mode 100644 index 0000000..bd0809d --- /dev/null +++ b/scripts/dc-dc-ceph-disk-budget.sh @@ -0,0 +1,218 @@ +#!/usr/bin/env bash +# scripts/dc-dc-ceph-disk-budget.sh --total-disk --dc1-nodes +# --dc1-per-node-osd --dc2-nodes --dc2-per-node-osd +# --backup-overhead-fraction +# +# Phase-0 gate calculator (tooling gap register item #7): turns +# buildout-design Section 3 / D-101's Ceph size=3-vs-size=2 disk-budget +# prose into a runnable check, per `runbooks/dc-dc-phase0-vcloud-prep.md` +# Step 3. Mutates NOTHING -- read-only arithmetic against operator-supplied, +# MEASURED inputs; safe to re-run any number of times. +# +# Arithmetic quoted verbatim (docs/dc-dc-buildout-design.md Section 3, +# "Host and resource plan (Phase-0 gates)"): +# "the Phase-0 disk math must account for: DC1 Ceph (size=3) + DC2 Ceph +# (size=3) + per-DC radosgw backup pools (themselves replicated within +# each cluster, and duplicated cross-DC by multisite) + Glance rbd-mirror +# targets + images + per-DC mirrors. If size=3 across both clusters plus +# the backup/mirror footprint exceeds [the measured total disk], fall back +# to size=2 as an EXPLICIT, documented test-only deviation." +# +# Model: this script takes the MEASURED (or firmly planned) per-node OSD +# raw-disk footprint for each DC's cluster -- i.e. what that cluster's own +# replication factor already consumes on each host, exactly as `df`/`ceph +# osd df` would report it once the cluster exists -- multiplies by node +# count to get each DC's Ceph total, sums both DCs, then adds the +# backup/mirror/image overhead as an operator-supplied FRACTION of that +# combined Ceph total (radosgw backup pools + cross-DC multisite +# duplication + Glance rbd-mirror targets + images + per-DC artifact +# mirrors, per the quoted passage). That fraction is NOT specified anywhere +# in this repo as a hard number (checked docs/design-decisions.md and +# docs/dc-dc-buildout-design.md) -- so --backup-overhead-fraction has NO +# DEFAULT and the script refuses to run without it. It must be estimated +# from real Ceph/radosgw-admin usage measurements once DC1's cluster exists +# and reports actual pool sizes (Section 3: "the reserve split ... is set in +# Phase 0 from the measured DC0 per-node OSD footprint"). +# +# This script does NOT compute a size=2 disk requirement number: the exact +# per-node footprint under size=2 depends on how the operator would actually +# reconfigure the cluster (a distinct measurement this repo does not yet +# have), and inventing a scaling factor here would be exactly the kind of +# unmeasured arithmetic this repo's discipline forbids. If size=3 does not +# fit, the verdict says so and names size=2 as the ONLY documented fallback +# (D-101 / Section 3) -- and states plainly that adopting it is a decision +# the OPERATOR must explicitly log (its own decision note / changelog +# entry), never something this script silently applies. +# +# Size format: an integer number of bytes, or an integer with a binary +# suffix Ki/Mi/Gi/Ti or K/M/G/T (both forms treated as 1024-based, matching +# `df -h` binary units) -- e.g. 10T, 10Ti, and 10995116277760 are equivalent. +# No decimal suffixes (no SI/1000-based parsing) to avoid a silent unit +# mismatch against `df`/`lsblk` output. +# +# Usage: +# dc-dc-ceph-disk-budget.sh --total-disk \ +# --dc1-nodes --dc1-per-node-osd \ +# --dc2-nodes --dc2-per-node-osd \ +# --backup-overhead-fraction +# +# --total-disk SIZE REQUIRED. Measured total disk on the +# vcloud host (Step 2's `df`/`lsblk` +# output). No default. +# --dc1-nodes N REQUIRED. Measured/planned Ceph OSD node +# count, DC1. No default. +# --dc1-per-node-osd SIZE REQUIRED. Measured/planned per-node raw +# OSD disk footprint, DC1 (already +# reflecting DC1's own size=3 replication +# as it would actually consume disk). No +# default. +# --dc2-nodes N REQUIRED. Same, DC2. No default. +# --dc2-per-node-osd SIZE REQUIRED. Same, DC2. No default. +# --backup-overhead-fraction F REQUIRED. Decimal fraction (e.g. 0.4 for +# 40%) of the combined DC1+DC2 Ceph total +# representing radosgw backup pools + +# cross-DC multisite duplication + Glance +# rbd-mirror targets + images + per-DC +# mirrors. NO DEFAULT -- must come from real +# measurement once DC1 exists; this script +# refuses to run without it explicitly +# passed. +# -h|--help this help. +# +# Exit: 0 PASS (size=3 + overhead fits within measured total disk) | 1 +# FAIL/HOLD (does not fit -- size=2 fallback must be explicitly logged by +# the operator, not silently applied) | 2 usage error (missing/bad +# argument). ASCII + LF. +set -uo pipefail +shopt -s inherit_errexit 2>/dev/null || true + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +usage() { sed -n '2,85p' "$HERE/$(basename "${BASH_SOURCE[0]}")" | sed 's/^# \{0,1\}//'; } + +TOTAL_DISK=""; DC1_NODES=""; DC1_PER_NODE=""; DC2_NODES=""; DC2_PER_NODE=""; OVERHEAD_FRAC="" + +while [ $# -gt 0 ]; do + case "$1" in + --total-disk) TOTAL_DISK="${2:-}"; shift 2 ;; + --total-disk=*) TOTAL_DISK="${1#*=}"; shift ;; + --dc1-nodes) DC1_NODES="${2:-}"; shift 2 ;; + --dc1-nodes=*) DC1_NODES="${1#*=}"; shift ;; + --dc1-per-node-osd) DC1_PER_NODE="${2:-}"; shift 2 ;; + --dc1-per-node-osd=*) DC1_PER_NODE="${1#*=}"; shift ;; + --dc2-nodes) DC2_NODES="${2:-}"; shift 2 ;; + --dc2-nodes=*) DC2_NODES="${1#*=}"; shift ;; + --dc2-per-node-osd) DC2_PER_NODE="${2:-}"; shift 2 ;; + --dc2-per-node-osd=*) DC2_PER_NODE="${1#*=}"; shift ;; + --backup-overhead-fraction) OVERHEAD_FRAC="${2:-}"; shift 2 ;; + --backup-overhead-fraction=*) OVERHEAD_FRAC="${1#*=}"; shift ;; + -h|--help) usage; exit 0 ;; + *) echo "FAIL: unknown argument: $1" >&2; echo " run with --help for usage" >&2; exit 2 ;; + esac +done + +# --- validation: every input REQUIRED, no defaults (hard rule 2) ------------ +missing=0 +for pair in "--total-disk:$TOTAL_DISK" "--dc1-nodes:$DC1_NODES" \ + "--dc1-per-node-osd:$DC1_PER_NODE" "--dc2-nodes:$DC2_NODES" \ + "--dc2-per-node-osd:$DC2_PER_NODE" \ + "--backup-overhead-fraction:$OVERHEAD_FRAC"; do + name="${pair%%:*}"; val="${pair#*:}" + if [ -z "$val" ]; then + echo "FAIL: $name is REQUIRED (no default) -- see --help" >&2 + missing=1 + fi +done +[ "$missing" -eq 1 ] && exit 2 + +is_pos_int() { [[ "$1" =~ ^[0-9]+$ ]] && [ "$1" -gt 0 ]; } +if ! is_pos_int "$DC1_NODES"; then echo "FAIL: --dc1-nodes must be a positive integer; got '$DC1_NODES'" >&2; exit 2; fi +if ! is_pos_int "$DC2_NODES"; then echo "FAIL: --dc2-nodes must be a positive integer; got '$DC2_NODES'" >&2; exit 2; fi + +# fraction: decimal >= 0 (e.g. 0, 0.4, 1, 1.25) +if ! [[ "$OVERHEAD_FRAC" =~ ^[0-9]+(\.[0-9]+)?$ ]]; then + echo "FAIL: --backup-overhead-fraction must be a non-negative decimal (e.g. 0.4); got '$OVERHEAD_FRAC'" >&2 + exit 2 +fi + +# --- size parsing: integer bytes, or Ki/Mi/Gi/Ti / K/M/G/T (1024-based) ----- +parse_size() { #