LP report DRAFT (operator to file against maas) -- 2026-07-21

Copy-paste body below. Evidence pointers at the end are repo-internal -- do not paste those into the public report; attach sanitized excerpts instead (agent journal lines, dig outputs -- NO cluster secrets, the 2026-07-21 as-executed log contains SEC-014 material).

Title: maas-agent resolver on rack-only controller SERVFAILs authoritative maas-internal queries on internet-isolated racks (walks root hints; ignores resolv.conf); breaks commissioning cloud-config-url

Affects: MAAS 3.7.2 (snap 3.7/stable, rev 41649), rack-only controller.

Environment: one region+rack controller (site A) and one RACK-ONLY controller (site B) enrolled to it over a routed point-to-point link. The rack site is internet-isolated by design (no outbound DNS/IP reach from the rack or its nodes; the rack reaches the region over the point-to-point link only). Nodes at site B are KVM VMs PXE-booting from the rack.

What happens: commissioning nodes at the rack-only site PXE fine, boot the ephemeral image fine, then cloud-init finds no datasource and the node idles at a login prompt until the 30-minute node_timeout. The commissioning cloud-config-url uses a <subnet>.maas-internal name. Node-facing DNS at a rack-only site is served by the maas-agent resolver (the rack's dhcpd hands out the rack IP as DNS). That resolver answers SERVFAIL to EVERY query -- including names in its own authoritative maas-internal zone -- because it attempts full recursive resolution from the public root hints (unreachable on an isolated rack), and it ignores /etc/resolv.conf. There appears to be no on-disk configuration surface for it (config is Temporal-pushed).

Expected: the agent resolver should answer authoritatively for maas-internal without recursion (it owns the zone), and/or fall back to the host resolver configuration for everything else -- an isolated rack with a healthy region link should be able to commission nodes.

Why co-located deployments never see this: with region+rack on one host, node DNS is served by the region's BIND (authoritative, recursion via forwarders), so the defective path is only exercised by rack-only controllers -- i.e. exactly the multi-site topology racks exist for.

Workaround (proven): dnsmasq on the rack bound to an alias on the node-facing bridge, no-resolv, server=<region BIND IP> (reachable over the rack->region link); point the subnet's dns_servers at the alias with allow_dns=false. Commissioning then completes in minutes. Note: dhcpd still PREPENDS the rack IP to DHCP option 6 despite allow_dns=false, so clients succeed only via SERVFAIL failover to the second server -- possibly a separate template bug worth its own look.

Repo-internal evidence (do NOT paste): the agent journal excerpts and dig outputs are in ~/as-executed/2026-07-21-ops-commissioning-diag.log (0600; contains SEC-014 material -- sanitize before attaching); adjudication chain docs/audit/commissioning-diag-20260721.txt; D-131 context docs/design-decisions.md.