Newer
Older
openstack-caracal-ipv4 / clientdocs / tenant-skill / references / ci-automation.md

CI and automation against your tenancy

Distilled from the CI/Automation Integration Guide delivered at handover; that guide is authoritative if they ever disagree.

The one rule: pipelines use the application credential

ALL automation authenticates as the -svc account's application credential ({{TENANT_SHORT_NAME}}-svc-cred). Never a password. Why:

  • Scoped: it acts only inside your project, with exactly the -svc rights. It cannot touch identity, quotas, or anything outside your domain -- a leaked pipeline credential is contained by design.
  • Revocable without collateral: delete the credential and every pipeline using it stops, but the account and your other credentials are untouched. Rotation never needs the operator.
  • No interactive dependency: it does not expire with a password change.

Platform-enforced corollaries (expected refusals, not bugs): the application credential cannot create/delete Kubernetes clusters (see references/kubernetes.md), cannot create users or grant roles, and cannot change quotas.

Secret-store handling

The credential's id and secret live in your CI system's native secret store (Jenkins credentials, GitLab CI variables, GitHub Actions secrets), injected into jobs as environment variables. Never:

  • commit them to a repository (including a private one),
  • bake them into a container or VM image,
  • echo them in build output (mask them in the CI configuration).

When asked to verify a credential, check length/format in a script; never print the value.

Pipeline authentication

clouds.yaml and OS_* environment-variable forms are in SKILL.md; the same values work for Terraform/OpenTofu's OpenStack provider, Ansible's openstack.cloud collection, and the OpenStack SDK. In a pipeline step:

export OS_CLOUD={{TENANT_SHORT_NAME}}
openstack token issue        # smoke test: token scoped to your project

The delivered starter script bash scripts/smoke-test.sh wraps the full smoke test (token, catalog, quota headroom) as one pipeline-ready step.

Never hardcode service endpoints; authenticate against {{AUTH_URL}} and let the client discover the rest from the catalog. Always ship the delivered CA bundle to the job environment and point cacert / OS_CACERT at it.

Least privilege: one credential per pipeline, with expiry

Use the handover credential only as the "root" automation credential. Sign in as {{TENANT_SHORT_NAME}}-svc (password, interactively -- not from CI) and mint a narrower credential per pipeline:

openstack application credential create pipeline-nightly \
  --role member --expiration <YYYY-MM-DDTHH:MM:SS>

One credential per pipeline means you can revoke one without stopping the others, and the audit trail says which pipeline did what. Revocation is immediate and self-service:

openstack application credential delete pipeline-nightly

Rotate by creating the new credential first, cutting the pipeline over, then deleting the old one.

The worked 9-step sequence (canonical pipeline order)

The same order the Acceptance Checklist proves at onboarding. Name everything with your build id (e.g. ci-<build>-web).

  1. Authenticate + discover: openstack token issue, then openstack catalog list.
  2. Network build-out: network, subnet (any private range that does not overlap your own subnets), router gatewayed to provider-ext, attach subnet -- or reuse the handover network for simple runs.
  3. Security group with exactly the ports the deployment needs.
  4. Compute: boot a server from a shared base image with the keypair; wait for ACTIVE.
  5. Block storage: create a volume, attach, verify in-use, detach.
  6. Public access: allocate a floating IP from provider-ext, attach, verify reachability.
  7. Load balancer: create on your subnet with listener, pool, and the server as member; wait for ACTIVE/ONLINE.
  8. Secrets: store, retrieve, delete a test secret.
  9. Teardown in REVERSE order -- member/pool/listener/load balancer, floating IP (release it), server, volume, security group, router interface/router/subnet/network -- then verify with the list commands that nothing is left.

Command sequences for each step are in references/day2-operations.md. Keep a trimmed version of the sequence as the pipeline's recurring smoke test.

Cleanup and pacing discipline

  • Quota is the hard envelope; every create is checked server-side. Treat quota-exceeded as an expected, actionable failure (clean up leaked resources or request a raise via an authorized requester), never as a retryable platform fault. A pre-flight openstack limits show --absolute in long pipelines is cheap insurance.
  • Poll at 10-second intervals or slower, or use --wait.
  • Serialize heavyweight creates (load balancers, clusters).
  • Make teardown a step that runs ON FAILURE TOO. A periodic sweep job deleting ci-* resources older than your longest pipeline is the cheapest way to keep quota headroom. Floating IPs count against quota even when detached -- release them. The delivered starter script bash scripts/ci-cleanup-sweep.sh reports leftovers and detached floating IPs (dry-run by default -- it deletes nothing unless you pass --apply); schedule the dry-run and act on its report, or schedule --apply once you trust the prefix filter.

When a pipeline call fails

Triage in order -- see references/troubleshooting.md. Short form: token first, quota second, boundary third, verbatim error to {{ACCOUNT_CONTACT}} last.