Agent Runtime

anvil-agents is the sole controller implementation for the agent resources in control.anvil.hazyforge.io/v1alpha1. It can run independently of Anvil Primaris and does not read Application, Repository, task, build, release, or Hub resources.

Ownership

The operator owns these resources:

Application and target references are compatibility metadata. The operator compares their names for scope, concurrency, and launch controls, but never looks up another API group.

Run lifecycle

An AgentRun resolves its optional AgentRunProfile, selected AgentHarnessProfile, and ordered AgentSkillSet and AgentToolSet references, validates credentials and durable volume references, writes a payload ConfigMap, records the payload UID/digest and normalized Job execution digest in status, then records a create-attempt timestamp immediately before creating one Job. The Job and ConfigMap are owned by the run. Status is derived from the Job, pod state, logs, and the structured harness status contract.

Terminal phases are Succeeded, Failed, and NeedsHuman. Pending and running resources remain resumable. A controller restart observes the existing children rather than creating a replacement Job. If creation succeeded before the Job UID was persisted, recovery validates the pre-recorded payload and Job digests before accepting the child and does not depend on profiles that may have changed after launch planning. A missing Job after the create-attempt receipt is ambiguous and fails closed; the controller never creates a replacement that could duplicate external side effects.

The backend adapters are codex, openCode, hermesAgent, openClaw, grokBuild, piAgent, and custom. Backend images are selected by each harness profile or run. The six built-in adapters share the repository checkout, injected tool setup, tool verification, and prompt-context contract. The operator image does not bundle another control-plane CLI.

One run selects exactly one adapter. Named schedule templates can rotate or queue independent runs across adapters, but the controller does not create a shared multi-agent conversation. subagents are instructions to the selected harness, not controller-created child Jobs.

Profiles and schedules

Profiles contain durable role, scope, and policy defaults plus references to a runtime and capability packs. Run-local non-empty and non-zero compatibility fields override profile values; lists use field-specific append/deduplication rules. Use a harness-profile swap when inherited false/zero runtime values must be cleared. All profile, harness, skill-set, and tool-set references are namespace-local. See Agent Composition for precedence, atomic harness swaps, skill collision rules, and the four explicit override operations.

At Job materialization, status.resolvedComposition records the exact object versions and digests used. effectiveDigest covers the complete resolved AgentRun spec and payloadDigest covers the mounted ConfigMap data, including fetched remote skill content. This evidence is also available through the sanitized read API; it does not copy Secret values or skill contents into status. The sanitized snapshot also records inherited opaque application and target names, allowing the read API to return profile-owned scope without returning the profile itself.

Schedules create child runs from runTemplate or rotate through named runTemplates. Their concurrency policies are:

• Forbid: do not create a new run while a prior child is active.
• Allow: create due runs, optionally capped by maxConcurrentRuns.
• Queue: create due runs but launch only the oldest eligible pending runs.

maxRunsPerDay adds a UTC daily child-run budget across interval runs and manual run-now nudges. Omit it or set zero to preserve unlimited legacy behavior.

Allow, or Queue with a cap above one, exposes independent Job parallelism to the Kubernetes scheduler so heavy lanes can land on different machines. Forbid, a schedule cap of one, or an application control can deliberately serialize work even when the cluster has unused nodes. Adding workers does not bypass those policy and spend boundaries.

Set control.anvil.hazyforge.io/run-now to a new token for a replay-safe manual nudge. When named templates are configured, set control.anvil.hazyforge.io/run-template on the same update to choose one.

Runs that share spec.scope.applicationRef.name are subject to the matching active AgentRunControl.spec.maxConcurrentRuns values. The strictest matching positive value wins; the operator-wide --application-max-concurrent-runs value is the fallback when no control sets a limit. The default fallback is one. This key is opaque and does not require an Application CRD.

Launch controls

AgentRunControl is cluster-scoped. Paused blocks new Jobs for runs whose application key matches; it never terminates an existing Job. An optional expiresAt makes a pause inactive after the deadline. Source fields are audit metadata and do not establish authorization. Kubernetes API authorization is the authority boundary.

Durable storage

AgentDataVolume creates a PVC or accepts a compatible existing claim already controller-owned by the same AgentDataVolume identity. Topology-bound local storage can be migrated with append-only AgentDataVolumeCopy objects (see volume-copy.md): the controller creates a new destination volume and streams bytes across nodes; it never rewrites an existing claim. It exposes the resolved claim, mount, placement, and environment defaults in status. A VolumeProfile can provide reusable values. Cross-namespace mounts are rejected.

Storage requests may grow but never shrink. Claim names and storage classes are immutable after creation. When no storage class is selected, new claims use the cluster default. Compatible claims from the former embedded controller retain their current storage class when the same resource identity takes over. For WaitForFirstConsumer storage classes, a current Pending claim with the controller-owned ClaimPending status is allowed into the Job so that Job can be the binding consumer. Other Pending claims continue to block launch.

External object-store sync fields are declarative placeholders in v1alpha1; the operator reports them as stub-only and does not move data.

When a harness requests ttlSecondsAfterFinished, the controller records the request on the Job but leaves Kubernetes TTL cleanup disabled until terminal AgentRun status is durable. A crash can therefore delay Job cleanup, but it cannot delete the only execution evidence before the controller records the result and then launch a replacement.

Adverse streams

An AdverseSituation groups repeated events, deduplicates them, retains a bounded status buffer, and resolves after its quiet period. Agent responders are opt-in. Any application can submit immutable, same-namespace AdverseSignal evidence without importing its API into this repository.

The controller watches no external resource kinds by default. New pull integrations use structured adverseSources values with exact GVK/resource, namespace and label filters, destination routing, and status classification; the chart derives exact read RBAC. The legacy --adverse-source-gvks=apiVersion/kind option remains compatible and requires manual extraRBACRules. See Integrating Adverse Sources.

Credentials and identity

Secrets referenced by envSecretRefs are projected into the Job with envFrom. They must be in the run namespace. An optional ExternalSecret preflight can request and verify a fresh target Secret before Job creation. The controller still annotates each listed ExternalSecret with force-sync, but it does not require status.refreshTime to advance on every run: after a short grace it accepts a Ready ExternalSecret whose target Secret exists and whose last refreshTime is recent enough (default 15m). That covers static vault properties that reconcile healthy without rewriting the refresh stamp. The chart grants ExternalSecret mutation only when externalSecrets.enabled=true.

SPIFFE Workload API mounting is opt-in and requires an exact SPIFFE ID. The operator only mounts the CSI socket; workload registration and authorization remain external responsibilities.

Structured status

Harness images should emit newline-delimited JSON through ANVIL_AGENT_RUN_STATUS_TOOL or the configured status file. A terminal report should include the action, summary, remaining risk, human requirement, and pull request URL when applicable. The controller retains recent reports in status.

An optional Postgres archive is enabled with the Secret-backed ANVIL_AGENTS_ARCHIVE_DATABASE_URL environment variable. The controller does not accept the database URL as a CLI flag because process arguments are not a credential-safe transport. --terminal-retention enables pruning only after terminal status has been archived successfully. The historical table name is retained so deployments can adopt existing archive records.

The Helm chart supports an external database, a small standalone PostgreSQL StatefulSet, or a CloudNativePG Cluster; every mode still supplies the same Secret-backed URI. The legacy archive.databaseURLSecret input remains a deprecated external-mode alias. Set archive.terminalRetention only after a real archive row is verified. See PostgreSQL Archive for the mode, credential, upgrade, backup, and uninstall contracts.

Creating a workable agent

For a practical checklist covering ToolSets, skills, branch scope, schedules, human-comms hotline, and a first smoke run, see Creating a Workable Agent End to End.

Repository and branch policy live on spec.scope.repository (profile or run):

• destinationBranch — only allowed PR base / integration branch
• allowedBranches — heads the agent may analyze or use
• ref — workspace checkout (defaults to destinationBranch)

The controller injects ANVIL_AGENT_RUN_REPOSITORY*, ANVIL_AGENT_RUN_DESTINATION_BRANCH, and ANVIL_AGENT_RUN_ALLOWED_BRANCHES.

Installation

Install the CRDs and controller with Helm:

helm upgrade --install anvil-agents charts/anvil-agents \
  --namespace anvil-agents-system --create-namespace

Configure concrete backend image digests in profiles. The chart defaults do not create credentials, workload service accounts, external source RBAC, or agent profiles.

The CRDs carry helm.sh/resource-policy: keep; uninstalling the Helm release retains them. Deliberately deleting CRDs deletes their custom resources and may garbage-collect PVCs owned by AgentDataVolume.

The optional anvil-agents-api serves sanitized run summaries and bounded live SSE logs to OIDC-authorized clients without Kubernetes credentials. It is disabled by default and runs with a separate read-only ServiceAccount. See live-agent-run-stream.md for its security boundary, provider-neutral OIDC configuration, and client workflow.

Useful operator flags:

Local verification

make verify
make images

make verify regenerates deep copies and CRDs, copies CRDs into the chart, runs all Go tests, compiles both binaries, and lints/renders the Helm chart. make images calls hack/build-images.sh, which builds the controller and all six built-in runner images into local Docker by default. It can select individual components or push the same image set to any authenticated registry, so GitHub Actions is optional.

Compatibility contract

The API group, kind names, label keys, owner references, and archive table name remain stable for takeover. Only one controller deployment may reconcile these resources at a time. Follow migration-from-anvil-primaris.md for the handoff sequence.