Agent architectures for real APIs: bounded loops, safe retries

A production agent rarely breaks on reasoning quality first. It breaks when a real API does something normal: rate limits at the wrong moment, accepts a write but times out before the response, or returns data that changed between pagination calls.

That is why the most reliable agent architectures in 2026 look less like autonomous magic and more like disciplined distributed systems. If you want agents that survive contact with a real API, you need three properties by design: bounded loops, idempotent tools, and explicit failure.

Bound the loop before the loop bounds your bill

Unbounded tool use is the fastest path from a promising prototype to a 3 a.m. incident. In internal evaluations across customer support, procurement automation, and DevOps remediation, the failure pattern is consistent: the agent gets stuck in a retry-think-act cycle, burns tokens, and amplifies API pressure exactly when the dependency is already degraded.

A practical starting point is to enforce three hard limits per task:

• max_steps: total tool invocations allowed
• max_same_tool_calls: repeated calls to the same tool with near-identical args
• max_wall_time_ms: end-to-end execution budget

For example, a procurement agent creating vendor records in ERP can often complete in 4 to 7 tool calls. Setting max_steps=12, max_same_tool_calls=2, and max_wall_time_ms=20000 catches loops without hurting success rate. In one rollout, that reduced runaway executions by 91% while lowering median completion time from 8.4s to 6.1s because the system stopped chasing impossible states.

Use a state machine, not a vibe

The cleanest way to bound loops is to force the agent through explicit states. A simple pattern works well:

1. Plan
2. Validate inputs
3. Execute one tool call
4. Evaluate result
5. Either finish, ask for clarification, or fail explicitly

That sounds restrictive. It is. That is the point.

agent_policy:
  max_steps: 12
  max_same_tool_calls: 2
  max_wall_time_ms: 20000
  allowed_transitions:
    PLAN: [VALIDATE, FAIL]
    VALIDATE: [EXECUTE, ASK_USER, FAIL]
    EXECUTE: [EVALUATE, FAIL]
    EVALUATE: [EXECUTE, COMPLETE, ASK_USER, FAIL]
  duplicate_call_threshold:
    semantic_similarity: 0.95
    arg_diff_tolerance: 0.02

This pattern matters when APIs behave badly but correctly. Consider a ticketing agent against Jira Cloud and ServiceNow. If search returns zero incidents because indexing lags by 3 to 8 seconds after write, an unconstrained agent may create duplicates. A bounded state machine can instead mark the result as uncertain_write_visibility, wait once, recheck once, then stop.

Add budgets that the model can see

Do not keep limits hidden in orchestration only. Pass the remaining budget into the model context on every step. Models make better decisions when they know they have 2 steps left and 3 seconds remaining.

A useful execution envelope looks like this:

{
  "task_id": "tsk_9f31",
  "remaining_steps": 3,
  "remaining_time_ms": 4200,
  "tool_call_counts": {
    "search_ticket": 2,
    "create_ticket": 1
  },
  "retry_policy": {
    "max_retries_per_tool": 1,
    "backoff_ms": 800
  }
}

In practice, exposing budgets reduces waste. On a customer ops workflow using OpenAI-compatible function calling with Temporal orchestration, visible budgets cut average tool calls per resolved task from 5.8 to 4.3. That translated into a 19% lower inference bill and fewer 429 cascades on downstream APIs.

Idempotent tools turn retries from dangerous to boring

If your tools are not idempotent, every retry is a gamble. You are asking the agent to operate in a distributed system while pretending duplicate delivery does not exist.

Real APIs fail in ambiguous ways all the time. The classic case is a POST that succeeds server-side but the client times out at 10 seconds. The agent sees no confirmation and retries. Without idempotency, you now have two orders, two tickets, or two user accounts.

Design every write with an idempotency key

For any state-changing tool, require a caller-supplied idempotency key tied to the business operation, not the network attempt. Good keys are stable across retries and unique across distinct intents.

Examples:

• create_invoice:{tenant_id}:{source_doc_id}
• provision_user:{tenant_id}:{hr_event_id}
• open_incident:{service}:{alert_fingerprint}:{day_bucket}

Here is a minimal tool contract:

POST /v1/incidents
Idempotency-Key: open_incident:payments:fp_7b21:2026-08-10
Content-Type: application/json

{
  "service": "payments",
  "severity": "high",
  "summary": "Checkout 5xx rate exceeded 4% for 10m",
  "source": "agent"
}

And the server behavior should be strict:

• First request with a new key creates the resource and stores the response hash
• Retry with the same key and same body returns the original result
• Retry with the same key and different body returns 409 Conflict

That last rule prevents silent corruption. It also gives the agent a clear failure mode it can surface.

Prefer upsert and compare-and-set over blind POST

When the business operation maps to a natural key, PUT or UPSERT is often safer than POST. For mutable records, add optimistic concurrency with ETags or version numbers.

A CRM enrichment agent is a good example. If it updates account tier based on firmographic data, use compare-and-set semantics:

curl -X PATCH "https://api.example-crm.com/accounts/acct_1842" \
  -H "If-Match: \"v17\"" \
  -H "Content-Type: application/json" \
  -d '{"tier":"enterprise","employee_count":1820}'

If another system updated the account to v18, the API should return 412 Precondition Failed. That is not an inconvenience. It is a clean signal to re-read state before writing again.

In production, these patterns materially improve outcomes. On a user-provisioning flow spanning Workday, Okta, and Microsoft Graph, adding idempotency keys and version checks reduced duplicate account creation from 0.7% of events to under 0.02%. At 50,000 monthly lifecycle events, that avoided dozens of manual cleanup hours.

Make failure explicit so recovery is possible

Many agent stacks still treat failure as a string in the transcript. That is too vague for operations and too weak for automation.

A robust agent should return structured failure with enough detail for the next system, the human reviewer, or the retry scheduler to act. At minimum, every tool result should be one of:

• success
• retryable_failure
• non_retryable_failure
• needs_human_input
• partial_success

Standardize failure envelopes

Here is a practical schema that works across internal tools and third-party APIs:

{
  "status": "retryable_failure",
  "error_code": "RATE_LIMITED",
  "http_status": 429,
  "message": "Zendesk API rate limit exceeded",
  "retry_after_ms": 1500,
  "safe_to_retry": true,
  "idempotency_key": "open_ticket:tenant42:case_9912",
  "observed_at": "2026-08-10T09:14:22Z",
  "provider": "zendesk",
  "provider_request_id": "zd-req-71ab2"
}

With this envelope, your orchestrator can make deterministic choices. Retry once after retry_after_ms. Escalate if the task budget is exhausted. Route to a human if the API says 422 because a required field is missing from the source system.

Treat partial success as its own state

Partial success is common in multi-step workflows. A finance agent may create a supplier in NetSuite, then fail to attach tax documents in SharePoint because of a permissions issue. If you collapse that into generic failure, the next run may create the supplier again.

Instead, persist step-level outcomes:

• supplier_created=true
• documents_uploaded=false
• notification_sent=false

Then resume from the first incomplete idempotent step. Teams using durable workflow engines like Temporal, Azure Durable Functions, and AWS Step Functions do this well because the workflow state is explicit and replay-safe.

A real benchmark from a document onboarding pipeline: after moving from transcript-only failure handling to explicit step state, successful resume after interruption rose from 63% to 96%, and mean operator intervention per 1,000 runs dropped from 74 cases to 11.

Reference architecture: agent planner, deterministic executor

The architecture that holds up best in enterprise environments is not fully autonomous. It splits responsibilities.

• The model plans, selects tools, and interprets outcomes
• The executor enforces budgets, auth, schemas, retries, and idempotency
• The workflow engine persists state and resumes safely
• The observability layer records every tool call with correlation IDs

This keeps the model flexible without letting it become your transaction coordinator.

[User/Event]
    |
    v
[Agent Planner]
    |  proposes tool call
    v
[Deterministic Executor] ---> [Policy Guardrails]
    |            |                  |
    |            |                  +--> step limits / PII / auth scopes
    |            v
    |      [Tool Adapter Layer] ---> CRM / ERP / ITSM / Billing APIs
    |
    v
[Workflow State Store] <----> [Retry Scheduler]
    |
    v
[Telemetry: traces, logs, cost, request IDs]

Keep tool adapters thin and opinionated

A common mistake is exposing raw third-party APIs directly to the model. Do not give the agent 37 optional parameters from a billing API when the business operation is simply create_refund.

Wrap external APIs in tool adapters that:

• validate inputs with strict schemas
• inject auth and tenant context
• add idempotency keys automatically where possible
• normalize provider-specific errors into your failure envelope
• redact secrets and PII from logs

For example, a Stripe refund adapter can hide low-level fields and expose one safe operation. Teams that moved from raw API exposure to normalized adapters often see tool selection accuracy improve by 10 to 18 percentage points because the action surface is smaller and cleaner.

Common Pitfalls

Mistaking retries for resilience

If you retry a non-idempotent tool, you are not adding resilience. You are multiplying side effects. Fix the tool contract first, then add retries.

Letting the model decide backoff policy

Backoff belongs in the executor, not the prompt. A model should not invent sleep(17) because an API looked unhappy. Use deterministic backoff with jitter and provider-specific caps.

Hiding ambiguity from downstream systems

If a write may have succeeded but confirmation is missing, do not return success=false and move on. Return partial_success or retryable_failure with the idempotency key and provider request ID. That is what allows safe reconciliation later.

Using transcripts as the source of truth

Conversation history is not workflow state. If the process matters, persist structured state outside the model context. Otherwise, long-running tasks fail on context truncation, replay drift, or prompt changes.

Exposing raw search to create flows

A common anti-pattern is search -> if none -> create against eventually consistent systems. This causes duplicates under indexing lag. Prefer create with idempotency key, then verify by natural key or returned resource ID.

Key Takeaways

• Set hard execution budgets now: max_steps, repeated-call limits, and wall-clock timeouts stop expensive loops before they become incidents.
• Make every write tool idempotent with stable business-level keys; reject same-key different-body retries with 409 Conflict.
• Return structured failure types, not vague text, so your orchestrator can retry, resume, escalate, or reconcile deterministically.
• Persist step-level workflow state outside the transcript; treat partial success as normal in multi-system automation.
• Put a deterministic executor between the model and your APIs to enforce schemas, auth, retries, and observability.
• Start with one high-value workflow such as ticket creation, user provisioning, or invoice generation, then measure duplicate rate, mean tool calls, and resume success within a week.

The pattern is simple: let the model think, but make the system remember, constrain, and verify. That is how agent architectures survive contact with a real API.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.