9. AI Agents, Orchestration, and Durable Workflows

Understand agents versus workflows, planning, memory, delegation, state, and stopping conditions.

Agents vs Deterministic Workflows

An agent uses a model to choose actions or next steps based on context. A workflow follows a predefined graph or state machine. Agents can adapt to ambiguous tasks but are harder to predict, evaluate, and budget. Workflows are usually preferable for compliance-sensitive processes, payment flows, and repeatable business operations.

A hybrid design is common: deterministic orchestration controls stages and permissions, while a model handles bounded subproblems such as extracting fields, selecting among allowed routes, or drafting a message. Define explicit stop conditions, maximum steps, timeouts, and escalation paths.

Hybrid workflow

1. Validate request (deterministic)
2. Retrieve relevant context (deterministic)
3. Model proposes a plan (bounded)
4. Validate plan against policy (deterministic)
5. Execute approved steps (deterministic)
6. Verify outcome and report

💡 This pattern keeps flexible reasoning inside explicit boundaries. Each step can be tested, logged, retried, and audited independently.

Key Takeaways

Choose agents only when adaptivity is worth the added complexity.
Bound every model-controlled loop.
Keep critical policy decisions and side effects under deterministic control.

Knowledge Check

Which design is usually easiest to audit for a fixed regulated process?

Agent State, Memory, and Planning

Agent state is the structured record of what has happened and what remains. Short-term state may include current plan, tool results, and unresolved tasks. Long-term memory may store user preferences or durable facts, but should have consent, provenance, expiration, and deletion controls. Do not write every conversation detail into permanent memory.

Planning can be useful for multi-step tasks, but plans are hypotheses rather than proof. Validate each step, re-check current facts before irreversible actions, and detect loops. Memory retrieval should be scoped to the current user and purpose, and sensitive information needs special handling.

Typed task state

const taskState = {
  goal: "Prepare a course outline",
  steps: [
    { id: "research", status: "completed" },
    { id: "draft", status: "in_progress" },
    { id: "review", status: "pending" }
  ],
  attempts: 1,
  maxAttempts: 3
};

💡 Explicit state makes retries and resumability visible. In production, persist state durably, validate transitions, and prevent the model from editing privileged fields such as user identity or approval status.

Key Takeaways

Separate short-term state from long-term memory.
Track provenance and deletion requirements for memory.
Treat model-generated plans as proposals to validate.

Knowledge Check

Why should agent state be structured and validated?

Durability, Retries, and Human-in-the-Loop

Long-running workflows can outlive a single HTTP request or process. Use a durable job/workflow system when tasks may suspend, retry, wait for approval, or resume after deployment. Persist step status and idempotency keys; make each step safe to retry or compensate where possible.

Retries must be selective. Retry transient network errors, rate limits, and temporary service failures with exponential backoff and jitter, respecting provider retry guidance. Do not blindly retry invalid input, policy denials, or actions that may have already succeeded. Escalate ambiguous outcomes for side-effecting operations.

Retry with bounded exponential backoff

async function retry(operation, attempts = 4) {
  let lastError;
  for (let i = 0; i < attempts; i++) {
    try { return await operation(); }
    catch (error) {
      lastError = error;
      if (!isRetryable(error) || i === attempts - 1) throw error;
      const delay = Math.min(500 * 2 ** i, 8000) + Math.random() * 250;
      await new Promise(resolve => setTimeout(resolve, delay));
    }
  }
  throw lastError;
}

💡 isRetryable must classify errors carefully. A retry policy should also honor Retry-After headers where applicable, cap total elapsed time, and avoid repeating non-idempotent side effects without deduplication.

Key Takeaways

Persist long-running state durably.
Retry only errors classified as transient.
Use human approval for sensitive or ambiguous actions.

Knowledge Check

Which failure should generally not be retried automatically without changing the request?

Agents vs Deterministic Workflows

Agent State, Memory, and Planning

Durability, Retries, and Human-in-the-Loop

STAY UPDATED WITH BYTES!