AX: Google's Open Agentic Orchestrator Explained — Building Production AI Agent Workflows

Every framework promising to “orchestrate agents” eventually collapses into the same three problems: state management across long-running tasks, tool-call reliability, and observability when something silently breaks at 3am. AX — Google’s open agentic orchestrator — is the first framework in this space that treats those three problems as first-class citizens instead of afterthoughts bolted onto a prompt-chaining library. If you’ve shipped anything with LangChain or a homegrown agent loop, you already know why that matters.

This is a practical breakdown of what AX actually does, how it’s architected, and how to wire it into a real Node.js/TypeScript backend with Docker for deployment.

What AX Actually Is

AX is an open-source orchestration layer for AI agents that sits between your LLM calls and your application logic. Unlike prompt-chaining libraries that treat agents as a sequence of function calls, AX models agents as stateful graphs with explicit transitions, retries, and checkpoints.

The core primitives:

• Executors — units of work that can be an LLM call, a tool invocation, or a sub-agent
• Graphs — directed workflows connecting executors with conditional edges
• Checkpoints — durable state snapshots so a crashed workflow resumes instead of restarting
• Policies — retry/backoff/timeout rules attached per-node, not globally

This is closer to a workflow engine (think Temporal or AWS Step Functions) than a chatbot SDK — which is exactly the gap it fills. Most agent frameworks are great for demos and terrible for anything that needs to survive a network blip or a rate-limited API three steps into a 20-step task.

Why This Matters for Backend Engineers

If you’re building agentic features into a product — not a research notebook — you care about things AX explicitly designs for:

• Idempotency — re-running a node shouldn’t duplicate side effects
• Observability — every state transition is logged and traceable
• Horizontal scaling — the orchestrator itself is stateless; state lives in a pluggable store (Redis, Postgres, or GCS)
• Language-agnostic executors — nodes can be HTTP calls to services written in Go, Node.js, or Python

That last point is the sleeper feature. You don’t need to rewrite your Go microservices in Python to make them agent-callable — AX treats any HTTP/gRPC endpoint as a valid executor.

Installing and Bootstrapping AX

npm install @google/ax-orchestrator
npm install @google/ax-store-redis

A minimal agent graph in TypeScript:

import { Graph, Executor, RetryPolicy } from '@google/ax-orchestrator';
import { RedisStateStore } from '@google/ax-store-redis';

const store = new RedisStateStore({
  url: process.env.REDIS_URL!,
});

const fetchData: Executor = {
  id: 'fetch-user-data',
  type: 'tool',
  run: async (ctx) => {
    const res = await fetch(`https://api.internal/users/${ctx.input.userId}`);
    if (!res.ok) throw new Error(`Upstream failed: ${res.status}`);
    return res.json();
  },
  policy: RetryPolicy.exponentialBackoff({ maxAttempts: 3, baseMs: 500 }),
};

const summarize: Executor = {
  id: 'summarize-with-llm',
  type: 'llm',
  model: 'gemini-2.5-pro',
  prompt: (ctx) => `Summarize this user profile in 2 sentences:\n${JSON.stringify(ctx.state.fetchData)}`,
};

const graph = new Graph({ store })
  .addNode(fetchData)
  .addNode(summarize)
  .connect('fetch-user-data', 'summarize-with-llm');

export async function runAgent(userId: string) {
  const run = await graph.start({ input: { userId } });
  return run.result;
}

Notice there’s no manual try/catch retry loop, no ad-hoc setTimeout backoff — that’s the whole point. The policy is declared once, attached to the node, and the orchestrator enforces it.

Architecture: How AX Differs from LangChain/CrewAI

Feature	AX	LangChain	CrewAI
State persistence	Durable, pluggable store	In-memory by default	In-memory by default
Crash recovery	Resumes from checkpoint	Restarts from scratch	Restarts from scratch
Retry policies	Per-node, declarative	Manual wrapping	Limited
Language interop	Any HTTP/gRPC service	Python/JS SDKs only	Python only
Observability	Built-in trace export (OpenTelemetry)	Requires LangSmith	Minimal
Deployment model	Stateless orchestrator + external store	Embedded in app process	Embedded in app process
Best fit	Production multi-step workflows	Prototyping, RAG pipelines	Role-based agent teams

The architectural bet AX makes is that agents are workflows, not chat sessions. That reframing is why it plugs so cleanly into existing DevOps tooling — checkpoints are just rows in Postgres, traces are just OTel spans, and scaling is just adding more orchestrator replicas behind a load balancer.

Building a Multi-Agent Workflow

Real-world use cases rarely involve a single LLM call. Here’s a research-and-report pipeline with conditional branching:

import { Graph, Executor, Condition } from '@google/ax-orchestrator';

const searchWeb: Executor = {
  id: 'search-web',
  type: 'tool',
  run: async (ctx) => searchAPI(ctx.input.query),
};

const validateResults: Executor = {
  id: 'validate-results',
  type: 'llm',
  model: 'gemini-2.5-flash',
  prompt: (ctx) => `Are these search results relevant to "${ctx.input.query}"? Answer YES or NO.\n${JSON.stringify(ctx.state.searchWeb)}`,
};

const draftReport: Executor = {
  id: 'draft-report',
  type: 'llm',
  model: 'gemini-2.5-pro',
  prompt: (ctx) => `Write a report using:\n${JSON.stringify(ctx.state.searchWeb)}`,
};

const refineQuery: Executor = {
  id: 'refine-query',
  type: 'llm',
  model: 'gemini-2.5-flash',
  prompt: (ctx) => `Rewrite this search query to be more specific: "${ctx.input.query}"`,
};

const graph = new Graph()
  .addNode(searchWeb)
  .addNode(validateResults)
  .addNode(draftReport)
  .addNode(refineQuery)
  .connect('search-web', 'validate-results')
  .connectConditional('validate-results', {
    onTrue: 'draft-report',
    onFalse: 'refine-query',
  })
  .connect('refine-query', 'search-web'); // loop back

This loop-back edge is where most naive agent implementations fall apart — infinite loops without a cap. AX solves it with built-in cycle guards:

const graph = new Graph({
  maxCycles: 5,
  onCycleLimitExceeded: 'fail-with-partial-result',
});

Observability and Debugging

AX ships OpenTelemetry instrumentation out of the box. Every node execution emits a span with input/output/token-usage metadata:

import { AXTracer } from '@google/ax-orchestrator';

const tracer = new AXTracer({
  exporter: 'otlp',
  endpoint: process.env.OTEL_COLLECTOR_URL,
});

graph.attachTracer(tracer);

Pipe that into Grafana/Tempo and you get a flame graph of your agent’s decision tree — which node retried, how many tokens each LLM call burned, and where latency actually lives. This is the difference between debugging an agent by re-reading chat transcripts versus debugging it like you’d debug a distributed system, because that’s what it is.

Dockerizing an AX-Based Service

Because the orchestrator is stateless, containerizing it is straightforward:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/server.js"]

docker-compose.yml for local development with Redis as the state store:

version: "3.9"
services:
  ax-orchestrator:
    build: .
    ports:
      - "3000:3000"
    environment:
      - REDIS_URL=redis://redis:6379
      - OTEL_COLLECTOR_URL=http://otel-collector:4318
    depends_on:
      - redis
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
  otel-collector:
    image: otel/opentelemetry-collector:latest
    ports:
      - "4318:4318"

Scale horizontally by just increasing replicas — since state lives in Redis/Postgres, any orchestrator instance can pick up a checkpointed run.

docker compose up --scale ax-orchestrator=3

Handling Failures Gracefully

The pattern that separates toy agents from production agents is explicit failure handling per node:

const paymentExecutor: Executor = {
  id: 'process-payment',
  type: 'tool',
  run: async (ctx) => chargeCard(ctx.state.amount, ctx.state.cardToken),
  policy: {
    retry: { maxAttempts: 2, baseMs: 1000 },
    onFailure: 'compensate',
    compensate: async (ctx) => {
      await refundIfCharged(ctx.state.transactionId);
    },
  },
};

That compensate hook is essentially the Saga pattern from distributed transactions, applied to agent workflows. If you’ve built payment or booking systems, this should feel immediately familiar — because it’s solving the same problem.

When Not to Use AX

AX is overkill for:

• Single-turn chatbot responses
• Simple RAG lookups with no multi-step reasoning
• Prototypes where you’re still iterating on prompts hourly

It earns its complexity when you have:

• Multi-step workflows with real side effects (payments, emails, database writes)
• SLA requirements around retries and failure recovery
• Need for audit trails on what the agent did and why

Comparing State Store Options

Store	Best for	Tradeoff
Redis	Low-latency, ephemeral workflows	No long-term durability guarantees without persistence config
Postgres	Audit trails, long-running workflows (days)	Higher write latency per checkpoint
GCS/S3	Very large state payloads (large documents, embeddings)	Higher latency, not ideal for high-frequency checkpoints

Pick based on workflow duration and payload size — don’t default to Postgres for a workflow that completes in under 10 seconds; Redis will save you real latency.

Key Takeaways

• AX treats AI agents as durable, resumable workflow graphs — not chat sessions — which is the right abstraction for production systems.
• Per-node retry policies and cycle guards eliminate the manual error-handling boilerplate that plagues homegrown agent loops.
• The orchestrator is stateless by design; state lives in Redis, Postgres, or GCS, making horizontal scaling trivial with Docker Compose or Kubernetes.
• Built-in OpenTelemetry support gives you real distributed tracing for agent decision paths, not just chat transcripts.
• Language-agnostic executors mean your existing Go or Node.js microservices become agent-callable without rewrites.
• The Saga-style compensate hook makes AX suitable for workflows with real side effects like payments or bookings.
• Don’t reach for AX on single-turn chatbot use cases — it’s built for multi-step, stateful, failure-prone workflows.
• Choosing the right state store (Redis vs Postgres vs object storage) matters more for performance than picking the “best” LLM model in the graph.