Building an Agentic Orchestrator in Node.js: A Practical Guide to Multi-Agent Workflows

Every AI agent demo looks the same: a single LLM call wrapped in a while loop, calling tools until it decides it’s done. That works for a hackathon. It falls apart the moment you need retries, parallel execution, state persistence, or more than one agent cooperating on a task. Google’s AX orchestrator launch this week is a good excuse to talk about what an actual production-grade agentic orchestrator looks like — and how to build a lightweight version yourself in Node.js without buying into a heavyweight framework.

This post walks through the architecture of an agentic orchestrator, why naive agent loops break in production, and a working TypeScript implementation you can extend.

Why Agent Loops Aren’t Orchestration

A basic agent loop looks like this:

async function runAgent(prompt) {
  let messages = [{ role: "user", content: prompt }];

  while (true) {
    const response = await llm.chat(messages);
    if (response.toolCalls.length === 0) return response.content;

    for (const call of response.toolCalls) {
      const result = await executeTool(call);
      messages.push({ role: "tool", content: result });
    }
  }
}

This is fine for one agent, one task, no failures. Real orchestration needs to answer questions this loop doesn’t:

• What happens when a tool call times out or throws?
• How do you run three sub-agents in parallel and merge their outputs?
• How do you resume a workflow after a crash instead of restarting from scratch?
• How do you cap cost/token usage per task without hardcoding limits everywhere?
• How do you observe what the agent actually did, after the fact?

An orchestrator is the layer that answers these. It treats agent execution as a workflow graph, not a chat loop.

Core Architecture of an Agentic Orchestrator

A minimal but production-viable orchestrator has four components:

Component	Responsibility	Analogous To
Task Queue	Schedules units of work, handles retries/backoff	BullMQ, Temporal
Agent Executor	Runs a single agent step (LLM call + tool execution)	Worker process
State Store	Persists conversation/task state between steps	Redis, Postgres
Router	Decides which agent/tool handles the next step	Control plane

The key insight: each agent step should be idempotent and resumable. If your process dies mid-execution, you replay from the last persisted checkpoint, not from zero.

Setting Up the Project

mkdir agent-orchestrator && cd agent-orchestrator
npm init -y
npm install bullmq ioredis zod openai
npm install -D typescript tsx @types/node
npx tsc --init

tsconfig.json essentials:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "outDir": "dist"
  }
}

Defining the Task Contract

Everything in the orchestrator revolves around a strongly-typed task shape. Use zod so runtime validation and TypeScript types come from the same source.

// task.ts
import { z } from "zod";

export const TaskSchema = z.object({
  id: z.string(),
  type: z.enum(["research", "codegen", "review", "summarize"]),
  input: z.record(z.any()),
  parentId: z.string().nullable(),
  status: z.enum(["pending", "running", "completed", "failed"]),
  attempts: z.number().default(0),
  maxAttempts: z.number().default(3),
  result: z.any().nullable(),
});

export type Task = z.infer<typeof TaskSchema>;

Every agent step consumes a Task and produces either a result or spawns child tasks (sub-agents).

Building the Queue Layer

BullMQ gives you retries, backoff, and concurrency control for free — don’t reinvent this with setInterval polling.

// queue.ts
import { Queue, Worker, QueueEvents } from "bullmq";
import IORedis from "ioredis";

const connection = new IORedis(process.env.REDIS_URL ?? "redis://localhost:6379", {
  maxRetriesPerRequest: null,
});

export const agentQueue = new Queue("agent-tasks", { connection });

export const queueEvents = new QueueEvents("agent-tasks", { connection });

export function createWorker(processor: (job: any) => Promise<any>) {
  return new Worker("agent-tasks", processor, {
    connection,
    concurrency: 5,
    limiter: { max: 20, duration: 1000 }, // rate limit LLM calls
  });
}

The Agent Executor

This is where the actual LLM + tool logic lives. Notice it’s a pure function of Task in, result out — no hidden state.

// executor.ts
import OpenAI from "openai";
import { Task } from "./task.js";
import { toolRegistry } from "./tools.js";

const client = new OpenAI();

export async function executeAgentStep(task: Task): Promise<{
  result?: any;
  spawnedTasks?: Partial<Task>[];
}> {
  const systemPrompt = getPromptForType(task.type);

  const response = await client.chat.completions.create({
    model: "gpt-4.1-mini",
    messages: [
      { role: "system", content: systemPrompt },
      { role: "user", content: JSON.stringify(task.input) },
    ],
    tools: toolRegistry.getSchemas(),
    tool_choice: "auto",
  });

  const choice = response.choices[0];
  const toolCalls = choice.message.tool_calls ?? [];

  if (toolCalls.length === 0) {
    return { result: choice.message.content };
  }

  const spawnedTasks = toolCalls.map((call) => ({
    type: mapToolToTaskType(call.function.name),
    input: JSON.parse(call.function.arguments),
    parentId: task.id,
  }));

  return { spawnedTasks };
}

function getPromptForType(type: Task["type"]): string {
  const prompts: Record<Task["type"], string> = {
    research: "You gather and summarize information. Be concise.",
    codegen: "You write production-quality code. No explanations, code only.",
    review: "You review code for bugs, security issues, and style.",
    summarize: "You compress input into 3 bullet points max.",
  };
  return prompts[type];
}

Wiring the Worker to the State Store

The worker pulls tasks, executes them, persists results, and enqueues children. This is the loop that replaces the naive while(true).

// worker.ts
import { createWorker, agentQueue } from "./queue.js";
import { executeAgentStep } from "./executor.js";
import { saveTaskResult, createTask } from "./store.js";
import { Task } from "./task.js";

createWorker(async (job) => {
  const task: Task = job.data;

  try {
    const { result, spawnedTasks } = await executeAgentStep(task);

    if (result) {
      await saveTaskResult(task.id, "completed", result);
      return result;
    }

    if (spawnedTasks?.length) {
      const children = await Promise.all(
        spawnedTasks.map((t) => createTask(t))
      );

      await Promise.all(
        children.map((child) => agentQueue.add("step", child, { jobId: child.id }))
      );

      await saveTaskResult(task.id, "running", { childIds: children.map((c) => c.id) });
    }
  } catch (err) {
    if (task.attempts + 1 >= task.maxAttempts) {
      await saveTaskResult(task.id, "failed", { error: String(err) });
    }
    throw err; // let BullMQ retry with backoff
  }
});

Handling Parallel Sub-Agents

The real value of an orchestrator over a single agent loop is fan-out/fan-in: spawning multiple sub-agents concurrently and joining their results.

// aggregator.ts
import { agentQueue, queueEvents } from "./queue.js";

export async function waitForChildren(childIds: string[]): Promise<any[]> {
  return Promise.all(
    childIds.map((id) =>
      new Promise((resolve, reject) => {
        queueEvents.on("completed", ({ jobId, returnvalue }) => {
          if (jobId === id) resolve(returnvalue);
        });
        queueEvents.on("failed", ({ jobId, failedReason }) => {
          if (jobId === id) reject(new Error(failedReason));
        });
      })
    )
  );
}

A parent task that spawns three “research” sub-agents can join them once all resolve, then feed the merged result into a “summarize” task — a real DAG, not a linear chain.

Comparing Orchestration Strategies

Strategy	Best For	Failure Recovery	Complexity
Single agent loop	Quick prototypes, single-tool tasks	None — restart from scratch	Low
Queue-based orchestrator (this post)	Multi-step workflows, parallel sub-agents	Automatic retry via job queue	Medium
State machine (XState/Temporal)	Long-running workflows with strict transitions	Full replay from event log	High
Graph frameworks (LangGraph, AX)	Complex conditional branching between agents	Built-in checkpointing	Medium-High

For most Node.js teams, the queue-based approach hits the sweet spot: you get retries and observability from BullMQ without adopting a full workflow engine like Temporal, and you keep total control over the agent logic instead of inheriting a framework’s abstractions.

Observability: You Can’t Debug What You Can’t See

Agent orchestration without logging is a black box. Emit structured events at every transition:

// telemetry.ts
export function logTaskEvent(task: Task, event: string, meta: Record<string, any> = {}) {
  console.log(JSON.stringify({
    timestamp: new Date().toISOString(),
    taskId: task.id,
    type: task.type,
    event,
    ...meta,
  }));
}

Pipe this into whatever you already use — Loki, Datadog, or a plain Postgres table. At minimum, track: task started, tool called, tool result, task completed/failed, retry count. When an agent does something wrong three levels deep in a sub-agent chain, this is the only way you’ll find it.

Cost and Token Guardrails

LLM calls inside a loop can spiral fast. Bake limits into the task schema itself:

const MAX_TOKENS_PER_WORKFLOW = 50_000;

async function checkBudget(parentId: string): Promise<void> {
  const usage = await getTokenUsageForWorkflow(parentId);
  if (usage >= MAX_TOKENS_PER_WORKFLOW) {
    throw new Error(`Token budget exceeded for workflow ${parentId}`);
  }
}

Call this before every executeAgentStep. It’s the difference between a bounded workflow and a runaway bill.

Dockerizing the Orchestrator

Ship the worker and Redis together for local dev and staging:

FROM node:20-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY dist ./dist
CMD ["node", "dist/worker.js"]

# docker-compose.yml
services:
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]

  worker:
    build: .
    environment:
      - REDIS_URL=redis://redis:6379
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    depends_on: [redis]
    deploy:
      replicas: 3

Scale replicas to increase concurrent agent throughput. BullMQ’s rate limiter prevents you from blowing past your LLM provider’s rate limits even with multiple workers.

When You Shouldn’t Build This Yourself

If your workflows involve strict human-in-the-loop approval steps, long-running processes spanning days, or complex conditional branching with dozens of states, reach for Temporal or a managed offering like AX instead. The queue-based pattern above is ideal for workflows measured in seconds to minutes with a handful of steps — past that, you’re better off with a purpose-built workflow engine that handles versioning and long-term durability for you.

Key Takeaways

• A single agent loop (while + tool calls) is not an orchestrator — it has no retry logic, no parallelism, and no crash recovery.
• Model tasks as a typed, persisted schema (zod + a state store) so every step is resumable independently of process lifetime.
• Use a battle-tested queue (BullMQ) instead of hand-rolling retries, backoff, and concurrency control.
• Fan-out/fan-in patterns — spawning parallel sub-agents and joining results — are where orchestration actually earns its complexity budget over a simple loop.
• Enforce token/cost budgets at the task level, not just at the API client level, or multi-agent workflows will silently balloon costs.
• Structured logging at every task transition is non-negotiable once you have more than one agent cooperating.
• Docker + Redis gives you a horizontally scalable worker pool with minimal ops overhead.
• Reach for Temporal, XState, or managed platforms like Google’s AX only when workflows need long-running durability or complex branching — don’t over-engineer simple pipelines.