我们如何构建我们的多智能体研究系统

Claude now has Research capabilities that allow it to search across the web, Google Workspace, and any integrations to accomplish complex tasks.

The journey of this multi-agent system from prototype to production taught us critical lessons about system architecture, tool design, and prompt engineering.

A multi-agent system consists of multiple agents (LLMs autonomously using tools in a loop) working together.

Our Research feature involves an agent that plans a research process based on user queries, and then uses tools to create parallel agents that search for information simultaneously.

Systems with multiple agents introduce new challenges in agent coordination, evaluation, and reliability.

This post breaks down the principles that worked for us—we hope you'll find them useful to apply when building your own multi-agent systems.

Research work involves open-ended problems where it’s very difficult to predict the required steps in advance.

You can’t hardcode a fixed path for exploring complex topics, as the process is inherently dynamic and path-dependent.

When people conduct research, they tend to continuously update their approach based on discoveries, following leads that emerge during investigation.

This unpredictability makes AI agents particularly well-suited for research tasks.

Research demands the flexibility to pivot or explore tangential connections as the investigation unfolds.

The model must operate autonomously for many turns, making decisions about which directions to pursue based on intermediate findings.

A linear, one-shot pipeline cannot handle these tasks.

The essence of search is compression: distilling insights from a vast corpus.

Subagents facilitate compression by operating in parallel with their own context windows, exploring different aspects of the question simultaneously before condensing the most important tokens for the lead research agent.

Each subagent also provides separation of concerns—distinct tools, prompts, and exploration trajectories—which reduces path dependency and enables thorough, independent investigations.

Once intelligence reaches a threshold, multi-agent systems become a vital way to scale performance.

For instance, although individual humans have become more intelligent in the last 100,000 years, human societies have become exponentially more capable in the information age because of our collective intelligence and ability to coordinate.

Even generally-intelligent agents face limits when operating as individuals; groups of agents can accomplish far more.

Our internal evaluations show that multi-agent research systems excel especially for breadth-first queries that involve pursuing multiple independent directions simultaneously.

We found that a multi-agent system with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval.

For example, when asked to identify all the board members of the companies in the Information Technology S&P 500, the multi-agent system found the correct answers by decomposing this into tasks for subagents, while the single agent system failed to find the answer with slow, sequential searches.

Multi-agent systems work mainly because they help spend enough tokens to solve the problem.

In our analysis, three factors explained 95% of the performance variance in the BrowseComp evaluation (which tests the ability of browsing agents to locate hard-to-find information).

We found that token usage by itself explains 80% of the variance, with the number of tool calls and the model choice as the two other explanatory factors.

This finding validates our architecture that distributes work across agents with separate context windows to add more capacity for parallel reasoning.

The latest Claude models act as large efficiency multipliers on token use, as upgrading to Claude Sonnet 4 is a larger performance gain than doubling the token budget on Claude Sonnet 3.7.

Multi-agent architectures effectively scale token usage for tasks that exceed the limits of single agents.

There is a downside: in practice, these architectures burn through tokens fast.

In our data, agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.

For economic viability, multi-agent systems require tasks where the value of the task is high enough to pay for the increased performance.

Further, some domains that require all agents to share the same context or involve many dependencies between agents are not a good fit for multi-agent systems today.

For instance, most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time.

We’ve found that multi-agent systems excel at valuable tasks that involve heavy parallelization, information that exceeds single context windows, and interfacing with numerous complex tools.

Our Research system uses a multi-agent architecture with an orchestrator-worker pattern, where a lead agent coordinates the process while delegating to specialized subagents that operate in parallel.

When a user submits a query, the lead agent analyzes it, develops a strategy, and spawns subagents to explore different aspects simultaneously.

As shown in the diagram above, the subagents act as intelligent filters by iteratively using search tools to gather information, in this case on AI agent companies in 2025, and then returning a list of companies to the lead agent so it can compile a final answer.

Traditional approaches using Retrieval Augmented Generation (RAG) use static retrieval.

That is, they fetch some set of chunks that are most similar to an input query and use these chunks to generate a response.

In contrast, our architecture uses a multi-step search that dynamically finds relevant information, adapts to new findings, and analyzes results to formulate high-quality answers.

Multi-agent systems have key differences from single-agent systems, including a rapid growth in coordination complexity.

Early agents made errors like spawning 50 subagents for simple queries, scouring the web endlessly for nonexistent sources, and distracting each other with excessive updates.

Since each agent is steered by a prompt, prompt engineering was our primary lever for improving these behaviors.

Below are some principles we learned for prompting agents:

Our prompting strategy focuses on instilling good heuristics rather than rigid rules.

We studied how skilled humans approach research tasks and encoded these strategies in our prompts—strategies like decomposing difficult questions into smaller tasks, carefully evaluating the quality of sources, adjusting search approaches based on new information, and recognizing when to focus on depth (investigating one topic in detail) vs. breadth (exploring many topics in parallel).

We also proactively mitigated unintended side effects by setting explicit guardrails to prevent the agents from spiraling out of control.

Finally, we focused on a fast iteration loop with observability and test cases.

Good evaluations are essential for building reliable AI applications, and agents are no different.

However, evaluating multi-agent systems presents unique challenges.

Traditional evaluations often assume that the AI follows the same steps each time: given input X, the system should follow path Y to produce output Z.

But multi-agent systems don't work this way.

Even with identical starting points, agents might take completely different valid paths to reach their goal.

One agent might search three sources while another searches ten, or they might use different tools to find the same answer.

Because we don’t always know what the right steps are, we usually can't just check if agents followed the “correct” steps we prescribed in advance.

Instead, we need flexible evaluation methods that judge whether agents achieved the right outcomes while also following a reasonable process.

Start evaluating immediately with small samples.

In early agent development, changes tend to have dramatic impacts because there is abundant low-hanging fruit.

A prompt tweak might boost success rates from 30% to 80%.

With effect sizes this large, you can spot changes with just a few test cases.

We started with a set of about 20 queries representing real usage patterns.

Testing these queries often allowed us to clearly see the impact of changes.

We often hear that AI developer teams delay creating evals because they believe that only large evals with hundreds of test cases are useful.

However, it’s best to start with small-scale testing right away with a few examples, rather than delaying until you can build more thorough evals.

LLM-as-judge evaluation scales when done well.

Research outputs are difficult to evaluate programmatically, since they are free-form text and rarely have a single correct answer.

LLMs are a natural fit for grading outputs.

We used an LLM judge that evaluated each output against criteria in a rubric: factual accuracy (do claims match sources?), citation accuracy (do the cited sources match the claims?), completeness (are all requested aspects covered?), source quality (did it use primary sources over lower-quality secondary sources?), and tool efficiency (did it use the right tools a reasonable number of times?).

We experimented with multiple judges to evaluate each component, but found that a single LLM call with a single prompt outputting scores from 0.0-1.0 and a pass-fail grade was the most consistent and aligned with human judgements.

This method was especially effective when the eval test cases did have a clear answer, and we could use the LLM judge to simply check if the answer was correct (i.e. did it accurately list the pharma companies with the top 3 largest R&D budgets?).

Using an LLM as a judge allowed us to scalably evaluate hundreds of outputs.

Human evaluation catches what automation misses.

People testing agents find edge cases that evals miss.

These include hallucinated answers on unusual queries, system failures, or subtle source selection biases.

In our case, human testers noticed that our early agents consistently chose SEO-optimized content farms over authoritative but less highly-ranked sources like academic PDFs or personal blogs.

Adding source quality heuristics to our prompts helped resolve this issue.

Even in a world of automated evaluations, manual testing remains essential.

Multi-agent systems have emergent behaviors, which arise without specific programming.

For instance, small changes to the lead agent can unpredictably change how subagents behave.

Success requires understanding interaction patterns, not just individual agent behavior.

Therefore, the best prompts for these agents are not just strict instructions, but frameworks for collaboration that define the division of labor, problem-solving approaches, and effort budgets.

Getting this right relies on careful prompting and tool design, solid heuristics, observability, and tight feedback loops.

See the open-source prompts in our Cookbook for example prompts from our system.

In traditional software, a bug might break a feature, degrade performance, or cause outages.

In agentic systems, minor changes cascade into large behavioral changes, which makes it remarkably difficult to write code for complex agents that must maintain state in a long-running process.

Agents are stateful and errors compound.

Agents can run for long periods of time, maintaining state across many tool calls.

This means we need to durably execute code and handle errors along the way.

Without effective mitigations, minor system failures can be catastrophic for agents.

When errors occur, we can't just restart from the beginning: restarts are expensive and frustrating for users.

Instead, we built systems that can resume from where the agent was when the errors occurred.

We also use the model’s intelligence to handle issues gracefully: for instance, letting the agent know when a tool is failing and letting it adapt works surprisingly well.

We combine the adaptability of AI agents built on Claude with deterministic safeguards like retry logic and regular checkpoints.

Debugging benefits from new approaches.

Agents make dynamic decisions and are non-deterministic between runs, even with identical prompts.

This makes debugging harder.

For instance, users would report agents “not finding obvious information,” but we couldn't see why.

Were the agents using bad search queries?

Choosing poor sources?

Hitting tool failures?

Adding full production tracing let us diagnose why agents failed and fix issues systematically.

Beyond standard observability, we monitor agent decision patterns and interaction structures—all without monitoring the contents of individual conversations, to maintain user privacy.

This high-level observability helped us diagnose root causes, discover unexpected behaviors, and fix common failures.

Deployment needs careful coordination.

Agent systems are highly stateful webs of prompts, tools, and execution logic that run almost continuously.

This means that whenever we deploy updates, agents might be anywhere in their process.

We therefore need to prevent our well-meaning code changes from breaking existing agents.

We can’t update every agent to the new version at the same time.

Instead, we use rainbow deployments to avoid disrupting running agents, by gradually shifting traffic from old to new versions while keeping both running simultaneously.

Synchronous execution creates bottlenecks.

Currently, our lead agents execute subagents synchronously, waiting for each set of subagents to complete before proceeding.

This simplifies coordination, but creates bottlenecks in the information flow between agents.

For instance, the lead agent can’t steer subagents, subagents can’t coordinate, and the entire system can be blocked while waiting for a single subagent to finish searching.

Asynchronous execution would enable additional parallelism: agents working concurrently and creating new subagents when needed.

But this asynchronicity adds challenges in result coordination, state consistency, and error propagation across the subagents.

As models can handle longer and more complex research tasks, we expect the performance gains will justify the complexity.

When building AI agents, the last mile often becomes most of the journey.

Codebases that work on developer machines require significant engineering to become reliable production systems.

The compound nature of errors in agentic systems means that minor issues for traditional software can derail agents entirely.

One step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes.

For all the reasons described in this post, the gap between prototype and production is often wider than anticipated.

Despite these challenges, multi-agent systems have proven valuable for open-ended research tasks.

Users have said that Claude helped them find business opportunities they hadn’t considered, navigate complex healthcare options, resolve thorny technical bugs, and save up to days of work by uncovering research connections they wouldn't have found alone.

Multi-agent research systems can operate reliably at scale with careful engineering, comprehensive testing, detail-oriented prompt and tool design, robust operational practices, and tight collaboration between research, product, and engineering teams who have a strong understanding of current agent capabilities.

We're already seeing these systems transform how people solve complex problems.

Written by Jeremy Hadfield, Barry Zhang, Kenneth Lien, Florian Scholz, Jeremy Fox, and Daniel Ford.

This work reflects the collective efforts of several teams across Anthropic who made the Research feature possible.

Special thanks go to the Anthropic apps engineering team, whose dedication brought this complex multi-agent system to production.

We're also grateful to our early users for their excellent feedback.

Below are some additional miscellaneous tips for multi-agent systems.

End-state evaluation of agents that mutate state over many turns.

Evaluating agents that modify persistent state across multi-turn conversations presents unique challenges.

Unlike read-only research tasks, each action can change the environment for subsequent steps, creating dependencies that traditional evaluation methods struggle to handle.

We found success focusing on end-state evaluation rather than turn-by-turn analysis.

Instead of judging whether the agent followed a specific process, evaluate whether it achieved the correct final state.

This approach acknowledges that agents may find alternative paths to the same goal while still ensuring they deliver the intended outcome.

For complex workflows, break evaluation into discrete checkpoints where specific state changes should have occurred, rather than attempting to validate every intermediate step.

Long-horizon conversation management.

Production agents often engage in conversations spanning hundreds of turns, requiring careful context management strategies.

As conversations extend, standard context windows become insufficient, necessitating intelligent compression and memory mechanisms.

We implemented patterns where agents summarize completed work phases and store essential information in external memory before proceeding to new tasks.

When context limits approach, agents can spawn fresh subagents with clean contexts while maintaining continuity through careful handoffs.

Further, they can retrieve stored context like the research plan from their memory rather than losing previous work when reaching the context limit.

This distributed approach prevents context overflow while preserving conversation coherence across extended interactions.

Subagent output to a filesystem to minimize the ‘game of telephone.’

Direct subagent outputs can bypass the main coordinator for certain types of results, improving both fidelity and performance.

Rather than requiring subagents to communicate everything through the lead agent, implement artifact systems where specialized agents can create outputs that persist independently.

Subagents call tools to store their work in external systems, then pass lightweight references back to the coordinator.

This prevents information loss during multi-stage processing and reduces token overhead from copying large outputs through conversation history.

The pattern works particularly well for structured outputs like code, reports, or data visualizations where the subagent's specialized prompt produces better results than filtering through a general coordinator.