Building effective agents
构建有效代理
Over the past year, we've worked with dozens of teams building large language model (LLM) agents across industries. Consistently, the most successful implementations weren't using complex frameworks or specialized libraries. Instead, they were building with simple, composable patterns.
过去一年中,我们与数十个团队合作,在各个行业构建了大型语言模型(LLM)代理。一致的是,最成功的实施并没有使用复杂的框架或专门的库。相反,他们使用的是简单、可组合的模式。
In this post, we share what we’ve learned from working with our customers and building agents ourselves, and give practical advice for developers on building effective agents.
在这篇文章中,我们分享了我们与客户合作以及自己构建代理人的经验,并为开发者提供构建有效代理人的实用建议。
What are agents? 什么是代理?
"Agent" can be defined in several ways. Some customers define agents as fully autonomous systems that operate independently over extended periods, using various tools to accomplish complex tasks. Others use the term to describe more prescriptive implementations that follow predefined workflows. At Anthropic, we categorize all these variations as agentic systems, but draw an important architectural distinction between workflows and agents:
"代理"可以有多种定义方式。一些客户将代理定义为在长时间内独立运行的完全自主系统,使用各种工具来完成复杂任务。其他人则用这个术语来描述遵循预定义工作流程的更具体实现。在 Anthropic,我们将所有这些变体归类为代理系统,但在工作流程和代理之间做出重要的架构区分:
- Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
工作流程是通过预定义的代码路径对LLMs和工具进行编排的系统。 - Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.
代理,另一方面,是系统,其中LLMs动态地指导自己的流程和工具使用,保持对完成任务方式的控制。
Below, we will explore both types of agentic systems in detail. In Appendix 1 (“Agents in Practice”), we describe two domains where customers have found particular value in using these kinds of systems.
以下,我们将详细探讨这两种代理系统的类型。在附录 1(《实践中的代理》)中,我们描述了两个领域,客户在这些领域中特别重视使用这类系统。
When (and when not) to use agents
何时(以及何时不)使用代理
When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all. Agentic systems often trade latency and cost for better task performance, and you should consider when this tradeoff makes sense.
在构建带有LLMs的应用程序时,我们建议寻找尽可能简单的解决方案,并在必要时才增加复杂性。这可能意味着根本不构建代理系统。代理系统通常以牺牲延迟和成本来换取更好的任务性能,你应该考虑这种权衡何时合理。
When more complexity is warranted, workflows offer predictability and consistency for well-defined tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale. For many applications, however, optimizing single LLM calls with retrieval and in-context examples is usually enough.
当需要更多复杂性时,工作流程为定义明确的任务提供可预测性和一致性,而代理在需要大规模的灵活性和模型驱动决策时是更好的选择。然而,对于许多应用来说,优化单个LLM调用与检索和上下文示例通常就足够了。
When and how to use frameworks
何时以及如何使用框架
There are many frameworks that make agentic systems easier to implement, including:
有许多框架使实现代理系统变得更容易,包括:
- LangGraph from LangChain;
LangGraph 来自 LangChain - Amazon Bedrock's AI Agent framework;
亚马逊 Bedrock 的 AI 代理框架 - Rivet, a drag and drop GUI LLM workflow builder; and
铆钉,一个拖放式 GUI LLM 工作流程构建器;和 - Vellum, another GUI tool for building and testing complex workflows.
羊皮纸,另一个用于构建和测试复杂工作流的图形用户界面工具。
These frameworks make it easy to get started by simplifying standard low-level tasks like calling LLMs, defining and parsing tools, and chaining calls together. However, they often create extra layers of abstraction that can obscure the underlying prompts and responses, making them harder to debug. They can also make it tempting to add complexity when a simpler setup would suffice.
这些框架通过简化标准低级任务(如调用LLMs、定义和解析工具以及链式调用)使开始变得容易。然而,它们通常创建额外的抽象层,这可能会掩盖底层提示和响应,使得调试更加困难。它们还可能让人倾向于添加不必要的复杂性。
We suggest that developers start by using LLM APIs directly: many patterns can be implemented in a few lines of code. If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about what's under the hood are a common source of customer error.
我们建议开发者首先直接使用LLM API:许多模式可以在几行代码中实现。如果您确实使用框架,请确保您理解底层代码。对底层代码的错误假设是客户错误的一个常见来源。
See our cookbook for some sample implementations.
查看我们的食谱以获取一些示例实现。
Building blocks, workflows, and agents
积木、工作流和代理
In this section, we’ll explore the common patterns for agentic systems we’ve seen in production. We'll start with our foundational building block—the augmented LLM—and progressively increase complexity, from simple compositional workflows to autonomous agents.
在这一节中,我们将探讨我们在生产中看到的代理系统的常见模式。我们将从我们的基础构建块——增强的LLM——开始,并逐步增加复杂性,从简单的组合工作流程到自主代理。
Building block: The augmented LLM
构建块:增强 LLM
The basic building block of agentic systems is an LLM enhanced with augmentations such as retrieval, tools, and memory. Our current models can actively use these capabilities—generating their own search queries, selecting appropriate tools, and determining what information to retain.
代理系统的基本构建块是一个增强的LLM,其中包含检索、工具和记忆等增强功能。我们的当前模型可以积极使用这些功能——生成自己的搜索查询、选择适当的工具以及确定要保留的信息。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2Fd3083d3f40bb2b6f477901cc9a240738d3dd1371-2401x1000.png&w=3840&q=75)
We recommend focusing on two key aspects of the implementation: tailoring these capabilities to your specific use case and ensuring they provide an easy, well-documented interface for your LLM. While there are many ways to implement these augmentations, one approach is through our recently released Model Context Protocol, which allows developers to integrate with a growing ecosystem of third-party tools with a simple client implementation.
我们建议关注实施的两个关键方面:将这些功能定制到您的特定用例中,并确保它们为您的LLM提供简单、易于文档化的接口。虽然实现这些增强有多种方法,但一种方法是通过我们最近发布的模型上下文协议,该协议允许开发者通过简单的客户端实现与不断增长的第三方工具生态系统集成。
For the remainder of this post, we'll assume each LLM call has access to these augmented capabilities.
对于本帖剩余部分,我们将假设每个LLM调用都能访问这些增强功能。
Workflow: Prompt chaining
工作流程:提示链
Prompt chaining decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one. You can add programmatic checks (see "gate” in the diagram below) on any intermediate steps to ensure that the process is still on track.
提示链将任务分解为一系列步骤,其中每个LLM调用处理上一个步骤的输出。您可以在任何中间步骤添加程序性检查(见下图中“门”),以确保过程仍在正轨上。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F7418719e3dab222dccb379b8879e1dc08ad34c78-2401x1000.png&w=3840&q=75)
提示链工作流程
When to use this workflow: This workflow is ideal for situations where the task can be easily and cleanly decomposed into fixed subtasks. The main goal is to trade off latency for higher accuracy, by making each LLM call an easier task.
何时使用此工作流程:此工作流程适用于任务可以轻松且干净地分解为固定子任务的情况。主要目标是牺牲延迟以换取更高的准确性,通过使每个LLM调用成为一个更简单的任务。
Examples where prompt chaining is useful:
示例中,提示链式操作是有用的:
- Generating Marketing copy, then translating it into a different language.
生成营销文案,然后将其翻译成另一种语言。 - Writing an outline of a document, checking that the outline meets certain criteria, then writing the document based on the outline.
撰写文档大纲,检查大纲是否符合某些标准,然后根据大纲撰写文档。
Workflow: Routing 工作流程:路由
Routing classifies an input and directs it to a specialized followup task. This workflow allows for separation of concerns, and building more specialized prompts. Without this workflow, optimizing for one kind of input can hurt performance on other inputs.
路由将输入分类并引导它执行专门的后续任务。这种工作流程允许关注点的分离,并构建更专业的提示。如果没有这种工作流程,针对一种输入进行优化可能会损害其他输入的性能。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F5c0c0e9fe4def0b584c04d37849941da55e5e71c-2401x1000.png&w=3840&q=75)
When to use this workflow: Routing works well for complex tasks where there are distinct categories that are better handled separately, and where classification can be handled accurately, either by an LLM or a more traditional classification model/algorithm.
何时使用此工作流程:当存在明确的类别且更适合单独处理,且分类可以准确处理时,无论是通过LLM还是更传统的分类模型/算法,路由工作得很好。
Examples where routing is useful:
路由有用的例子:
- Directing different types of customer service queries (general questions, refund requests, technical support) into different downstream processes, prompts, and tools.
将不同类型的客户服务查询(一般问题、退款请求、技术支持)导向不同的下游流程、提示和工具。 - Routing easy/common questions to smaller models like Claude 3.5 Haiku and hard/unusual questions to more capable models like Claude 3.5 Sonnet to optimize cost and speed.
将简单/常见问题路由到较小的模型如 Claude 3.5 Haiku,将复杂/不寻常问题路由到更强大的模型如 Claude 3.5 Sonnet,以优化成本和速度。
Workflow: Parallelization
工作流程:并行化
LLMs can sometimes work simultaneously on a task and have their outputs aggregated programmatically. This workflow, parallelization, manifests in two key variations:
LLMs有时可以同时处理一个任务并程序化地汇总其输出。这种工作流程,即并行化,体现在两种关键变化中:
- Sectioning: Breaking a task into independent subtasks run in parallel.
章节划分:将任务分解为独立子任务并行运行。 - Voting: Running the same task multiple times to get diverse outputs.
投票:多次运行相同任务以获得不同的输出。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F406bb032ca007fd1624f261af717d70e6ca86286-2401x1000.png&w=3840&q=75)
并行化工作流程
When to use this workflow: Parallelization is effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed for higher confidence results. For complex tasks with multiple considerations, LLMs generally perform better when each consideration is handled by a separate LLM call, allowing focused attention on each specific aspect.
何时使用此工作流程:当可以将子任务并行化以提高速度,或者需要多个视角或尝试以获得更高置信度的结果时,并行化是有效的。对于需要考虑多个因素的复杂任务,通常当每个考虑因素由单独的LLM调用处理时,LLMs的表现更好,这样可以专注于每个具体方面。
Examples where parallelization is useful:
示例中并行化有用的场景:
- Sectioning: 章节划分:
- Implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.
实现一个机制,其中一个模型实例处理用户查询,而另一个实例筛选不适当的内容或请求。这通常比让同一个LLM调用同时处理防护栏和核心响应表现得更好。 - Automating evals for evaluating LLM performance, where each LLM call evaluates a different aspect of the model’s performance on a given prompt.
自动化评估LLM性能,其中每个LLM调用评估给定提示下模型性能的不同方面。
- Implementing guardrails where one model instance processes user queries while another screens them for inappropriate content or requests. This tends to perform better than having the same LLM call handle both guardrails and the core response.
- Voting: 投票:
- Reviewing a piece of code for vulnerabilities, where several different prompts review and flag the code if they find a problem.
审查代码以发现漏洞,其中几个不同的提示会审查代码并在发现问题时标记它。 - Evaluating whether a given piece of content is inappropriate, with multiple prompts evaluating different aspects or requiring different vote thresholds to balance false positives and negatives.
评估给定内容是否不适当,通过多个提示评估不同方面或需要不同的投票阈值来平衡误报和漏报。
- Reviewing a piece of code for vulnerabilities, where several different prompts review and flag the code if they find a problem.
Workflow: Orchestrator-workers
工作流程:编排器-工作者
In the orchestrator-workers workflow, a central LLM dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results.
在编排器-工作者工作流程中,一个中央LLM动态分解任务,将它们委托给工作者LLMs,并综合他们的结果。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F8985fc683fae4780fb34eab1365ab78c7e51bc8e-2401x1000.png&w=3840&q=75)
指挥员-工作人员工作流程
When to use this workflow: This workflow is well-suited for complex tasks where you can’t predict the subtasks needed (in coding, for example, the number of files that need to be changed and the nature of the change in each file likely depend on the task). Whereas it’s topographically similar, the key difference from parallelization is its flexibility—subtasks aren't pre-defined, but determined by the orchestrator based on the specific input.
何时使用此工作流程:此工作流程非常适合复杂任务,在这些任务中,您无法预测所需的子任务(例如,在编码中,需要更改的文件数量以及每个文件更改的性质可能取决于任务)。而它与并行化在拓扑上相似,但关键区别在于其灵活性——子任务不是预先定义的,而是由协调者根据特定输入确定的。
Example where orchestrator-workers is useful:
示例中 orchestrator-workers 的有用之处:
- Coding products that make complex changes to multiple files each time.
编写每次对多个文件进行复杂更改的产品代码。 - Search tasks that involve gathering and analyzing information from multiple sources for possible relevant information.
搜索涉及从多个来源收集和分析信息以获取可能相关信息的任务。
Workflow: Evaluator-optimizer
工作流程:评估器-优化器
In the evaluator-optimizer workflow, one LLM call generates a response while another provides evaluation and feedback in a loop.
在评估-优化工作流程中,一个LLM调用生成响应,另一个在循环中提供评估和反馈。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F14f51e6406ccb29e695da48b17017e899a6119c7-2401x1000.png&w=3840&q=75)
评估优化工作流程
When to use this workflow: This workflow is particularly effective when we have clear evaluation criteria, and when iterative refinement provides measurable value. The two signs of good fit are, first, that LLM responses can be demonstrably improved when a human articulates their feedback; and second, that the LLM can provide such feedback. This is analogous to the iterative writing process a human writer might go through when producing a polished document.
何时使用此工作流程:当我们有明确的评估标准,并且迭代优化能提供可衡量的价值时,此工作流程特别有效。良好的匹配的两个迹象是,首先,当人类明确表达他们的反馈时,LLM的响应可以明显改进;其次,LLM可以提供此类反馈。这类似于人类作家在创作精炼文档时可能经历的迭代写作过程。
Examples where evaluator-optimizer is useful:
示例中,评估优化器是有用的:
- Literary translation where there are nuances that the translator LLM might not capture initially, but where an evaluator LLM can provide useful critiques.
文学翻译中,译者LLM可能最初无法捕捉到的细微差别,但评估者LLM可以提供有用的批评。 - Complex search tasks that require multiple rounds of searching and analysis to gather comprehensive information, where the evaluator decides whether further searches are warranted.
复杂搜索任务,需要多轮搜索和分析以收集全面信息,其中评估者决定是否需要进行进一步搜索。
Agents 代理商
Agents are emerging in production as LLMs mature in key capabilities—understanding complex inputs, engaging in reasoning and planning, using tools reliably, and recovering from errors. Agents begin their work with either a command from, or interactive discussion with, the human user. Once the task is clear, agents plan and operate independently, potentially returning to the human for further information or judgement. During execution, it's crucial for the agents to gain “ground truth” from the environment at each step (such as tool call results or code execution) to assess its progress. Agents can then pause for human feedback at checkpoints or when encountering blockers. The task often terminates upon completion, but it’s also common to include stopping conditions (such as a maximum number of iterations) to maintain control.
代理在生产中逐渐出现,随着LLMs在关键能力上的成熟——理解复杂输入、进行推理和规划、可靠地使用工具以及从错误中恢复。代理开始工作,要么是来自人类用户的命令,要么是与人类用户的交互式讨论。一旦任务明确,代理将独立规划和操作,可能需要返回人类用户以获取更多信息或判断。在执行过程中,代理在每一步从环境中获取“事实真相”(如工具调用结果或代码执行)以评估其进度至关重要。代理可以在检查点或遇到阻碍时暂停以获取人类反馈。任务通常在完成时终止,但通常也会包括停止条件(如最大迭代次数)以保持控制。
Agents can handle sophisticated tasks, but their implementation is often straightforward. They are typically just LLMs using tools based on environmental feedback in a loop. It is therefore crucial to design toolsets and their documentation clearly and thoughtfully. We expand on best practices for tool development in Appendix 2 ("Prompt Engineering your Tools").
代理可以处理复杂的任务,但它们的实现通常很简单。它们通常只是使用基于环境反馈的工具在循环中LLMs。因此,设计工具集及其文档清晰和周到至关重要。我们在附录 2(“你的工具提示工程”)中扩展了工具开发的最佳实践。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F58d9f10c985c4eb5d53798dea315f7bb5ab6249e-2401x1000.png&w=3840&q=75)
When to use agents: Agents can be used for open-ended problems where it’s difficult or impossible to predict the required number of steps, and where you can’t hardcode a fixed path. The LLM will potentially operate for many turns, and you must have some level of trust in its decision-making. Agents' autonomy makes them ideal for scaling tasks in trusted environments.
何时使用代理:代理可用于开放性问题,在这些问题上难以或无法预测所需的步骤数量,并且无法硬编码固定路径。LLM 可能会运行许多回合,你必须对其决策有一定程度的信任。代理的自主性使它们在受信任的环境中扩展任务变得理想。
The autonomous nature of agents means higher costs, and the potential for compounding errors. We recommend extensive testing in sandboxed environments, along with the appropriate guardrails.
智能体的自主性意味着更高的成本和累积错误的潜在可能性。我们建议在沙盒环境中进行广泛的测试,并配备适当的防护措施。
Examples where agents are useful:
示例中代理有用的场景:
The following examples are from our own implementations:
以下示例来自我们自己的实现:
- A coding Agent to resolve SWE-bench tasks, which involve edits to many files based on a task description;
一个用于解决 SWE-bench 任务的编码代理,这些任务涉及根据任务描述对许多文件进行编辑; - Our “computer use” reference implementation, where Claude uses a computer to accomplish tasks.
我们的“计算机使用”参考实现,其中 Claude 使用计算机完成任务。
![](/_next/image?url=https%3A%2F%2Fwww-cdn.anthropic.com%2Fimages%2F4zrzovbb%2Fwebsite%2F4b9a1f4eb63d5962a6e1746ac26bbc857cf3474f-2400x1666.png&w=3840&q=75)
高级编码代理的流程
Combining and customizing these patterns
结合和自定义这些模式
These building blocks aren't prescriptive. They're common patterns that developers can shape and combine to fit different use cases. The key to success, as with any LLM features, is measuring performance and iterating on implementations. To repeat: you should consider adding complexity only when it demonstrably improves outcomes.
这些构建块不是规定性的。它们是开发者可以塑造和组合以适应不同用例的常见模式。成功的关键,就像任何LLM功能一样,是衡量性能并在实现上进行迭代。重复一下:你应该只在明显改善结果时才考虑增加复杂性。
Summary 摘要
Success in the LLM space isn't about building the most sophisticated system. It's about building the right system for your needs. Start with simple prompts, optimize them with comprehensive evaluation, and add multi-step agentic systems only when simpler solutions fall short.
在LLM空间的成功不在于构建最复杂的系统。它关乎构建适合您需求的正确系统。从简单的提示开始,通过全面的评估进行优化,只有当简单解决方案不足时才添加多步骤的代理系统。
When implementing agents, we try to follow three core principles:
当实现代理时,我们试图遵循三个核心原则:
- Maintain simplicity in your agent's design.
保持您的代理设计简洁。 - Prioritize transparency by explicitly showing the agent’s planning steps.
优先展示代理的规划步骤,以明确提高透明度。 - Carefully craft your agent-computer interface (ACI) through thorough tool documentation and testing.
精心设计您的代理-计算机界面(ACI),通过详尽的工具文档和测试。
Frameworks can help you get started quickly, but don't hesitate to reduce abstraction layers and build with basic components as you move to production. By following these principles, you can create agents that are not only powerful but also reliable, maintainable, and trusted by their users.
框架可以帮助您快速入门,但在迁移到生产环境时,不要犹豫减少抽象层,使用基本组件进行构建。遵循这些原则,您可以创建既强大又可靠、易于维护且受用户信任的代理。
Acknowledgements 致谢
Written by Erik Schluntz and Barry Zhang. This work draws upon our experiences building agents at Anthropic and the valuable insights shared by our customers, for which we're deeply grateful.
由 Erik Schluntz 和 Barry Zhang 撰写。本作品借鉴了我们构建 Anthropic 代理的经验以及客户分享的宝贵见解,对此我们深表感激。
Appendix 1: Agents in practice
附录 1:实践中的代理
Our work with customers has revealed two particularly promising applications for AI agents that demonstrate the practical value of the patterns discussed above. Both applications illustrate how agents add the most value for tasks that require both conversation and action, have clear success criteria, enable feedback loops, and integrate meaningful human oversight.
我们的客户合作中揭示了两个特别有前景的 AI 代理应用,这些应用展示了上述讨论的模式的实际价值。这两个应用都说明了代理如何为需要对话和行动的任务增加最大价值,具有明确的成功标准,能够实现反馈循环,并整合有意义的监督。
A. Customer support A. 客户支持
Customer support combines familiar chatbot interfaces with enhanced capabilities through tool integration. This is a natural fit for more open-ended agents because:
客户支持将熟悉的聊天机器人界面与通过工具集成增强的功能相结合。这对于更开放式的代理来说是一种自然的选择,因为:
- Support interactions naturally follow a conversation flow while requiring access to external information and actions;
支持的自然交互紧随对话流程,同时需要访问外部信息和执行操作; - Tools can be integrated to pull customer data, order history, and knowledge base articles;
工具可以集成以拉取客户数据、订单历史和知识库文章; - Actions such as issuing refunds or updating tickets can be handled programmatically; and
操作如退款或更新票务可以程序化处理; - Success can be clearly measured through user-defined resolutions.
成功可以通过用户定义的决议明显衡量。
Several companies have demonstrated the viability of this approach through usage-based pricing models that charge only for successful resolutions, showing confidence in their agents' effectiveness.
数家公司通过基于使用情况的定价模式展示了这种方法的可行性,仅对成功的解决方案收费,显示出对代理有效性的信心。
B. Coding agents B. 编码代理
The software development space has shown remarkable potential for LLM features, with capabilities evolving from code completion to autonomous problem-solving. Agents are particularly effective because:
软件开发领域在LLM功能方面展现出显著潜力,功能从代码补全发展到自主解决问题。代理特别有效,因为:
- Code solutions are verifiable through automated tests;
代码解决方案可以通过自动化测试进行验证; - Agents can iterate on solutions using test results as feedback;
代理可以使用测试结果作为反馈来迭代解决方案; - The problem space is well-defined and structured; and
问题空间定义良好且结构化; - Output quality can be measured objectively.
输出质量可以客观衡量。
In our own implementation, agents can now solve real GitHub issues in the SWE-bench Verified benchmark based on the pull request description alone. However, whereas automated testing helps verify functionality, human review remains crucial for ensuring solutions align with broader system requirements.
在我们的实现中,代理现在可以仅根据拉取请求描述解决 SWE-bench 验证基准中的真实 GitHub 问题。然而,虽然自动化测试有助于验证功能,但人工审查对于确保解决方案与更广泛的系统要求一致仍然至关重要。
Appendix 2: Prompt engineering your tools
附录 2:工具的提示工程
No matter which agentic system you're building, tools will likely be an important part of your agent. Tools enable Claude to interact with external services and APIs by specifying their exact structure and definition in our API. When Claude responds, it will include a tool use block in the API response if it plans to invoke a tool. Tool definitions and specifications should be given just as much prompt engineering attention as your overall prompts. In this brief appendix, we describe how to prompt engineer your tools.
无论您正在构建哪种代理系统,工具都可能成为您代理的重要组成部分。工具使 Claude 能够通过指定它们在 API 中的确切结构和定义来与外部服务和 API 交互。当 Claude 响应时,如果它打算调用工具,它将在 API 响应中包含一个工具使用块。工具的定义和规范应该像您的整体提示一样得到足够的提示工程关注。在本附录中,我们描述了如何提示工程您的工具。
There are often several ways to specify the same action. For instance, you can specify a file edit by writing a diff, or by rewriting the entire file. For structured output, you can return code inside markdown or inside JSON. In software engineering, differences like these are cosmetic and can be converted losslessly from one to the other. However, some formats are much more difficult for an LLM to write than others. Writing a diff requires knowing how many lines are changing in the chunk header before the new code is written. Writing code inside JSON (compared to markdown) requires extra escaping of newlines and quotes.
通常有几种方式可以指定相同的行为。例如,你可以通过编写差异或重写整个文件来指定文件编辑。对于结构化输出,你可以在 Markdown 或 JSON 中返回代码。在软件工程中,这些差异是表面的,可以从一种格式无损转换为另一种格式。然而,某些格式对于LLM来说比其他格式更难编写。编写差异需要知道在写入新代码之前,块标题中有多少行正在更改。在 JSON 中编写代码(与 Markdown 相比)需要额外的换行符和引号转义。
Our suggestions for deciding on tool formats are the following:
我们的关于决定工具格式的建议如下:
- Give the model enough tokens to "think" before it writes itself into a corner.
给模型足够的 token 来“思考”,在它将自己写进死胡同之前。 - Keep the format close to what the model has seen naturally occurring in text on the internet.
保持格式与模型在互联网文本中自然出现的格式相近。 - Make sure there's no formatting "overhead" such as having to keep an accurate count of thousands of lines of code, or string-escaping any code it writes.
确保没有格式“开销”,例如需要准确计算数千行代码,或者对它所写的代码进行字符串转义。
One rule of thumb is to think about how much effort goes into human-computer interfaces (HCI), and plan to invest just as much effort in creating good agent-computer interfaces (ACI). Here are some thoughts on how to do so:
一条经验法则是思考投入人机界面(HCI)的努力有多少,并计划投入同样多的努力来创建良好的代理计算机界面(ACI)。以下是一些关于如何做到这一点的想法:
- Put yourself in the model's shoes. Is it obvious how to use this tool, based on the description and parameters, or would you need to think carefully about it? If so, then it’s probably also true for the model. A good tool definition often includes example usage, edge cases, input format requirements, and clear boundaries from other tools.
把自己放在模型的立场上。根据描述和参数,使用这个工具显然吗,还是你需要仔细思考?如果是这样,那么对模型来说可能也是如此。一个好的工具定义通常包括示例用法、边缘情况、输入格式要求以及与其他工具的清晰界限。 - How can you change parameter names or descriptions to make things more obvious? Think of this as writing a great docstring for a junior developer on your team. This is especially important when using many similar tools.
如何更改参数名称或描述以使事物更明显?把这当作为团队中的初级开发者编写优秀的文档字符串。当使用许多类似工具时,这尤为重要。 - Test how the model uses your tools: Run many example inputs in our workbench to see what mistakes the model makes, and iterate.
测试模型如何使用您的工具:在我们的工作台中运行许多示例输入,以查看模型犯的错误,并进行迭代。 - Poka-yoke your tools. Change the arguments so that it is harder to make mistakes.
请对您的工具进行防错设计。更改参数,使其更难出错。
While building our agent for SWE-bench, we actually spent more time optimizing our tools than the overall prompt. For example, we found that the model would make mistakes with tools using relative filepaths after the agent had moved out of the root directory. To fix this, we changed the tool to always require absolute filepaths—and we found that the model used this method flawlessly.
在构建我们的 SWE-bench 代理时,我们实际上花在优化工具上的时间比整体提示还要多。例如,我们发现代理离开根目录后,模型会使用相对文件路径的工具出错。为了解决这个问题,我们将工具改为始终要求绝对文件路径——我们发现模型使用这种方法毫无瑕疵。