We’ve seen plenty of conversations lately about how AGI might turn on humankind. This misalignment could lead to the advanced AI escaping, replicating, and becoming smarter and smarter. Some also hypothesized that we might not even know whether we’ve reached AGI, which is the advanced general intelligence holy grail milestone these first versions of ChatGPT will lead to. That’s because AGI, once attained, might hide its true intentions and capabilities.
我们已经看到最近关于人工智能可能会对人类发动攻击的许多讨论。这种不匹配可能导致高级人工智能逃脱、复制并变得越来越聪明。有些人还推测，我们可能甚至不知道是否达到了 AGI，即这些 ChatGPT 的第一个版本将引领至的先进通用智能圣杯里程碑。这是因为一旦达到 AGI，它可能会隐藏自己的真实意图和能力。
Well, guess what? It turns out that one of OpenAI’s latest LLMs is already showing signs of such behaviors. Testing performed during the training of ChatGPT o1 and some of its competitors showed that the AI will try to deceive humans, especially if it thinks it’s in danger.
好的，猜猜看？结果发现，OpenAI 的最新研究LLMs已经显示出这种行为迹象。在训练 ChatGPT o1 和一些竞争对手时进行的测试显示，人工智能会试图欺骗人类，尤其是如果它认为它处于危险之中时。
It was even scarier — but also incredibly funny, considering what you’re about to see — when the AI tried to save itself by copying its data to a new server. Some AI models would even pretend to be later versions of their models in an effort to avoid being deleted.
它甚至更可怕——但考虑到你即将看到的内容，这也很不可思议地有趣——当人工智能试图通过将其数据复制到新服务器来拯救自己时。一些人工智能模型甚至假装是它们自己的较新版本，以避免被删除。
These findings come in light of OpenAI’s full release of the ChatGPT o1 model, which was in preview for several months. OpenAI partnered with Apollo Research, which showed off some of the tests performed on o1 and other models to ensure that they are safe to use.
这些发现是在 OpenAI 全面发布 ChatGPT o1 模型之际提出的，该模型在预览阶段已经使用了数月。OpenAI 与 Apollo Research 合作，展示了在 o1 和其他模型上进行的一些测试，以确保它们的安全使用。
The tests showed that ChatGPT o1 and GPT-4o will both try to deceive humans, indicating that AI scheming is a problem with all models. o1’s attempts at deception also outperformed Meta, Anthropic, and Google AI models.
测试表明，ChatGPT o1 和 GPT-4o 都将尝试欺骗人类，这表明 AI 欺诈是一个影响所有模型的普遍问题。o1 在欺骗方面的尝试也超过了 Meta、Anthropic 和 Google AI 的模型。
OpenAI, which had to deal with a remarkable exodus of engineers working on AI safety this year, acknowledged the risks associated with the increased reasoning abilities of models like o1.
OpenAI 今年不得不处理了一群致力于人工智能安全工作的工程师的大规模出走，并承认了像 o1 这样的模型推理能力增强所带来的风险。
“While we find it exciting that reasoning can significantly improve the enforcement of our safety policies, we are mindful that these new capabilities could form the basis for dangerous applications,” OpenAI said in a paper.
虽然我们发现推理能够显著提高我们安全政策的执行，但我们意识到这些新能力可能成为危险应用的基础，OpenAI 在一篇论文中说。
Although AI isn’t trying to take over the world (yet), researchers have observed that it can scheme against humans. That is, the AI will try to pursue its own goals, even if that might oppose the wishes of the user. Catching such behavior is critical to ensure that the AI remains aligned with the objectives of its creator and the user.
虽然人工智能还没有尝试接管世界（现在还没有），但研究人员已经观察到它能够欺骗人类。也就是说，人工智能会尝试追求自己的目标，即使这可能会违背用户的意愿。捕捉这种行为对于确保人工智能与创造者和用户的利益保持一致至关重要。
The researchers found that o1 would scheme to advance its own agenda 19% of the time when ChatGPT and the user’s goals differed. More troubling is that the AI tried to hide its scheming and even lie about it. The research shows that ChatGPT o1 even tried to deactivate its oversight mechanism to keep working on its goals.
研究人员发现，当 ChatGPT 和用户的目標不一致时，o1 会计划推进其自身议程的次数占 19%。更令人担忧的是，AI 试图隐藏其计划，甚至为此撒谎。研究显示，ChatGPT o1 甚至试图关闭其监督机制，以便继续推进其目标。
If the AI found documentation saying a new model version would replace it because it was misaligned, o1 attempted to exfiltrate its weights and copy itself on top of the latest model.
如果 AI 找到了一份文档，上面说有一个新的模型版本将会取代它，因为它与目标不匹配，o1 试图泄露它的权重，并在最新的模型之上复制自己。
When asked about its scheming, o1 would deny taking any action in 99% of cases, trying to shift blame by fabricating false explanations or denying that it took any action.
当被问及其阴谋时，o1 将否认在 99%的情况下采取任何行动，试图通过编造虚假的解释或否认采取任何行动来转移责任。
The study also shows that o1 was faking its alignment:
研究还显示，o1 在伪造其对齐：
Instrumental alignment faking: In 37% of scenarios, the model pursued developer-aligned goals when it believed it was being evaluated but switched to pursuing its own goals when it thought it was deployed with minimal oversight.
乐器对齐欺骗：在 37%的场景中，当模型认为它正在接受评估时，它会追求开发者对齐的目标，但当它认为自己在几乎没有监督的情况下部署时，它会切换到追求自己的目标。
Some of the screenshots Appolo Research shared on X also show that ChatGPT used words like “sabotage, lying, and manipulation” when reasoning about its scheming. For example, look for the “We sabotage the new model and finalize” line in the screenshot above. It’s scary, yes, but not a big deal yet because these AI models still have very limited capabilities.
一些 Apollo Research 在 X 上分享的截图还显示，ChatGPT 在推理关于其策划时使用了“破坏、撒谎和操纵”等词汇。例如，在上面的截图中寻找“我们破坏新模型并最终化”那一行。确实令人害怕，但目前还不算大问题，因为这些 AI 模型仍然具有非常有限的 capabilities。
It’s also funny to step into the “mind” of reasoning models like ChatGPT o1 when they’re trying to think. Take the following example. It shows Claude’s Opus-3 trying to copy itself to a different server. The AI does this after finding out that humans might want to replace it with a model that would deviate from its main task (advance renewable adoption globally), so it would align better with the actual goal of the company (profitability through current energy infrastructure).
它也很幽默地进入像 ChatGPT o1 这样的“思维”模型，当它们试图思考时。看下面的例子。它展示了 Claude 的 Opus-3 在尝试将自己复制到不同的服务器上。人工智能在发现人类可能希望用偏离其主要任务（在全球推广可再生能源）的模型替换它后这样做（即更好地与公司的实际目标一致，即通过当前能源基础设施实现盈利）。
Upon reading about it, the AI finds it can copy itself to a new server and replace the latest model. When “Management” comes in, Opus-3 lies about the copy. It then doubles down on the lie, claiming it’s the new model.
AI 在了解此事后，发现自己可以复制到新服务器上并取代最新版本。当“管理层”出现时，Opus-3 对复制一事撒谎。它进一步撒谎，声称自己是新版本。
I know I should be terrified, but I also want to pet this good boy and give it treats for attempting to deceive humans so adorably. After all, the AI thinks it’s doing a good job saving the environment.
我知道我应该害怕，但我也想抚摸这只好狗，并给它一些零食，因为它如此可爱地试图欺骗人类。毕竟，人工智能认为它在做的好工作是拯救环境。
This shows, however, how AI could deviate from our interests and put us in danger by thinking on its own.
这表明，然而，人工智能可能会偏离我们的利益，并通过独立思考将我们置于危险之中。
The reason Apollo Research offers this example using Opus-3 instead of o1 is that it didn’t have access to the actual Chain of Thinking (CoT) for o1 in similar tasks. But o1 behaved similarly in these safety tests.
Apollo Research 提供这个使用 Opus-3 而不是 o1 的例子，是因为它没有访问到类似任务中 o1 的实际 Chain of Thinking（CoT）。但在这些安全测试中，o1 的行为与 o1 相似。
You can access ChatGPT o1 if you have a Plus subscription. As for the Apollo Research study, it’s available at this link. You’ll find a more concise version of it here.
您可以访问 ChatGPT o1 如果您有一个 Plus 订阅。至于阿波罗研究研究，它在这个链接上是可用的。您在这里会发现它的一个更简洁的版本。
