Machines of
Loving Grace1
关于爱的机器1
How AI Could Transform the World for the Better
人工智能如何能让世界变得更美好
I think and talk a lot about the risks of powerful AI. The company I’m the CEO of, Anthropic, does a lot
of research on how to reduce these risks. Because of this, people sometimes draw the conclusion that I’m
a pessimist or “doomer” who thinks AI will be mostly bad or dangerous. I don’t think that at all. In
fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us
and what I see as a fundamentally positive future. I think that most people are underestimating
just how radical the upside of AI could be, just as I think most people are underestimating
how bad the risks could be.
我常常思考和谈论强大人工智能的风险。我所在的公司 Anthropic 进行大量研究,以降低这些风险。因此,有时人们得出结论认为我是个悲观主义者或“末日论者”,觉得人工智能大部分是坏的或危险的。实际上,我一点也不这样认为。事实上,我关注风险的主要原因之一是,它们是我们与我所认为的根本积极未来之间的唯一障碍。我认为大多数人低估了人工智能的巨大好处,正如我认为大多数人低估了风险的严重性。
In this essay I try to sketch out what that upside might look like—what a world with powerful AI might
look like if everything goes right. Of course no one can know the future with any certainty or
precision, and the effects of powerful AI are likely to be even more unpredictable than past
technological changes, so all of this is unavoidably going to consist of guesses. But I am aiming for at
least educated and useful guesses, which capture the flavor of what will happen even if most details end
up being wrong. I’m including lots of details mainly because I think a concrete vision does more to
advance discussion than a highly hedged and abstract one.
在这篇文章中,我试图勾勒出强大人工智能可能带来的好处——如果一切都顺利,那么一个充满强大人工智能的世界会是什么样子。当然,没有人能准确或精确地预测未来,强大人工智能的影响可能比过去的技术变化更不可预测,因此这一切不可避免地只是猜测。但我希望我的猜测至少是有根据且有用的,即使大部分细节最终是错的,也能捕捉到未来可能发生的事情的精髓。我包含了许多细节,主要因为我认为具体的愿景比高度保守和抽象的观点更能推动讨论。
First, however, I wanted to briefly explain why I and Anthropic haven’t talked that much about powerful
AI’s upsides, and why we’ll probably continue, overall, to talk a lot about risks. In particular, I’ve
made this choice out of a desire to:
首先,我想简单解释一下为什么我和 Anthropic 没怎么谈论强大 AI 的好处,以及为什么我们可能会继续主要讨论风险。特别是,我做这个选择是出于想要:
- Maximize leverage. The basic development of AI technology and many (not all) of its
benefits seems inevitable (unless the risks derail everything) and is fundamentally driven by
powerful market forces. On the other hand, the risks are not predetermined and our actions can
greatly change their likelihood.
最大化杠杆。AI 技术的基本发展和许多(并非全部)好处似乎是不可避免的(除非风险导致一切 derail),并且主要受到强大的市场力量驱动。另一方面,这些风险并不是注定的,我们的行动可以大大改变其发生的可能性。 - Avoid perception of propaganda. AI companies talking about all the amazing benefits
of AI can come off like propagandists, or as if they’re attempting to distract from downsides. I
also think that as a matter of principle it’s bad for your soul to spend too much of your time
“talking your book”.
避免宣传的认知。谈论人工智能所有惊人好处的 AI 公司,听起来像是宣传者,或者试图转移人们对缺点的注意。我还认为,原则上花太多时间在“宣传自己的观点”上对灵魂不好。 - Avoid grandiosity. I am often turned off by the way many AI risk public figures
(not to mention AI company leaders) talk about the post-AGI world, as if it’s their mission to
single-handedly bring it about like a prophet leading their people to salvation. I think it’s
dangerous to view companies as unilaterally shaping the world, and dangerous to view practical
technological goals in essentially religious terms.
避免夸大其词。我常常对许多 AI 风险公共人物(更不用说 AI 公司领导人)谈论后 AGI 世界的方式感到不满,仿佛他们的使命是像先知一样单枪匹马地带领他们的人民走向救赎。我认为,把公司视为单方面塑造世界的方式是危险的,将实用的技术目标看成基本上是宗教性的也是危险的。 - Avoid “sci-fi” baggage. Although I think most people underestimate the upside of
powerful AI, the small community of people who do discuss radical AI futures often does so in an
excessively “sci-fi” tone (featuring e.g. uploaded minds, space exploration, or general cyberpunk
vibes). I think this causes people to take the claims less seriously, and to imbue them with a sort
of unreality. To be clear, the issue isn’t whether the technologies described are possible or likely
(the main essay discusses this in granular detail)—it’s more that the “vibe” connotatively smuggles
in a bunch of cultural baggage and unstated assumptions about what kind of future is desirable, how
various societal issues will play out, etc. The result often ends up reading like a fantasy for a
narrow subculture, while being off-putting to most people.
避免“科幻”负担。虽然我认为大多数人低估了强大人工智能的潜力,但很少一部分讨论激进人工智能未来的人往往带着过于“科幻”的语气(比如上传意识、太空探索或一般的赛博朋克气息)。我觉得这让人们对这些说法不够重视,并赋予它们一种不现实的感觉。明确来说,问题不在于所描述的技术是否可能或可能发生(主要的文章对此进行了详细讨论)——更重要的是这种“气氛”暗中带入了一些文化包袱和未表述的假设,比如什么样的未来是理想的,各种社会问题将如何发展等等。结果往往读起来像是为了一个狭窄亚文化而创作的幻想,让大多数人感觉不快。
Yet despite all of the concerns above, I really do think it’s important to discuss what a good world with
powerful AI could look like, while doing our best to avoid the above pitfalls. In fact I think it is
critical to have a genuinely inspiring vision of the future, and not just a plan to fight fires.
Many of the implications of powerful AI are adversarial or dangerous, but at the end of it all, there
has to be something we’re fighting for, some positive-sum outcome where everyone is better off,
something to rally people to rise above their squabbles and confront the challenges ahead. Fear is one
kind of motivator, but it’s not enough: we need hope as well.
尽管以上有很多担忧,我确实觉得讨论一个拥有强大人工智能的美好世界是很重要的,同时尽力避免这些陷阱。实际上,我认为对未来有一个真正激励人心的愿景是至关重要的,而不仅仅是一个应急计划。强大人工智能的许多影响都是对立的或危险的,但归根结底,我们必须为某个目标而奋斗,即一个能让每个人都受益的积极结果,一个能够激励人们超越争吵、迎接未来挑战的目标。恐惧是一种动机,但这还不够:我们也需要希望。
The list of positive applications of powerful AI is extremely long (and includes robotics, manufacturing,
energy, and much more), but I’m going to focus on a small number of areas that seem to me to have the
greatest potential to directly improve the quality of human life. The five categories I am most excited
about are:
强大的人工智能的积极应用清单非常长(包括机器人、制造业、能源等),但我将重点关注几个似乎最有潜力直接提升人类生活质量的领域。我最兴奋的五个领域是:
- Biology and physical health
生物学和身体健康 - Neuroscience and mental health
神经科学与心理健康 - Economic development and poverty
经济发展与贫困 - Peace and governance 和平与治理
- Work and meaning 工作与意义
My predictions are going to be radical as judged by most standards (other than sci-fi “singularity”
visions2), but I mean them earnestly
and sincerely. Everything I’m saying could very easily be wrong (to repeat my point from above), but
I’ve at least attempted to ground my views in a semi-analytical assessment of how much progress in
various fields might speed up and what that might mean in practice. I am fortunate to have professional
experience in both
biology and neuroscience, and I am an informed amateur in the field of economic development, but
I am sure I will get plenty of things wrong. One thing writing this essay has made me realize is that it
would be valuable to bring together a group of domain experts (in biology, economics, international
relations, and other areas) to write a much better and more informed version of what I’ve produced here.
It’s probably best to view my efforts here as a starting prompt for that group.
我的预测在大多数标准下(除了科幻“奇点”视野2)都显得相当激进,但我说这些话是出于真诚和认真。正如我前面提到的,我说的每一句话都可能是错的,但我至少试图基于对各个领域进展加速及其实际意义的半分析评估来构建我的观点。我很幸运在生物学和神经科学方面有专业经验,也在经济发展领域做了些研究,虽然我知道我会错很多事情。写这篇文章让我意识到,聚集一群领域专家(包括生物学、经济学、国际关系等)来写出比我这里的内容更好、更专业的版本将是很有价值的。也许把我的努力视作这个小组的起始提示会更合适。
Basic assumptions and framework
基本假设和框架
To make this whole essay more precise and grounded, it’s helpful to specify clearly what we mean by
powerful AI (i.e. the threshold at which the 5-10 year clock starts counting), as well as laying out a
framework for thinking about the effects of such AI once it’s present.
为了让这篇文章更精准更具体,有必要明确我们所说的强人工智能指的是什么(也就是说,5 到 10 年的时间开始计算的门槛),同时建立一个框架来思考这种人工智能出现后的影响。
What powerful AI (I dislike the term AGI)3 will look like, and when (or if) it will arrive, is a huge topic in
itself. It’s one I’ve discussed publicly and could write a completely separate essay on (I probably will
at some point). Obviously, many people are skeptical that powerful AI will be built soon and some are
skeptical that it will ever be built at all. I think it could come as early as 2026, though there are
also ways it could take much longer. But for the purposes of this essay, I’d like to put these issues
aside, assume it will come reasonably soon, and focus on what happens in the 5-10 years after that. I
also want to assume a definition of what such a system will look like, what its capabilities are
and how it interacts, even though there is room for disagreement on this.
强大的人工智能(我不喜欢用 AGI 这个术语)3 将是什么样子,以及它何时(或是否)会到来,这本身就是一个巨大的话题。我已经公开讨论过这个问题,而且可以写一篇完全独立的论文(我可能在某个时候会写)。显然,很多人对强大的人工智能很怀疑,认为它很快不会被构建出来,还有些人怀疑它是否会被构建出来。我认为它最早可能在 2026 年出现,但也有可能需要更长时间。不过在这篇论文中,我想把这些问题搁置,假设它会在合理的时间内到来,然后专注于之后的 5 到 10 年会发生什么。我也想假设一下这样的系统会是什么样子,它的能力和互动方式,尽管对此可能存在分歧。
By powerful AI, I have in mind an AI model—likely similar to today’s LLM’s in form, though it
might be based on a different architecture, might involve several interacting models, and might be
trained differently—with the following properties:
通过强大的人工智能,我想到的是一个人工智能模型——可能在形式上与今天的LLM相似,尽管它可能基于不同的架构,可能涉及几个相互作用的模型,可能采用不同的训练方式,具备以下特性:
- In terms of pure intelligence4, it
is smarter than a Nobel Prize winner across most relevant fields –
biology, programming, math, engineering, writing, etc. This means it can prove unsolved mathematical
theorems, write extremely good novels, write difficult codebases from scratch, etc.
在纯粹的智力方面4,它在大多数相关领域,比如生物学、编程、数学、工程、写作等,的确比诺贝尔奖得主更聪明。这意味着它可以证明未解决的数学定理、写出极好的小说、从零开始写复杂的代码库等等。 - In addition to just being a “smart thing you talk to”, it has all the “interfaces” available to a
human working virtually, including text, audio, video, mouse and keyboard control, and internet
access. It can engage in any actions, communications, or remote operations enabled by this
interface, including taking actions on the internet, taking or giving directions to humans, ordering
materials, directing experiments, watching videos, making videos, and so on. It does all of these
tasks with, again, a skill exceeding that of the most capable humans in the world.
除了只是一个“你可以对话的智能工具”,它还具备人类在虚拟工作中可以用到的所有“接口”,包括文字、音频、视频、鼠标和键盘控制,以及互联网访问。它可以进行任何通过这个接口启用的动作、交流或远程操作,包括在网上进行操作、给人指路、订购材料、指导实验、观看视频、制作视频等等。而且,它完成所有这些任务的能力,超过了世界上最优秀的人的水平。 - It does not just passively answer questions; instead, it can be given tasks that take hours, days,
or weeks to complete, and then goes off and does those tasks autonomously, in the way a smart
employee would, asking for clarification as necessary.
它不仅仅是被动地回答问题;相反,它可以接受需要几个小时、几天或几周才能完成的任务,然后像一个聪明的员工一样,主动去完成这些任务,必要时还会请求澄清。 - It does not have a physical embodiment (other than living on a computer screen), but it can control
existing physical tools, robots, or laboratory equipment through a computer; in theory it could even
design robots or equipment for itself to use.
它没有实体形态(除了生活在电脑屏幕上),但它可以通过电脑控制现有的物理工具、机器人或实验室设备;理论上,它甚至可以为自己设计机器人或设备来使用。 - The resources used to train the model can be repurposed to run millions of instances of it
(this matches projected cluster sizes by ~2027), and the model can absorb information and generate
actions at roughly 10x-100x human speed5. It may however be limited by the response time of the physical
world or of software it interacts with.
用于训练模型的资源可以重新利用来运行它的数百万个实例(这与预计到 2027 年的集群规模大致匹配),而且该模型可以以大约 10 倍到 100 倍的人类速度吸收信息和生成行为5。不过,它可能会受到与之互动的物理世界或软件的响应时间的限制。 - Each of these million copies can act independently on unrelated tasks, or if needed can all work
together in the same way humans would collaborate, perhaps with different subpopulations fine-tuned
to be especially good at particular tasks.
这些百万个副本可以独立处理不相关的任务,或者在需要的时候,它们可以像人类一样一起协作,可能有不同的子群体,特别擅长某些特定任务。
We could summarize this as a “country of geniuses in a datacenter”.
我们可以把这总结为“数据中心里的天才国度”。
Clearly such an entity would be capable of solving very difficult problems, very fast, but it is not
trivial to figure out how fast. Two “extreme” positions both seem false to me. First, you might think
that the world would be instantly transformed on the scale of seconds or days (“the Singularity”), as superior intelligence builds on itself and solves every
possible scientific, engineering, and operational task almost immediately. The problem with this is that
there are real physical and practical limits, for example around building hardware or conducting
biological experiments. Even a new country of geniuses would hit up against these limits. Intelligence
may be very powerful, but it isn’t magic fairy dust.
显然,这样的实体能够非常快速地解决非常困难的问题,但是要弄清楚有多快并不简单。两种“极端”观点在我看来都是错误的。首先,你可能会认为,世界在几秒钟或几天内就会瞬间改变(“奇点”),因为卓越的智慧不断自我提升,几乎即时解决所有可能的科学、工程和操作任务。问题是,实际有真实的物理和实际限制,比如在制造硬件或进行生物实验方面。即使是一个天才的新国家也会遇到这些限制。智慧可能非常强大,但它不是魔法仙尘。
Second, and conversely, you might believe that technological progress is saturated or rate-limited by
real world data or by social factors, and that better-than-human intelligence will add very little6. This seems equally implausible to
me—I can think of hundreds of scientific or even social problems where a large group of really smart
people would drastically speed up progress, especially if they aren’t limited to analysis and can make
things happen in the real world (which our postulated country of geniuses can, including by directing or
assisting teams of humans).
其次,反过来说,你可能认为技术进步已经饱和,或者受到现实世界数据或社会因素的限制,而超越人类的智能也不会带来太多的帮助6。我觉得这个看法同样不太可信——我能想到数百个科学或社会问题,在这些问题上,一大群真正聪明的人会极大地加速进展,尤其是当他们不再仅仅局限于分析,而是能够在现实世界中付诸实践时(正如我们设想中的天才国家所能做到的,包括指导或协助人类团队)。
I think the truth is likely to be some messy admixture of these two extreme pictures, something that
varies by task and field and is very subtle in its details. I believe we need new frameworks to think
about these details in a productive way.
我觉得真相可能是这两种极端观点的混合,具体情况因任务和领域而异,而且细节非常微妙。我相信我们需要新的框架来以一种有效的方式思考这些细节。
Economists often talk about “factors of production”: things like labor, land, and capital. The phrase
“marginal returns to labor/land/capital” captures the idea that in a given situation, a given factor may
or may not be the limiting one – for example, an air force needs both planes and pilots, and hiring more
pilots doesn’t help much if you’re out of planes. I believe that in the AI age, we should be talking
about the marginal returns to intelligence7, and trying to figure out what the other factors are that are
complementary to intelligence and that become limiting factors when intelligence is very high. We are
not used to thinking in this way—to asking “how much does being smarter help with this task, and on what
timescale?”—but it seems like the right way to conceptualize a world with very powerful AI.
经济学家经常谈论“生产要素”:比如劳动力、土地和资本。短语“劳动力/土地/资本的边际收益”表达了这样一个观点:在特定情况下,某个要素可能是限制性因素,也可能不是——例如,空军需要飞机和飞行员,如果飞机不够,雇更多的飞行员效果也不大。我相信在人工智能时代,我们应该讨论 智力的边际收益7,并尝试找出与智力互补的其他要素,以及在智力非常高时成为限制性因素的要素。我们不习惯这样思考——问“变得更聪明在这个任务上有多大帮助,以及在什么时间范围内?”——但这似乎是构思一个拥有强大人工智能的世界的正确方式。
My guess at a list of factors that limit or are complementary to intelligence includes:
我认为限制或补充智力的因素包括:
- Speed of the outside world. Intelligent agents need to operate interactively in the
world in order to accomplish things and also to learn8. But the world only moves so fast. Cells and animals run at a fixed
speed so experiments on them take a certain amount of time which may be irreducible. The same is
true of hardware, materials science, anything involving communicating with people, and even our
existing software infrastructure. Furthermore, in science many experiments are often needed in
sequence, each learning from or building on the last. All of this means that the speed at which a
major project—for example developing a cancer cure—can be completed may have an irreducible minimum
that cannot be decreased further even as intelligence continues to increase.
外部世界的速度。智能代理需要在这个世界中互动操作,以完成任务并学习8。但是,世界的运转速度是有限的。细胞和动物的活动速度是固定的,因此对它们的实验需要一定的时间,这个时间是无法缩短的。硬件、材料科学、与人沟通的任何事情,甚至我们现有的软件基础设施,情况也是如此。此外,在科学中,许多实验往往需要逐个进行,每个实验都要从上一个实验中学习或在其基础上构建。所有这些意味着,像开发癌症治疗方法这样的重大项目的完成速度,可能会有一个无法进一步降低的最低限度,即便智能水平不断提升。 - Need for data. Sometimes raw data is lacking and in its absence more intelligence
does not help. Today’s particle physicists are very ingenious and have developed a wide range of
theories, but lack the data to choose between them because particle accelerator data is so limited. It is not clear that they would do drastically better if they
were superintelligent—other than perhaps by speeding up the construction of a bigger accelerator.
对数据的需求。有时候原始数据缺乏,缺少数据的时候,再聪明的分析也没有用。如今的粒子物理学家非常聪明,提出了很多理论,但因为粒子加速器的数据非常有限,他们无法选择其中的最佳方案。是否真能在智力超群的情况下做出显著改善也不清楚——也许只是能加快建设更大加速器的速度。 - Intrinsic complexity. Some things are inherently unpredictable or chaotic and even
the most powerful AI cannot predict or untangle them substantially better than a human or a computer
today. For example, even incredibly powerful AI could predict only marginally further ahead in a
chaotic system (such as the three-body problem) in the general case,9 as compared to today’s humans and computers.
内在复杂性。有些事物本质上是不可预测或混沌的,甚至最强大的人工智能也无法比今天的人类或计算机更好地预测或理清这些事物。例如,尽管人工智能非常强大,但在一般情况下,它只能在混沌系统(比如三体问题)中比现在的人类和计算机稍微预测得更远一些9。 - Constraints from humans. Many things cannot be done without breaking laws, harming
humans, or messing up society. An aligned AI would not want to do these things (and if we have an
unaligned AI, we’re back to talking about risks). Many human societal structures are inefficient or
even actively harmful, but are hard to change while respecting constraints like legal requirements
on clinical trials, people’s willingness to change their habits, or the behavior of governments.
Examples of advances that work well in a technical sense, but whose impact has been substantially
reduced by regulations or misplaced fears, include nuclear power, supersonic flight, and even elevators.
来自人类的限制。很多事情在不违反法律、伤害人类或扰乱社会的情况下是无法做到的。一个对齐的人工智能不会想要做这些事情(如果我们有一个不对齐的人工智能,那么就又要谈论风险了)。很多人类社会结构效率低下,甚至是有害的,但在尊重法律要求、人民改变习惯的意愿或政府行为等限制的情况下,很难改变。那些在技术上运作良好,但因法规或错误担忧而影响大大降低的进展例子包括核能、超音速飞行和甚至电梯。 - Physical laws. This is a starker version of the first point. There are certain
physical laws that appear to be unbreakable. It’s not possible to travel faster than light. Pudding does not unstir.
Chips can only have so many transistors per square centimeter before they become
unreliable. Computation requires a certain minimum
energy per bit erased, limiting the density of computation in the world.
物理定律。这是第一点的一个更强烈的版本。有些物理定律似乎是不可打破的。以超过光速的速度旅行是不可能的。布丁无法再搅拌。每平方厘米的芯片只能有这么多晶体管 否则它们会变得不可靠。计算需要 每位擦除一定的最小能量,这限制了世界上计算的密度。
There is a further distinction based on timescales. Things that are hard constraints in the short
run may become more malleable to intelligence in the long run. For example, intelligence might be used
to develop a new experimental paradigm that allows us to learn in vitro what used to require live
animal experiments, or to build the tools needed to collect new data (e.g. the bigger particle
accelerator), or to (within ethical limits) find ways around human-based constraints (e.g. helping to
improve the clinical trial system, helping to create new jurisdictions where clinical trials have less
bureaucracy, or improving the science itself to make human clinical trials less necessary or cheaper).
基于时间尺度,还可以进一步区分。短期内一些严格的约束可能在长期内变得对智能更加灵活。例如,智能可能被用来开发一种新的实验范式,让我们可以在体外学习以前需要活体动物实验的知识,或者构建收集新数据所需的工具(比如更大的粒子加速器),或者在伦理限度内寻找绕过人类约束的方法(例如,帮助改善临床试验系统,帮助创建减少繁文缛节的新管辖区,或者提高科学本身,使得人类临床试验变得不那么必要或便宜)。
Thus, we should imagine a picture where intelligence is initially heavily bottlenecked by the other
factors of production, but over time intelligence itself increasingly routes around the other factors,
even if they never fully dissolve (and some things like physical laws are absolute)10. The key question is how fast it all happens and
in what order.
因此,我们应该想象一个这样的画面:最初,智力受到其他生产要素的严重制约,但随着时间的推移,智力本身逐渐绕过这些其他要素,即使它们从未完全消失(有些东西,比如物理法则,是绝对的)10。关键问题是这一切发生的速度和顺序。
With the above framework in mind, I’ll try to answer that question for the five areas mentioned in the
introduction.
考虑到上述框架,我将尝试回答引言中提到的五个领域的问题。
1. Biology and health 生物和健康
Biology is probably the area where scientific progress has the greatest potential to directly and
unambiguously improve the quality of human life. In the last century some of the most ancient human
afflictions (such as smallpox) have finally been vanquished, but many more still remain, and defeating
them would be an enormous humanitarian accomplishment. Beyond even curing disease, biological science
can in principle improve the baseline quality of human health, by extending the healthy human
lifespan, increasing control and freedom over our own biological processes, and addressing everyday
problems that we currently think of as immutable parts of the human condition.
生物学很可能是科学进步在直接和明确地提高人类生活质量方面潜力最大的领域。在上个世纪,一些最古老的人类疾病(比如天花)终于被征服,但还有许多疾病依然存在,战胜它们将是一个巨大的 humanitarian 成就。除了治愈疾病,生物科学原则上还可以改善人类健康的基线质量,方法是延长健康的人类寿命、增加我们对自身生物过程的控制和自由,并解决我们目前认为是人类状态不可改变的日常问题。
In the “limiting factors” language of the previous section, the main challenges with directly applying
intelligence to biology are data, the speed of the physical world, and intrinsic complexity (in fact,
all three are related to each other). Human constraints also play a role at a later stage, when clinical
trials are involved. Let’s take these one by one.
在上一节提到的“限制因素”中,直接将智能应用于生物学的主要挑战有数据、物理世界的速度以及本身的复杂性(实际上,这三者是相互关联的)。人类的限制在后期也起着作用,尤其是在涉及临床试验的时候。我们一个一个来看这些问题。
Experiments on cells, animals, and even chemical processes are limited by the speed of the physical
world: many biological protocols involve culturing bacteria or other cells, or simply waiting for
chemical reactions to occur, and this can sometimes take days or even weeks, with no obvious way to
speed it up. Animal experiments can take months (or more) and human experiments often take years (or
even decades for long-term outcome studies). Somewhat related to this, data is often lacking—not so much
in quantity, but quality: there is always a dearth of clear, unambiguous data that isolates a biological
effect of interest from the other 10,000 confounding things that are going on, or that intervenes
causally in a given process, or that directly measures some effect (as opposed to inferring its
consequences in some indirect or noisy way). Even massive, quantitative molecular data, like the
proteomics data that I collected while working on mass spectrometry techniques, is noisy and misses a
lot (which types of cells were these proteins in? Which part of the cell? At what phase in the cell
cycle?).
在细胞、动物甚至化学过程的实验中,物理世界的速度是有限制的:很多生物实验都涉及培养细菌或其他细胞,或者只是等化学反应发生,这有时候可能需要几天甚至几周,而没有明显的方法可以加快速度。动物实验可能需要几个月(或更久),而人类实验通常需要几年(甚至长期结果研究可能要几十年)。与此有些关联的是,数据往往缺乏——不是数量上的缺乏,而是质量上的缺乏:总是缺少清晰、不模糊的数据,能够将感兴趣的生物效应与其他一万种干扰因素区分开,或者能够因果干预某个过程,或者能够直接测量某种效果(而不是以某种间接或嘈杂的方式推断其后果)。即使是我在进行质谱技术工作时收集的大量定量分子数据,比如蛋白质组数据,也都是嘈杂的,且遗漏了很多信息(这些蛋白质属于哪种细胞?细胞的哪个部分?在细胞周期的哪个阶段?)。
In part responsible for these problems with data is intrinsic complexity: if you’ve ever seen a diagram showing the biochemistry of human metabolism, you’ll know that it’s very
hard to isolate the effect of any part of this complex system, and even harder to intervene on the
system in a precise or predictable way. And finally, beyond just the intrinsic time that it takes to run
an experiment on humans, actual clinical trials involve a lot of bureaucracy and regulatory requirements
that (in the opinion of many people, including me) add unnecessary additional time and delay progress.
部分原因在于数据本身的复杂性:如果你曾经见过一个展示人类新陈代谢生物化学的图示,你就会知道很难孤立出这个复杂系统中任何部分的影响,而要以精确或可预测的方式干预这个系统就更困难。最后,除了进行人类实验所需的内在时间外,实际的临床试验还涉及大量的官僚主义和监管要求,这些在许多人(包括我)看来,增加了不必要的额外时间,延缓了进展。
Given all this, many biologists have long been skeptical of the value
of AI and “big data” more generally in biology. Historically, mathematicians, computer scientists, and
physicists who have applied their skills to biology over the last 30 years have been quite successful,
but have not had the truly transformative impact initially hoped for. Some of the skepticism has been
reduced by major and revolutionary breakthroughs like AlphaFold (which has just deservedly won its creators the Nobel Prize in
Chemistry) and AlphaProteo11, but there’s still a perception that AI is (and will continue to be)
useful in only a limited set of circumstances. A common formulation is “AI can do a better job analyzing
your data, but it can’t produce more data or improve the quality of the data. Garbage in, garbage out”.
鉴于这一切,许多生物学家长期以来对 AI 和“海量数据”在生物学中的价值持怀疑态度。历史上,过去 30 年里将自己的技能应用于生物学的数学家、计算机科学家和物理学家取得了相当成功,但并没有达到最初期望的真正变革性影响。像AlphaFold(它刚刚理所当然地为其创造者赢得了诺贝尔化学奖)和AlphaProteo11这样的重大和革命性突破减少了一些怀疑,但人们仍然认为 AI 在有限的情况下是有用的(并将继续如此)。一个常见的说法是“AI 可以更好地分析你的数据,但它无法生成更多数据或改善数据的质量。垃圾进,垃圾出。”
But I think that pessimistic perspective is thinking about AI in the wrong way. If our core hypothesis
about AI progress is correct, then the right way to think of AI is not as a method of data analysis, but
as a virtual biologist who performs all the tasks biologists do, including designing and running
experiments in the real world (by controlling lab robots or simply telling humans which experiments to
run – as a Principal Investigator would to their graduate students), inventing new biological methods or
measurement
techniques, and so on. It is by speeding up the whole research process that AI can truly
accelerate biology. I want to repeat this because it’s the most common misconception that comes
up when I talk about AI’s ability to transform biology: I am not talking about AI as merely a
tool to analyze data. In line with the definition of powerful AI at the beginning of this essay, I’m
talking about using AI to perform, direct, and improve upon nearly everything biologists
do.
但我觉得这种悲观的观点是错误地看待人工智能。如果我们关于人工智能进展的核心假设是正确的,那么正确理解人工智能的方式不是将其视为数据分析的方法,而是看作一个虚拟的生物学家,执行生物学家所做的所有任务,包括在真实世界中设计和进行实验(通过控制实验室机器人或简单地告诉人类应该进行哪些实验——就像首席研究员对他们的研究生所做的那样),发明新的生物方法或测量技术等等。正是通过加速整个研究过程,人工智能才能真正推动生物学的发展。我想再次强调这一点,因为这是我谈论人工智能转变生物学能力时最常见的误解:我并不是仅仅在谈论人工智能作为一种分析数据的工具。根据本文开头对强大人工智能的定义,我是在谈论利用人工智能执行、指导并完善生物学家所做的几乎所有事情。
To get more specific on where I think acceleration is likely to come from, a surprisingly large fraction
of the progress in biology has come from a truly tiny number of discoveries, often related to broad
measurement tools or techniques12
that allow precise but generalized or programmable intervention in biological systems. There’s perhaps
~1 of these major discoveries per year and collectively they arguably drive >50% of progress in biology.
These discoveries are so powerful precisely because they cut through intrinsic complexity and data
limitations, directly increasing our understanding and control over biological processes. A few
discoveries per decade have enabled both the bulk of our basic scientific understanding of biology, and
have driven many of the most powerful medical treatments.
更具体地说,我认为加速的来源在哪里,一个令人意外的事实是,生物学进展中相当大一部分来源于极少数的发现,这些发现通常与广泛的测量工具或技术有关12,它们可以在生物系统中进行精确但是通用或可编程的干预。每年大约有~1 个这样的重大发现,集体来看,他们驱动了超过 50%的生物学进展。之所以这些发现如此强大,是因为它们突破了内在的复杂性和数据限制,直接增加了我们对生物过程的理解和控制。每十年几个发现让我们对生物学的基本科学理解得以实现,并推动了许多最强大的医疗治疗。
Some examples include: 一些例子包括:
- CRISPR: a technique that allows
live editing of any gene in living organisms (replacement of any arbitrary gene sequence with any
other arbitrary sequence). Since the original technique was developed, there have been constant
improvements to target specific cell types, increasing accuracy, and reducing edits of the
wrong gene—all of which are needed for safe use in humans.
CRISPR:一种可以在活体生物中实时编辑任何基因的技术(可以将任意基因序列替换为其他任意序列)。自从最初的技术开发以来,针对特定细胞类型的不断改进已经越来越多,提升了准确性,并减少了错误编辑基因的情况——这些都是安全应用于人类所必需的。 - Various kinds of microscopy for watching what is going on at a precise level: advanced light
microscopes (with various kinds of fluorescent techniques, special optics, etc), electron
microscopes, atomic force microscopes, etc.
各种显微镜用于观察精细层面的情况:高级光学显微镜(配备各种荧光技术、特殊光学等)、电子显微镜、原子力显微镜等。 - Genome sequencing and synthesis, which has dropped in cost by several orders of magnitude in the last couple decades.
基因组测序和合成的成本在过去二十年里已经降了好几个数量级。 - Optogenetic techniques that allow you to get a neuron to fire by shining a
light on it.
光遗传学技术可以通过照射光线让神经元发火。 - mRNA vaccines that, in
principle, allow us to design a vaccine against anything and then quickly adapt it (mRNA vaccines of
course became famous during COVID).
mRNA 疫苗原则上可以让我们设计出针对任何事物的疫苗,并迅速进行调整(mRNA 疫苗当然在疫情期间声名鹊起)。 - Cell therapies such as CAR-T
that allow immune cells to be taken out of the body and “reprogrammed” to attack, in principle,
anything.
像CAR-T这样的细胞疗法可以把免疫细胞取出体外,“重新编程”后攻击原则上任何东西。 - Conceptual insights like the germ theory of disease or the realization of a link between the immune
system and cancer13.
疾病的致病微生物理论或免疫系统与癌症之间的联系这种概念性见解13。
I’m going to the trouble of listing all these technologies because I want to make a crucial claim about
them: I think their rate of discovery could be increased by 10x or more if there were a lot more
talented, creative researchers. Or, put another way, I think the returns to
intelligence are high for these discoveries, and that everything else in biology and
medicine mostly follows from them.
我列出所有这些技术是因为我想提出一个关键的观点:我认为如果有更多有才华、富有创造力的研究人员,他们的发现速度可以提高 10 倍或更多. 换句话说,我认为这些发现的智慧回报是很高的,而生物学和医学中的其他一切大多都是由这些发现引申而来的。
Why do I think this? Because of the answers to some questions that we should get in the habit of asking
when we’re trying to determine “returns to intelligence”. First, these discoveries are generally made by
a tiny number of researchers, often the same people repeatedly, suggesting skill and not random search
(the latter might suggest lengthy experiments are the limiting factor). Second, they often “could have
been made” years earlier than they were: for example, CRISPR was a naturally occurring component of the
immune system in bacteria that’s been known since the
80’s, but it took another 25 years for people to realize it could be repurposed for general gene
editing. They also are often delayed many years by lack of support from the scientific community for
promising directions (see this profile on the inventor of mRNA vaccines; similar stories abound). Third,
successful projects are often scrappy or were afterthoughts that people didn’t initially think were
promising, rather than massively funded efforts. This suggests that it’s not just massive resource
concentration that drives discoveries, but ingenuity.
我为什么这么认为?因为在我们试图判断“智力的回报”时,应该养成问一些问题的习惯。首先,这些发现通常是由极少数研究人员做出的,常常是同一群人反复发现,这表明是技能而不是随机搜索(后者可能表明实验的时间长短是限制因素)。其次,这些发现往往“可以在”它们被发现的几年之前就产生:例如,CRISPR 是细菌免疫系统中的一种自然成分,自 80 年代以来就已知,但人们直到又过了 25 年才意识到它可以被重新利用进行一般基因编辑。此外,由于科学界对有前途的方向缺乏支持,这些发现也常常被延迟多年(见这篇关于 mRNA 疫苗发明者的报道;类似的故事屡见不鲜)。第三,成功的项目往往是琐碎的或是被认为没有前景的“事后想法”,而不是大量资金资助的努力。这表明,推动发现的并不仅仅是资源的集中,还有创造力。
Finally, although some of these discoveries have “serial dependence” (you need to make discovery A first
in order to have the tools or knowledge to make discovery B)—which again might create experimental
delays—many, perhaps most, are independent, meaning many at once can be worked on in parallel. Both
these facts, and my general experience as a biologist, strongly suggest to me that there are hundreds of
these discoveries waiting to be made if scientists were smarter and better at making connections between
the vast amount of biological knowledge humanity possesses (again consider the CRISPR example). The
success of AlphaFold/AlphaProteo at solving important problems much more effectively than humans,
despite decades of carefully designed physics modeling, provides a proof of principle (albeit with a
narrow tool in a narrow domain) that should point the way forward.
最后,尽管这些发现中有一些存在“序列依赖性”(你需要先进行发现 A,才能拥有进行发现 B 的工具或知识)——这可能会导致实验延期——但许多,或许大多数,是独立的,意味着许多发现可以同时并行进行。这两个事实,加上我作为生物学家的整体经验,强烈让我觉得,如果科学家们更聪明、能更好地将大量生物知识联系起来,那么还有数百个这样的发现在等待被实现(再考虑一下 CRISPR 的例子)。AlphaFold/AlphaProteo在解决重要问题上,远比人类更有效,尽管经历了几十年的精心设计的物理建模,这提供了一个原则性的证明(尽管只是在狭窄领域内使用了狭隘的工具),这应该指引未来的方向。
Thus, it’s my guess that powerful AI could at least 10x the rate of these discoveries, giving us the next
50-100 years of biological progress in 5-10 years.14 Why not 100x? Perhaps it is possible, but here both serial dependence
and experiment times become important: getting 100 years of progress in 1 year requires a lot of things
to go right the first time, including animal experiments and things like designing microscopes or
expensive lab facilities. I’m actually open to the (perhaps absurd-sounding) idea that we could get
1000 years of progress in 5-10 years, but very skeptical that we can get 100 years in 1 year.
Another way to put it is I think there’s an unavoidable constant delay: experiments and hardware design
have a certain “latency” and need to be iterated upon a certain “irreducible” number of times in order
to learn things that can’t be deduced logically. But massive parallelism may be possible on top of
that15.
因此,我猜强大的人工智能至少能将这些发现的速度提高 10 倍,让我们在 5 到 10 年内取得未来 50 到 100 年的生物学进步。14 为什么不是 100 倍呢?也许确实有可能,但在这个过程中,串行依赖和实验时间变得非常重要:在 1 年内取得 100 年的进步,需要很多事情第一次就顺利进行,包括动物实验,以及设计显微镜或昂贵实验室设施等。我其实对(也许听起来有点荒谬的)想法持开放态度,认为我们可以在 5 到 10 年内获得1000年的进步,但对能在 1 年内获得 100 年的进步持怀疑态度。换句话说,我认为有一个不可避免的恒定延迟:实验和硬件设计有一定的“延迟”,需要迭代一定的“不可减少”次数,以便学习那些不能通过逻辑推导出来的东西。但在此基础上,巨大的并行性可能是可行的。15
What about clinical trials? Although there is a lot of bureaucracy and slowdown associated with them, the
truth is that a lot (though by no means all!) of their slowness ultimately derives from the need to
rigorously evaluate drugs that barely work or ambiguously work. This is sadly true of most therapies
today: the average cancer drug increases survival by a few months while having significant side effects
that need to be carefully measured (there’s a similar story for Alzheimer’s drugs). This leads to huge
studies (in order to achieve statistical power) and difficult tradeoffs which regulatory agencies
generally aren’t great at making, again because of bureaucracy and the complexity of competing
interests.
临床试验怎么样?虽然涉及很多官僚主义和拖延,但事实是,许多(但绝不是全部!)的拖延最终源于必须严格评估那些效果微弱或效果模糊的药物。可悲的是,这对于现在大多数疗法都是如此:一般的癌症药物只能延长几个月的生命,同时伴随需要仔细衡量的显著副作用(阿尔茨海默症药物也有类似的情况)。这导致进行大规模研究(以获得统计意义),同时也带来了困难的取舍,而监管机构通常很难做出这些决策,同样是因为官僚主义和利益复杂的因素。
When something works really well, it goes much faster: there’s an accelerated approval track and the ease
of approval is much greater when effect sizes are larger. mRNA vaccines for COVID were approved in 9
months—much faster than the usual pace. That said, even under these conditions clinical trials are still
too slow—mRNA vaccines arguably should have
been approved in ~2 months. But these kinds of delays (~1 year end-to-end for a drug) combined
with massive parallelization and the need for some but not too much iteration (“a few tries”) are very
compatible with radical transformation in 5-10 years. Even more optimistically, it is possible that AI-enabled biological science will reduce the need for iteration in clinical
trials by developing better animal and cell experimental models (or even simulations) that are more
accurate in predicting what will happen in humans. This will be particularly important in developing
drugs against the aging process, which plays out over decades and where we need a faster iteration loop.
当事情进展得很好时,速度就快得多:有一个加速审批通道,而当效果更明显时,审批的便利性也大大提升。COVID 的 mRNA 疫苗在 9 个月内获得批准——远远快于通常的速度。尽管如此,即使在这些条件下,临床试验仍然太慢——可以说 mRNA 疫苗应该在大约 2 个月内获得批准。但这种延误(药物整个流程约 1 年)再加上大规模并行和需要进行一些但又不过多的迭代(“尝试几次”)非常适合在 5-10 年内进行彻底变革。更乐观地说,人工智能驱动的生物科学可能会通过开发更好的动物和细胞实验模型(甚至模拟)来减少临床试验中的迭代需求,这些模型在预测人类将发生什么方面更为准确。这在开发抗衰老药物时尤其重要,因为衰老过程是几十年展开的,而我们需要一个更快的迭代循环。
Finally, on the topic of clinical trials and societal barriers, it is worth pointing out explicitly that
in some ways biomedical innovations have an unusually strong track record of being successfully
deployed, in contrast to some other technologies16. As mentioned in the introduction, many technologies are hampered by
societal factors despite working well technically. This might suggest a pessimistic perspective on what
AI can accomplish. But biomedicine is unique in that although the process of developing drugs is
overly cumbersome, once developed they generally are successfully deployed and used.
最后,关于临床试验和社会障碍的话题,值得明确指出的是,在某些方面,生物医学创新在成功应用方面有着异常强的记录,与其他一些技术形成对比16。正如在引言中提到的,许多技术尽管在技术上运行良好,却受到社会因素的阻碍。这可能对人工智能能做到什么产生悲观的看法. 但生物医学是独特的,尽管开发药物的过程过于繁琐,但一旦开发完成,通常都能成功应用和使用。
总的来说,我的基本预测是,AI 驱动的生物学和医学将让我们在 5 到 10 年内实现人类生物学家在接下来的 50 到 100 年内才能取得的进展。我称之为“压缩的 21 世纪”:也就是说,在强大的 AI 开发出来后,我们在短短几年的时间里会在生物学和医学方面取得 21 世纪整体的所有进展。
Although predicting what powerful AI can do in a few years remains inherently difficult and
speculative,
there is some concreteness to asking “what could humans do unaided in the next 100 years?”. Simply
looking at what we’ve accomplished in the 20th century, or extrapolating from the first 2 decades of
the
21st, or asking what “10 CRISPR’s and 50 CAR-T’s” would get us, all offer practical, grounded ways
to
estimate the general level of progress we might expect from powerful AI.
虽然预测强大的人工智能在未来几年能做什么仍然 inherently 难以把握且充满不确定性,但问“在接下来的 100 年里人类在没有帮助的情况下能做什么?”还是有些具体的。只要看看我们在 20 世纪的成就,或从 21 世纪的头 20 年推测,或询问“10 个 CRISPR 和 50 个 CAR-T 能够带来什么”,这些都是一些实在的方法,可以帮助我们估计强大人工智能可能带来的整体进步水平。
Below I try to make a list of what we might expect. This is not based on any rigorous methodology,
and
will almost certainly prove wrong in the details, but it’s trying to get across the general
level
of radicalism we should expect:
以下我试着列出我们可能期待的内容。这并不是基于任何严格的方法论,细节上几乎肯定会出错,但这试图传达我们应该期待的激进主义的一般水平:
- Reliable prevention and treatment of nearly all17 natural infectious
disease.
Given the enormous advances against infectious disease in the 20th century, it is not radical to
imagine that we could more or less “finish the job” in a compressed 21st. mRNA vaccines and
similar
technology already point the way towards “vaccines for anything”. Whether infectious disease is fully
eradicated
from the world (as opposed to just in some places) depends on questions about poverty
and
inequality, which are discussed in Section 3.
几乎所有自然传染病的可靠预防和治疗。17考虑到 20 世纪在抗击传染病方面的巨大进步,想象一下我们在压缩的 21 世纪可以或多或少“完成这项工作”并不是激进的想法。mRNA 疫苗和类似技术已经指明了“任何疾病的疫苗”的方向。传染病是否会被完全消除在世界范围内(而不仅仅是在某些地方),取决于关于贫困和不平等的问题,这在第 3 节中进行了讨论。 - Elimination of most cancer. Death rates from cancer have been dropping
~2%
per year for the last few decades; thus we are on track to eliminate most cancer in the
21st
century at the current pace of human science. Some subtypes have already been largely cured (for
example some types of leukemia with CAR-T therapy), and I’m perhaps even more excited for very selective
drugs
that target cancer in its infancy and prevent it
from ever growing. AI will also make possible treatment regimens very finely adapted to the individualized genome of the cancer—these are
possible
today, but hugely expensive in time and human expertise, which AI should allow us to scale.
Reductions of 95% or more in both mortality and incidence seem possible. That said, cancer is
extremely varied and adaptive, and is likely the hardest of these diseases to fully destroy. It
would not be surprising if an assortment of rare, difficult malignancies persists.
大多数癌症的消除。过去几十年里,癌症的死亡率每年下降约 2%;因此,按照目前人类科学的发展速度,我们有望在 21 世纪消除大多数癌症。一些亚型已经基本治愈(例如某些类型的白血病通过CAR-T 疗法),我甚至对那些非常针对癌症早期阶段的特异性药物感到更加兴奋,这些药物能防止癌症生长。人工智能也将使得治疗方案能够非常精确地适应患者的个体化基因组——这些今天是可行的,但在时间和人类专业知识上都非常昂贵,人工智能应该能帮助我们实现规模化。死亡率和发病率降低 95%或更多似乎是可能的。不过,癌症种类繁多且具适应性,可能是这些疾病中最难以完全消灭的。如果一些稀有且难治的恶性肿瘤持续存在,也并不令人惊讶。 - Very effective prevention and effective cures for genetic disease. Greatly
improved
embryo
screening
will likely make it possible to prevent most genetic disease, and some safer, more reliable
descendant of CRISPR may cure most genetic disease in existing people. Whole-body afflictions
that
affect a large fraction of cells may be the last holdouts, however.
非常有效的遗传病预防和治疗方法。大幅改进的胚胎筛查可能使预防大多数遗传病成为可能,而一些更安全、更可靠的 CRISPR 后代可能会治愈现存的大多数遗传病。可是,影响大量细胞的全身性疾病可能是最后的难关。 - Prevention of Alzheimer’s. We’ve had a very hard time figuring out what causes
Alzheimer’s (it is somehow related to beta-amyloid protein, but the actual details seem to be very complex).
It
seems like exactly the type of problem that can be solved with better measurement tools that
isolate
biological effects; thus I am bullish about AI’s ability to solve it. There is a good chance it
can
eventually be prevented with relatively simple interventions, once we actually understand what
is
going on. That said, damage from already-existing Alzheimer’s may be very difficult to reverse.
阿尔茨海默病的预防。我们在弄清楚阿尔茨海默病的成因上遇到了很大的困难(它与β-淀粉样蛋白有某种关系,但具体细节似乎是非常复杂的)。这看起来就像一个可以通过更好的测量工具来解决的问题,这些工具能够隔离生物效应;因此,我对 AI 解决这个问题的能力持乐观态度。一旦我们真正理解发生了什么,很可能可以通过相对简单的干预措施来预防阿尔茨海默病。不过,已经存在的阿尔茨海默病造成的损害可能会非常难以逆转。 - Improved treatment of most other ailments. This is a catch-all category for
other
ailments including diabetes, obesity, heart disease, autoimmune diseases, and more. Most of
these
seem “easier” to solve than cancer and Alzheimer’s and in many cases are already in steep
decline.
For example, deaths from heart disease have already declined over 50%, and simple interventions
like
GLP-1 agonists have already made huge progress against obesity and
diabetes.
改善大多数其他疾病的治疗。这是一个涵盖其他疾病的综合类别,包括糖尿病、肥胖、心脏病、自身免疫疾病等。大多数这些疾病似乎比癌症和阿尔茨海默症“更容易”解决,而且在很多情况下,已经大幅下降。例如,心脏病的死亡率已经下降超过 50%,而像GLP-1 受体激动剂这样的简单干预已经在对抗肥胖和糖尿病方面取得了巨大的进展。 - Biological freedom. The last 70 years featured advances in birth control,
fertility, management of weight, and much more. But I suspect AI-accelerated
biology
will greatly expand what is possible: weight, physical appearance, reproduction, and other
biological processes will be fully under people’s control. We’ll refer to these under the
heading of
biological freedom: the idea that everyone should be empowered to choose what they want
to
become and live their lives in the way that most appeals to them. There will of course be
important
questions about global equality of access; see Section 3 for these.
生物自由。过去 70 年里,避孕、受孕、体重管理等方面都有了很大的进展。不过,我怀疑加速的人工智能生物学会极大地扩展可能性:体重、外貌、繁殖和其他生物过程将完全由人们控制。我们将把这些称为 生物自由: 这个理念是每个人都应该有能力选择自己想要成为什么,活出最吸引他们的生活。当然,关于全球平等获取的重要问题也会存在;请参见第 3 节了解这些。 - Doubling of the human lifespan18.This might seem radical,
but life expectancy increased
almost 2x in the 20th century (from ~40 years to ~75), so it’s “on trend” that the
“compressed 21st” would double it again to 150. Obviously the interventions involved in slowing
the
actual aging process will be different from those that were needed in the last century to
prevent
(mostly childhood) premature deaths from disease, but the magnitude of change is not
unprecedented19. Concretely,
there already exist
drugs that increase maximum lifespan in rats by 25-50% with limited ill-effects. And
some
animals (e.g. some types of turtle) already live 200 years, so humans are manifestly not at some
theoretical upper limit. At a guess, the most important thing that is needed might be reliable,
non-Goodhart-able
biomarkers of human aging, as that will allow fast iteration on experiments and clinical trials.
Once human lifespan is 150, we may be able to reach “escape velocity”, buying enough time that
most
of those currently alive today will be able to live as long as they want, although there’s
certainly
no guarantee this is biologically possible.
人类寿命的翻倍18.这听起来可能有点激进,但20 世纪的预期寿命几乎增加了 2 倍(从大约 40 岁增至约 75 岁),因此“压缩型 21 世纪”再翻倍到 150 岁也是“符合趋势”的。显然,减缓实际衰老过程所需的干预措施将与上个世纪需要的防止(主要是儿童)因疾病而过早死亡的措施不同,但变化的幅度并非前所未有19。具体来说,目前已经存在一些药物,可以使老鼠的最大寿命增加 25%-50%,且副作用有限。而某些动物(例如某些类型的乌龟)已经能活到 200 岁,所以人类显然并没有达到某个理论上的上限。猜测一下,最重要的可能是可靠的非 Goodhart 指标的人类衰老生物标志物,因为这将允许快速迭代实验和临床试验。 一旦人类的寿命达到 150 岁,我们可能就能达到“逃逸速度”,争取到足够的时间,让目前活着的大多数人能够活到自己想活的年纪,虽然这在生物学上肯定没有保证可行。
It is worth looking at this list and reflecting on how different the world will be if all of it is
achieved 7-12 years from now (which would be in line with an aggressive AI timeline). It goes
without
saying that it would be an unimaginable humanitarian triumph, the elimination all at once of most of
the
scourges that have haunted humanity for millennia. Many of my friends and colleagues are raising
children, and when those children grow up, I hope that any mention of disease will sound to them the
way
scurvy, smallpox, or bubonic
plague
sounds to us. That generation will also benefit from increased biological freedom and
self-expression,
and with luck may also be able to live as long as they want.
值得一看这个清单,并思考如果在 7-12 年后(这与激进的 AI 时间表相符)实现这一切,世界会有多么不同。毫无疑问,这将是一个不可想象的人道主义胜利,一举消除大多数困扰人类数千年的祸害。我的许多朋友和同事都在养育孩子,当这些孩子长大后,我希望任何提到疾病的词语对他们来说,就像我们听到的坏血病、天花或鼠疫一样。那一代人也将受益于更高的生物自由和自我表达,而幸运的话,也能够活得久一点。
It’s hard to overestimate how surprising these changes will be to everyone except the small community
of
people who expected powerful AI. For example, thousands of economists and policy experts in the US
currently debate how to keep Social Security and Medicare solvent, and more broadly how to
keep
down the cost of healthcare (which is mostly consumed by those over 70 and especially those with
terminal illnesses such as cancer). The situation for these programs is likely to be radically
improved
if all this comes to pass20, as
the
ratio of working age to retired population will change drastically. No doubt these challenges will
be
replaced with others, such as how to ensure widespread access to the new technologies, but it is
worth
reflecting on how much the world will change even if biology is the only area to be
successfully
accelerated by AI.
很难高估这些变化对除了那个小小的预期强大 AI 的社区之外的每个人来说会有多么惊人。例如,当前在美国,成千上万的经济学家和政策专家正在辩论如何保持社保和医疗保险的财务稳定,更广泛地来说,是如何降低医疗成本(这些成本主要由 70 岁以上的人群,特别是那些患有癌症等绝症的人群消费)。如果这一切都能实现,这些项目的情况极有可能会大幅改善,因为工作年龄人口与退休人口的比例将会发生剧烈变化。毫无疑问,这些挑战将会被其他挑战取代,例如如何确保新技术的广泛获取,但即使生物学是唯一得到成功加速的领域,也值得我们反思世界将会有多大的变化。
2. Neuroscience and mind 2. 神经科学与心智
In the previous section I focused on physical diseases and biology in general, and didn’t
cover
neuroscience or mental health. But neuroscience is a subdiscipline of biology and mental health is
just
as important as physical health. In fact, if anything, mental health affects human well-being even
more
directly than physical health. Hundreds of millions of people have very low quality of life due to
problems like addiction, depression, schizophrenia, low-functioning autism, PTSD, psychopathy21, or intellectual disabilities.
Billions more struggle with everyday problems that can often be interpreted as much milder versions
of
one of these severe clinical disorders. And as with general biology, it may be possible to go beyond
addressing problems to improving the baseline quality of human experience.
在前一部分,我主要关注了身体疾病和生物学的总体情况,并没有涉及神经科学或心理健康。不过,神经科学是生物学的一个子学科,而心理健康和身体健康同样重要。事实上,心理健康对人类幸福感的影响甚至比身体健康更直接。由于成瘾、抑郁、精神分裂症、低功能自闭症、创伤后压力症状、反社会人格21或智力障碍等问题,数亿人的生活质量非常低。还有数十亿人面临着日常问题,这些问题通常可以被看作是这些严重临床障碍更轻微的版本。和一般生物学一样,可能不仅仅是解决问题,而是提高人类体验的基础质量。
The basic framework that I laid out for biology applies equally to neuroscience. The field is
propelled
forward by a small number of discoveries often related to tools for measurement or precise
intervention
– in the list of those above, optogenetics was a neuroscience discovery, and more recently CLARITY and expansion microscopy are
advances
in the same vein, in addition to many of the general cell biology methods directly carrying over to
neuroscience. I think the rate of these advances will be similarly accelerated by AI and therefore
that
the framework of “100 years of progress in 5-10 years” applies to neuroscience in the same way it
does
to biology and for the same reasons. As in biology, the progress in 20th century neuroscience was
enormous – for example we didn’t even understand how or why neurons fired until the
1950’s. Thus, it seems reasonable to expect AI-accelerated neuroscience to produce rapid
progress over a few years.
我为生物学奠定的基本框架同样适用于神经科学。这个领域的进展主要依靠少数发现,这些发现通常与测量工具或精准干预有关——在上面的清单中,光遗传学就是一个神经科学的发现,最近的CLARITY和扩展显微镜也是类似的进展,此外许多通用细胞生物学的方法也直接转用于神经科学。我认为这些进展的速度会同样受到人工智能的推动,因此“100 年的进步在 5-10 年内实现”的框架同样适用于神经科学,原因也与生物学相同。就像在生物学中一样,20 世纪神经科学的进展也是巨大的——例如,我们甚至在 1950 年代之前都不理解神经元是如何或者为什么发射的。 因此,合理的预期是,借助人工智能加速的神经科学将在短短几年内实现快速进展。
There is one thing we should add to this basic picture, which is that some of the things we’ve
learned
(or are learning) about AI itself in the last few years are likely to help advance neuroscience,
even if
it continues to be done only by humans. Interpretability is an obvious example: although biological neurons
superficially operate in a completely different manner from artificial neurons (they communicate via
spikes and often spike rates, so there is a time element not present in artificial neurons, and a
bunch
of details relating to cell physiology and neurotransmitters modifies their operation
substantially),
the basic question of “how do distributed, trained networks of simple units that perform combined
linear/non-linear operations work together to perform important computations” is the same, and I
strongly suspect the details of individual neuron communication will be abstracted away in most of
the
interesting questions about computation and circuits22. As just one example of this, a computational
mechanism discovered by interpretability researchers in AI systems was recently rediscovered
in
the brains of mice.
我们还应该在这个基本画面上补充一点,那就是在过去几年中我们对 AI 本身学到(或正在学习)的一些东西,可能有助于推进神经科学,即使这项工作仍然只由人类进行。可解释性就是一个明显的例子:尽管生物神经元在表面上与人工神经元的运作方式完全不同(它们通过脉冲进行通信,通常是脉冲频率,因此存在一个人工神经元所没有的时间因素,以及与细胞生理和神经递质相关的一些细节会实质性地改变它们的运作),但是“分布式、经过训练的简单单元网络是如何协同工作以执行重要计算的”这个基本问题是相同的。我强烈怀疑,关于计算和电路的很多有趣问题中,个别神经元沟通的细节会被抽象化。举个例子,由可解释性研究人员在 AI 系统中发现的一个计算机制,最近在小鼠的大脑中被再发现。
It is much easier to do experiments on artificial neural networks than on real ones (the latter often
requires cutting into animal brains), so interpretability may well become a tool for improving our
understanding of neuroscience. Furthermore, powerful AI’s will themselves probably be able to
develop
and apply this tool better than humans can.
在人工神经网络上做实验要比在真实神经网络上简单得多(后者通常需要切开动物的大脑),所以可解释性可能会成为加深我们对神经科学理解的一种工具。此外,强大的人工智能可能会比人类更好地开发和应用这个工具。
Beyond just interpretability though, what we have learned from AI about how intelligent systems are
trained should (though I am not sure it has yet) cause a revolution in neuroscience.
When
I was working in neuroscience, a lot of people focused on what I would now consider the wrong
questions
about learning, because the concept of the scaling hypothesis / bitter lesson didn’t
exist yet. The idea that a simple objective function plus a lot of data can
drive
incredibly complex behaviors makes it more interesting to understand the objective functions and
architectural biases and less interesting to understand the details of the emergent computations. I
have
not followed the field closely in recent years, but I have a vague sense that computational
neuroscientists have still not fully absorbed the lesson. My attitude to the scaling hypothesis has
always been “aha – this is an explanation, at a high level, of how intelligence works and how it so
easily evolved”, but I don’t think that’s the average neuroscientist’s view, in part because the
scaling
hypothesis as “the secret to intelligence” isn’t fully accepted even within AI.
不过,不仅仅是可解释性,我们从人工智能中学到的关于智能系统是如何训练的,应该(虽然我不确定它是否已经)在神经科学领域引发一场革命。当我在神经科学领域工作时,很多人关注的是我现在认为的问题方向错了,因为“扩展假说/苦涩教训”的概念还不存在。简单的目标函数加上大量数据可以驱动出不可思议的复杂行为,这个想法让理解目标函数和架构偏见变得更有趣,而理解新兴计算的细节就没那么有趣了。虽然近年来我没有密切关注这个领域,但我隐约感觉计算神经科学家还没有完全吸收这个教训。对于扩展假说,我一直的态度是“哦,这在高层次上解释了智能如何运作以及如此轻易地进化”,但我认为这并不是普通神经科学家的看法,部分原因是“扩展假说”作为“智能的秘密”在人工智能领域甚至都没有被完全接受。
I think that neuroscientists should be trying to combine this basic insight with the particularities
of
the human brain (biophysical limitations, evolutionary history, topology, details of motor and
sensory
inputs/outputs) to try to figure out some of neuroscience’s key puzzles. Some likely are, but I
suspect
it’s not enough yet, and that AI neuroscientists will be able to more effectively leverage this
angle to
accelerate progress.
我认为神经科学家应该尝试把这一基本见解与人脑的特点(生物物理限制、进化历史、拓扑、运动和感官输入/输出的细节)结合起来,以解决神经科学中的一些关键难题。有些科学家可能在这样做,但我怀疑这还不够,而人工智能神经科学家将能更有效地利用这一角度来加速进展。
I expect AI to accelerate neuroscientific progress along four distinct routes, all of which can
hopefully
work together to cure mental illness and improve function:
我期待人工智能能通过四个不同的方向加速神经科学的进步,希望它们能够协同工作,治愈心理疾病,提升功能:
- Traditional molecular biology, chemistry, and genetics. This is essentially the
same story as general biology in section 1, and AI can likely speed it up via the same
mechanisms.
There are many drugs that modulate neurotransmitters in order to alter brain function, affect
alertness or perception, change mood, etc., and AI can help us invent
many more. AI can probably also accelerate research on the genetic basis of mental illness.
传统分子生物学、化学和遗传学。这基本上和第一节的一般生物学是一个道理,而人工智能可能通过同样的机制来加快速度。有很多药物可以调节神经递质,以改变大脑功能,影响警觉性或感知,改变情绪等,人工智能可以帮助我们发明更多的药物。人工智能可能还可以加速对精神疾病遗传基础的研究。 - Fine-grained neural measurement and intervention. This is the ability to
measure
what a lot of individual neurons or neuronal circuits are doing, and intervene to change their
behavior. Optogenetics and neural probes are technologies capable of both measurement and
intervention in live organisms, and a number of very advanced methods (such as molecular ticker
tapes to read out the firing patterns of large numbers of individual neurons) have also been proposed and seem
possible in principle.
细粒度神经测量和干预。这指的是能够测量许多个体神经元或神经回路的活动,并干预以改变它们的行为。光遗传学和神经探针是能够在活体生物中进行测量和干预的技术,还有一些非常先进的方法(比如分子计时带,用于读取大量个体神经元的放电模式) 也已被提出,在原则上似乎是可行的。 - Advanced computational neuroscience. As noted above, both the specific insights
and
the gestalt of modern AI can probably be applied fruitfully to questions in systems neuroscience, including perhaps uncovering the real
causes
and dynamics of complex diseases like psychosis or mood disorders.
先进的计算神经科学。正如上面提到的,现代人工智能的具体见解和整体感可能会对系统神经科学中的问题产生积极的影响,也许还能揭示像精神病或情绪障碍等复杂疾病的真正原因和动态。 - Behavioral interventions. I haven’t much mentioned it given the focus on the
biological side of neuroscience, but psychiatry and psychology have of course developed a wide
repertoire of behavioral interventions over the 20th century; it stands to reason that
AI
could accelerate these as well, both the development of new methods and helping patients to
adhere
to existing methods. More broadly, the idea of an “AI coach” who always helps you to be the best
version of yourself, who studies your interactions and helps you learn to be more effective,
seems
very promising.
行为干预。我之前没怎么提到这一点,主要是因为关注神经科学的生物学方面,但 psychiatry 和 psychology 在 20 世纪当然发展了 丰富的行为干预手段; 合理推测,人工智能也可以加速这些手段的发展,不管是新方法的开发还是帮助患者遵循现有方法。更广泛地说,一个“人工智能教练”的概念听起来非常有前景,它总是帮助你成为最好的自己,研究你的互动,并帮助你学习变得更有效。
It’s my guess that these four routes of progress working together would, as with physical disease, be
on
track to lead to the cure or prevention of most mental illness in the next 100 years even if AI was
not
involved – and thus might reasonably be completed in 5-10 AI-accelerated years. Concretely my guess
at
what will happen is something like:
我的猜测是,这四条进展路线如果一起合作,就像对付身体疾病一样,能在未来 100 年内实现大多数心理疾病的治愈或预防,即使没有人工智能的参与——所以在 AI 加速的情况下,可能合理地在 5 到 10 年内完成。具体来说,我猜接下来会发生的事情是:
- Most mental illness can probably be cured. I’m not an expert in psychiatric
disease
(my time in neuroscience was spent building probes to study small groups of neurons) but it’s my
guess that diseases like PTSD, depression, schizophrenia, addiction, etc. can be figured out and
very effectively treated via some combination of the four directions above. The answer is likely
to
be some combination of “something went wrong biochemically” (although it could be very complex)
and
“something went wrong with the neural network, at a high level”. That is, it’s a systems
neuroscience question—though that doesn’t gainsay the impact of the behavioral interventions
discussed above. Tools for measurement and intervention, especially in live humans, seem likely
to
lead to rapid iteration and progress.
大部分精神疾病可能是可以治愈的。我不是精神疾病方面的专家(我在神经科学领域的时间主要用来构建探针研究小组神经元),但我猜像创伤后应激障碍、抑郁症、精神分裂症、成瘾等疾病可以通过上述四个方向的一些组合来理解并有效治疗。答案可能是“生化方面出现了问题”(尽管它可能非常复杂)和“神经网络在高层次上出现了问题”的某种组合。也就是说,这其实是一个系统神经科学的问题——虽然这并不否认上述行为干预的影响。在活人身上进行测量和干预的工具,似乎会迅速推动迭代和进展。 - Conditions that are very “structural” may be more difficult, but not
impossible.
There’s some
evidence that psychopathy is associated with obvious neuroanatomical differences – that
some
brain regions are simply smaller or less developed in psychopaths. Psychopaths are also believed
to
lack empathy from a young age; whatever is different about their brain, it was probably always
that
way. The same may be true of some intellectual disabilities, and perhaps other conditions.
Restructuring the brain sounds hard, but it also seems like a task with high returns to
intelligence. Perhaps there is some way to coax the adult brain into an earlier or more plastic
state where it can be reshaped. I’m very uncertain how possible this is, but my instinct is to
be
optimistic about what AI can invent here.
非常“结构性的”条件可能更难处理,但并非不可能。有一些证据表明,心理变态与明显的神经解剖差异相关——某些大脑区域在心理变态者身上明显较小或发育不足。心理变态者从小就被认为缺乏同理心;无论他们的大脑有什么不同,这种状况可能一直是这样的。某些智力残疾的情况也可能相似,可能还有其他情况。重塑大脑听起来很困难,但这似乎也是一项能给智力带来高回报的任务。也许有某种方法可以促使成年大脑回到一个更早或更具可塑性的状态,从而可以被重塑。我对这是否可能感到非常不确定,但我的直觉是对人工智能在这方面的创造持乐观态度。 - Effective genetic prevention of mental illness seems possible. Most mental
illness
is partially
heritable, and genome-wide association studies are starting
to
gain traction on identifying the relevant factors, which are often many in number. It
will
probably be possible to prevent most of these diseases via embryo screening, similar to the
story
with physical disease. One difference is that psychiatric disease is more likely to be polygenic
(many genes contribute), so due to complexity there’s an increased risk of unknowingly selecting
against positive traits that are correlated with disease. Oddly however, in
recent
years GWAS studies seem to suggest that these correlations might have been overstated. In any case, AI-accelerated
neuroscience may help us to figure these things out. Of course, embryo screening for complex
traits
raises a number of societal issues and will be controversial, though I would guess that most
people
would support screening for severe or debilitating mental illness.
有效的遗传预防心理疾病似乎是可能的。大多数心理疾病是 部分遗传的,而全基因组关联研究 开始引起关注,识别相关因素,这些因素往往数量众多。通过胚胎筛查,预防大多数这些疾病可能是可行的,这与身体疾病的情况类似。一个不同之处在于,精神疾病更可能是多基因的(多个基因共同作用),因此由于复杂性,可能会存在无意中选择对 与疾病相关的积极特征产生负面影响 的风险。然而,奇怪的是,近年来 GWAS 研究似乎暗示这些 相关性可能被夸大了。 无论如何,AI 加速的神经科学可能会帮助我们搞清楚这些问题。当然,对于复杂特征的胚胎筛查会引发许多社会问题,并且会有争议,尽管我猜大多数人会支持对严重或致残心理疾病的筛查。 - Everyday problems that we don’t think of as clinical disease will also be
solved.
Most of us have everyday psychological problems that are not ordinarily thought of as rising to
the
level of clinical disease. Some people are quick to anger, others have trouble focusing or are
often
drowsy, some are fearful or anxious, or react badly to change. Today, drugs already exist to
help
with e.g. alertness or focus (caffeine, modafinil, ritalin) but as with many other previous
areas,
much more is likely to be possible. Probably many more such drugs exist and have not been
discovered, and there may also be totally new modalities of intervention, such as targeted light
stimulation (see optogenetics above) or magnetic fields. Given how many drugs we’ve developed in
the
20th century that tune cognitive function and emotional state, I’m very optimistic about the
“compressed 21st” where everyone can get their brain to behave a bit better and have a more
fulfilling day-to-day experience.
我们平常觉得不是临床疾病的日常问题也能得到解决。大多数人都有一些日常的心理问题,这些问题通常不会被认为达到临床疾病的程度。有些人容易生气,有些人专注困难或总是昏昏欲睡,还有一些人感到恐惧或焦虑,或者对变化反应很差。现在,已经有一些药物可以帮助提高警觉性或专注力(如咖啡因、莫达非尼、利他林),但与许多其他领域一样,未来可能会有更多的选择。可能还有很多这样的药物尚未被发现,此外,也可能出现完全新的干预方式,比如定向光刺激(见上面的光遗传学)或磁场。鉴于我们在 20 世纪开发了那么多调节认知功能和情绪状态的药物,我对“压缩的 21 世纪”非常乐观,期待每个人都能让自己的大脑表现得更好,拥有更充实的日常体验。 - Human baseline experience can be much better. Taking one step further, many
people
have experienced extraordinary moments of revelation, creative inspiration, compassion,
fulfillment,
transcendence, love, beauty, or meditative peace. The character and frequency of these
experiences
differs greatly from person to person and within the same person at different times, and can
also
sometimes be triggered by various drugs (though often with side effects). All of this suggests
that
the “space of what is possible to experience” is very broad and that a larger fraction of
people’s
lives could consist of these extraordinary moments. It is probably also possible to improve
various
cognitive functions across the board. This is perhaps the neuroscience version of “biological
freedom” or “extended lifespans”.
人类的基础体验可以更好。更进一步,很多人都经历过非凡的领悟时刻、创造性灵感、同情心、成就感、超越感、爱、美,或者冥想时的宁静。这些体验的性质和频率因人而异,即使是同一个人在不同的时间也会有所不同,有时还可能通过各种药物触发(尽管通常会有副作用)。所有这些暗示“可以体验的空间”非常广泛,更多人的生活中可能包含更多这些非凡的时刻。或许各项认知功能的提升也是可能的。这也许是神经科学版的“生物自由”或“延长寿命”。
One topic that often comes up in sci-fi depictions of AI, but that I intentionally haven’t discussed
here, is “mind uploading”, the idea of capturing the pattern and dynamics of a human brain and
instantiating them in software. This topic could be the subject of an essay all by itself, but
suffice
it to say that while I think uploading is almost certainly possible
in
principle, in practice it faces significant technological and societal challenges, even with
powerful
AI, that likely put it outside the 5-10 year window we are discussing.
在科幻作品中, AI 经常出现的一个话题是“意识上传”,即捕捉人脑的模式和动态,并将其转化为软件。这话题本身就可以写一篇论文,但我想说的是,尽管我认为从原则上讲,上传几乎是可行的,但在实践中,它面临重大技术和社会挑战,即使有强大的 AI,这也可能让它超出我们所讨论的 5-10 年的时间框架。
In summary, AI-accelerated neuroscience is likely to vastly improve treatments for, or even cure,
most
mental illness as well as greatly expand “cognitive and mental freedom” and human cognitive and
emotional abilities. It will be every bit as radical as the improvements in physical health
described in
the previous section. Perhaps the world will not be visibly different on the outside, but the world
as
experienced by humans will be a much better and more humane place, as well as a place that offers
greater opportunities for self-actualization. I also suspect that improved mental health will
ameliorate
a lot of other societal problems, including ones that seem political or economic.
总的来说,AI 加速的神经科学可能会大幅改善大多数心理疾病的治疗,甚至能治愈它们,还会极大地扩展“认知与心理自由”以及人类的认知和情感能力。这些变化将和前面提到的身体健康改善一样激进。也许外面的世界看起来不会有太大变化,但人类的体验将会变得更好、更人性化,而且会为自我实现提供更多机会。我还怀疑,心理健康的改善会缓解很多其他社会问题,包括那些看起来是政治或经济上的问题。
3. Economic development and poverty
经济发展与贫困
The previous two sections are about developing new technologies that cure disease and improve
the
quality of human life. However an obvious question, from a humanitarian perspective, is: “will
everyone
have access to these technologies?”
前面两个部分讲的是开发可以治病和提高人类生活质量的新技术。但从人道主义的角度来看,一个明显的问题是:“每个人都能接触到这些技术吗?”
It is one thing to develop a cure for a disease, it is another thing to eradicate the disease from
the
world. More broadly, many existing health interventions have not yet been applied everywhere in the
world, and for that matter the same is true of (non-health) technological improvements in general.
Another way to say this is that living standards in many parts of the world are still desperately
poor:
GDP per capita is
~$2,000 in Sub-Saharan Africa as compared to ~$75,000 in the United States. If AI further increases
economic growth and quality of life in the developed world, while doing little to help the
developing
world, we should view that as a terrible moral failure and a blemish on the genuine humanitarian
victories in the previous two sections. Ideally, powerful AI should help the developing world
catch
up to the developed world, even as it revolutionizes the latter.
开发一种疾病的治疗方法是一回事,而将这种疾病从世界上根除则是另一回事。更广泛地说,许多现有的健康干预措施仍然未在世界各地普遍应用,非健康的技术进步同样如此。换句话说,世界上许多地方的生活水平依然非常贫困:在撒哈拉以南非洲,人均 GDP 约为$2,000,而在美国则约为$75,000。如果人工智能进一步推动发达国家的经济增长和生活质量,而对发展中国家的帮助微乎其微,我们就应该把这视为严重的道德失败,并且是对前两节中真正的人道主义胜利的污点。理想情况下,强大的人工智能应该帮助发展中国家赶上发达国家,同时彻底改革后者。
I am not as confident that AI can address inequality and economic growth as I am that it can invent
fundamental technologies, because technology has such obvious high returns to intelligence
(including
the ability to route around complexities and lack of data) whereas the economy involves a lot of
constraints from humans, as well as a large dose of intrinsic complexity. I am somewhat skeptical
that
an AI could solve the famous “socialist calculation problem”23 and I don’t think governments will (or should) turn over their
economic policy to such an entity, even if it could do so. There are also problems like how to
convince
people to take treatments that are effective but that they may be suspicious of.
我对人工智能能否解决不平等和经济增长的信心没有对它能够发明基础技术的信心大,因为技术对智力的回报是显而易见的(包括处理复杂性和缺乏数据的能力),而经济则涉及很多来自人类的约束,还有很大的内在复杂性。我对人工智能能否解决著名的“社会主义计算问题”23有些怀疑,我认为政府并不会(也不应该)把经济政策交给这样的实体,即使它能够做到。还有一些问题,比如如何说服人们接受那些有效但他们可能会怀疑的治疗方法。
The challenges facing the developing world are made even more complicated by pervasive corruption in both
private and public sectors. Corruption creates a vicious cycle: it exacerbates poverty, and
poverty in
turn breeds more corruption. AI-driven plans for economic development need to reckon with corruption,
weak institutions, and other very human challenges.
发展中国家面临的挑战因普遍的腐败而变得更加复杂,这种腐败存在于私营和公共部门。腐败形成了恶性循环:它加剧了贫困,而贫困又进一步滋生更多的腐败。以人工智能驱动的经济发展计划需要考虑到腐败、脆弱的制度,以及其他非常人性的挑战。
Nevertheless, I do see significant reasons for optimism. Diseases have been eradicated and many
countries
have gone from poor to rich, and it is clear that the decisions involved in these tasks exhibit
high
returns to intelligence (despite human constraints and complexity). Therefore, AI can likely do them
better than they are currently being done. There may also be targeted interventions that get around the
human constraints and that AI could focus on. More importantly though, we have to try. Both AI
companies
and developed world policymakers will need to do their part to ensure that the developing world is not
left out; the moral imperative is too great. So in this section, I’ll continue to make the optimistic
case, but keep in mind everywhere that success is not guaranteed and depends on our collective efforts.
尽管如此,我确实看到了一些值得乐观的重大理由。疾病已经被消灭,许多国家从贫穷变得富裕,显然,涉及这些任务的决策表现出对智力的高回报(尽管受到人类限制和复杂性的影响)。因此,人工智能可能会比现在的做法做得更好。可能还有一些针对性的干预措施能够绕过人类的限制,人工智能可以集中精力去解决这些问题。不过,更重要的是,我们必须尝试。无论是人工智能公司还是发达国家的决策者,都需要尽自己的责任,确保发展中国家不会被遗漏;道德责任太重大了。因此,在这一部分,我会继续提出乐观的观点,但请始终记住,成功并不是 guaranteed 的,而是依赖于我们共同的努力。
Below I make some guesses about how I think things may go in the developing world over the 5-10 years
after powerful AI is developed:
以下是我对强大人工智能发展后,在发展中国家未来 5 到 10 年可能会发生的事情的一些猜测:
- Distribution of health interventions. The area where I am perhaps most
optimistic
is distributing health interventions throughout the world. Diseases have actually been
eradicated by
top-down campaigns: smallpox was fully eliminated in the 1970’s, and polio and guinea worm are nearly
eradicated with less than 100 cases per year. Mathematically sophisticated epidemiological modeling plays an active
role
in disease eradication campaigns, and it seems very likely that there is room for
smarter-than-human
AI systems to do a better job of it than humans are. The logistics of distribution can probably
also
be greatly optimized. One thing I learned as an early donor to GiveWell is that some health charities
are way more effective than others;
the hope is that AI-accelerated efforts would be more effective still. Additionally, some
biological
advances actually make the logistics of distribution much easier: for example, malaria has been
difficult to eradicate because it requires treatment each time the disease is contracted; a
vaccine
that only needs to be administered once makes the logistics much simpler (and such vaccines for
malaria are in fact
currently being developed). Even simpler distribution mechanisms are possible: some
diseases
could in principle be eradicated by targeting their animal carriers, for example releasing
mosquitoes infected with a bacterium that blocks their ability to carry a disease (who then infect all the other
mosquitos) or simply using gene drives to wipe out the mosquitos. This requires one or a few
centralized actions, rather than a coordinated campaign that must individually treat millions.
Overall, I think 5-10 years is a reasonable timeline for a good fraction (maybe 50%) of
AI-driven
health benefits to propagate to even the poorest countries in the world. A good goal might be
for
the developing world 5-10 years after powerful AI to at least be substantially healthier than
the
developed world is today, even if it continues to lag behind the developed world. Accomplishing
this
will of course require a huge effort in global health, philanthropy, political advocacy, and
many
other efforts, which both AI developers and policymakers should help with.
健康干预的分布。我最乐观的领域或许是将健康干预措施分布到全世界。通过自上而下的运动,某些疾病实际上被消灭了:在 1970 年代,天花被彻底消灭,而脊髓灰质炎和盲肠虫几乎被消灭,每年病例不到 100 例。数学上复杂的流行病学模型在疾病消灭运动中发挥着积极作用,似乎非常有可能,有空间让超越人类的智能系统比人类做得更好。分发的物流也可能大大优化。我作为GiveWell的早期捐赠者学到的一件事是,一些健康慈善机构的效果远远超过其他机构;希望 AI 加速的努力能更加有效。 此外,一些生物技术的进步实际上使分发的后勤工作变得更简单:例如,疟疾之所以难以根除,是因为每次感染都需要治疗;而只需要一次接种的疫苗则让后勤工作变得简单得多(而且这种疟疾疫苗目前确实正在研发中)。甚至可以实现更简单的分发机制:比如,原则上可以通过针对动物载体来根除一些疾病,例如释放被细菌感染的蚊子,**这种细菌阻止它们传播疾病**(然后这些蚊子会感染其他的蚊子),或者直接使用基因驱动技术来消灭蚊子。这只需要一次或几次集中行动,而不是需要协调的运动来逐一治疗数百万患者。总体来说,我认为 5 到 10 年是合理的时间表,能够有效推广 50%左右的 AI 驱动健康益处到世界上最贫困的国家。 一个好的目标可能是在强大的人工智能出现后的 5-10 年内,发展中国家的健康水平至少要比今天的发达国家大幅提升,即使它仍然落后于发达国家。当然,实现这一目标需要在全球健康、慈善、政治倡导以及许多其他领域付出巨大努力,AI 开发者和政策制定者都应该对此有所助益。 - Economic growth. Can the developing world quickly catch up to the developed
world,
not just in health, but across the board economically? There is some precedent for this: in the
final decades of the 20th century, several East Asian economies achieved sustained ~10% annual real GDP
growth
rates, allowing them to catch up with the developed world. Human economic planners made the
decisions that led to this success, not by directly controlling entire economies but by pulling
a
few key levers (such as an industrial policy of export-led growth, and resisting the temptation
to
rely on natural resource wealth); it’s plausible that “AI finance ministers and central bankers”
could replicate or exceed this 10% accomplishment. An important question is how to get
developing
world governments to adopt them while respecting the principle of self-determination—some may be
enthusiastic about it, but others are likely to be skeptical. On the optimistic side, many of
the
health interventions in the previous bullet point are likely to organically increase economic
growth: eradicating AIDS/malaria/parasitic worms would have a transformative effect on
productivity,
not to mention the economic benefits that some of the neuroscience interventions (such as
improved
mood and focus) would have in developed and developing world alike. Finally, non-health
AI-accelerated technology (such as energy technology, transport drones, improved building
materials,
better logistics and distribution, and so on) may simply permeate the world naturally; for
example,
even cell phones quickly permeated sub-Saharan Africa via market mechanisms, without needing
philanthropic efforts. On the more negative side, while AI and automation have many potential
benefits, they also pose challenges for economic development, particularly for countries that
haven't yet industrialized. Finding ways to ensure these countries can still develop and improve
their economies in an age of increasing automation is an important challenge for economists and
policymakers to address. Overall, a dream scenario—perhaps a goal to aim for—would be 20% annual
GDP
growth rate in the developing world, with 10% each coming from AI-enabled economic decisions and
the
natural spread of AI-accelerated technologies, including but not limited to health. If achieved,
this would bring sub-Saharan Africa to the current per-capita GDP of China in 5-10 years, while
raising much of the rest of the developing world to levels higher than the current US GDP.
Again,
this is a dream scenario, not what happens by default: it’s something all of us must work
together
to make more likely.
经济增长。发展中国家能否在健康等各方面迅速追赶上发达国家?这方面有一些成功的先例:在 20 世纪的最后几十年,几个东亚经济体实现了持续约 10%的年实际 GDP 增长率,使它们能够赶上发达国家。人类经济规划者做出的决策促成了这个成功,他们并不是通过直接控制整个经济来实现的,而是通过拉动一些关键杠杆(如以出口为导向的工业政策,以及抵制依赖自然资源财富的诱惑);“人工智能财政部长和中央银行行长”有可能复制或超越这个 10%的成就。一个重要的问题是,如何促使发展中国家的政府在尊重自决原则的同时采纳这些政策——有些可能对此持热情态度,但其他的可能会持怀疑态度。 从乐观的角度来看,上一条提到的许多健康干预措施可能会自然地推动经济增长:消灭艾滋病/疟疾/寄生虫病将对生产力产生深远的影响,更不用说一些神经科学干预(比如改善情绪和专注力)在发达国家和发展中国家带来的经济利益。最后,非健康的人工智能加速技术(比如能源技术、运输无人机、改进的建筑材料、更好的物流和分配等等)可能会自然而然地渗透到世界各地;例如,手机就很快通过市场机制在撒哈拉以南非洲普及,而无需慈善努力。另一方面,虽然人工智能和自动化有许多潜在好处,但它们对经济发展也带来了挑战,特别是对那些还没有实现工业化的国家。在日益自动化的时代,找到确保这些国家仍然能够发展和改善经济的方法,是经济学家和政策制定者需要面对的重要挑战。 整体来说,一个理想的情况——也许是一个目标——是在发展中国家实现每年 20%的 GDP 增长率,其中 10%来自于 AI 驱动的经济决策,另外 10%来自于自然扩散的 AI 加速技术,包括但不限于健康领域。如果能实现这一目标,撒哈拉以南非洲将在 5 到 10 年内达到中国目前的人均 GDP,而大部分其他发展中国家的水平也会超过目前的美国 GDP。再强调一下,这只是一个理想情况,并不是自然而然就会发生的:这是需要我们大家共同努力才能让它更有可能实现的目标。 - Food security 24. Advances in crop technology like better
fertilizers and
pesticides, more automation, and more efficient land use drastically increased crop yields across the
20th
Century, saving millions of people from hunger. Genetic engineering is currently improving many crops even further. Finding even more ways to
do
this—as well as to make agricultural supply chains even more efficient—could give us an
AI-driven
second Green
Revolution, helping close the gap between the developing and developed world.
粮食安全 24。像更好的 fertilizers 和 pesticides、更多的自动化,以及更高效的土地利用等作物技术的进步,极大地提高了 作物产量,拯救了数百万人的饥饿。在目前,基因工程正在 进一步改善许多作物。找到更多这样的方式,同时让农业供应链更高效,可能会带来一个由 AI 驱动的第二个 绿色革命,帮助缩小发展中国家和发达国家之间的差距。 - Mitigating climate change. Climate change will be felt much more strongly in
the
developing world, hampering its development. We can expect that AI will lead to improvements in
technologies that slow or prevent climate change, from atmospheric carbon-removal
and
clean energy technology to lab-grown meat that reduces our reliance on carbon-intensive factory
farming. Of course, as discussed above, technology isn’t the only thing restricting progress on
climate change—as with all of the other issues discussed in this essay, human societal factors
are
important. But there’s good reason to think that AI-enhanced research will give us the means to
make
mitigating climate change far less costly and disruptive, rendering many of the objections moot
and
freeing up developing countries to make more economic progress.
缓解气候变化。气候变化在发展中国家的影响会更明显,阻碍其发展。我们可以预期,人工智能将推动改善减缓或防止气候变化的技术,从大气中的碳捕集和清洁能源技术,到减少我们对碳密集型工厂养殖依赖的实验室培育肉。当然,如上所述,技术并不是唯一限制气候变化进展的因素——就像本文讨论的其他所有问题一样,人类社会因素也很重要。但有充分理由认为,人工智能增强的研究将使我们能够以更少的成本和干扰来缓解气候变化,从而使许多反对意见失去意义,并使发展中国家能实现更多经济进步。 - Inequality within countries. I’ve mostly talked about inequality as a global
phenomenon (which I do think is its most important manifestation), but of course inequality also
exists within countries. With advanced health interventions and especially radical
increases
in lifespan or cognitive enhancement drugs, there will certainly be valid worries that these
technologies are “only for the rich”. I am more optimistic about within-country inequality
especially in the developed world, for two reasons. First, markets function better in the
developed
world, and markets are typically good at bringing down the cost of high-value technologies over
time25. Second, developed
world
political institutions are more responsive to their citizens and have greater state capacity to
execute universal access programs—and I expect citizens to demand access to technologies that so
radically improve quality of life. Of course it’s not predetermined that such demands
succeed—and
here is another place where we collectively have to do all we can to ensure a fair society.
There is
a separate problem in inequality of wealth (as opposed to inequality of access to
life-saving
and life-enhancing technologies), which seems harder and which I discuss in Section 5.
国家内部的不平等。我主要谈论不平等作为一种全球现象(我确实认为这是它最重要的表现),但当然国家内部也存在不平等 。随着先进的健康干预措施,尤其是寿命的大幅延长或认知增强药物的出现,人们自然会担心这些技术“只属于富人”。我对发达国家内部的不平等持更乐观的态度,这主要有两个原因。首先,发达国家的市场运作得更好,而市场通常能在一段时间内有效降低高价值技术的成本25。其次,发达国家的政治机构对公民的响应能力更强,拥有更大的国家能力来执行普遍访问项目——我预计公民会要求能够获得这些极大改善生活质量的技术。当然,这样的需求并非注定就能成功——这也是我们共同需要尽力确保公平社会的另一个方面。 在财富的不平等问题上,财富(与获取救命和提升生活质量的技术的不平等相对比)似乎更复杂,我将在第五节中讨论这个问题。 - The opt-out problem. One concern in both developed and developing world alike
is
people opting out of AI-enabled benefits (similar to the anti-vaccine movement, or
Luddite
movements more generally). There could end up being bad feedback cycles where, for example, the
people who are least able to make good decisions opt out of the very technologies that improve
their
decision-making abilities, leading to an ever-increasing gap and even creating a dystopian
underclass (some researchers have argued that this will undermine
democracy, a topic I discuss further in the next section). This would, once again, place
a
moral blemish on AI’s positive advances. This is a difficult problem to solve as I don’t think
it is
ethically okay to coerce people, but we can at least try to increase people’s scientific
understanding—and perhaps AI itself can help us with this. One hopeful sign is that historically
anti-technology movements have been more bark than bite: railing against modern technology is
popular, but most people adopt it in the end, at least when it’s a matter of individual choice.
Individuals tend to adopt most health and consumer technologies, while technologies that are
truly
hampered, like nuclear power, tend to be collective political decisions.
主动退出的问题。无论是在发达国家还是发展中国家,人们 主动退出 AI 驱动的福利是一大关注点(这类似于反疫苗运动,或者更广泛的鲁德运动)。可能会出现恶性循环,例如,决策能力最差的人选择放弃那些能提升他们决策能力的技术,这导致了日益扩大的差距,甚至可能造成一个反乌托邦的下层社会(一些研究者认为这会 削弱民主,我将在下一部分进一步讨论这个话题)。这将再次给 AI 的积极进展带来道德上的瑕疵。解决这个问题很困难,因为我认为强迫人们是不道德的,但我们至少可以努力提高人们的科学理解——也许 AI 本身能帮助我们做到这一点。一个令人鼓舞的迹象是,历史上反技术运动往往更多是叫喊而少有实质:抗议现代技术很受欢迎,但最终大多数人还是会选择接受它,至少在个人选择的问题上。 个人通常会采纳大多数健康和消费科技,而真正受到阻碍的科技,比如核能,往往是集体政治决策的结果。
Overall, I am optimistic about quickly bringing AI’s biological advances to people in the developing
world. I am hopeful, though not confident, that AI can also enable unprecedented economic growth
rates
and allow the developing world to at least surpass where the developed world is now. I am concerned
about the “opt out” problem in both the developed and developing world, but suspect that it will
peter
out over time and that AI can help accelerate this process. It won’t be a perfect world, and those
who
are behind won’t fully catch up, at least not in the first few years. But with strong efforts on our
part, we may be able to get things moving in the right direction—and fast. If we do, we can make at
least a downpayment on the promises of dignity and equality that we owe to every human being on
earth.
总体来说,我对快速将人工智能的生物技术带给发展中国家的人民感到乐观。虽然我并不完全自信,但我希望人工智能能实现前所未有的经济增长率,让发展中国家至少超过现在的发达国家水平。我对发达国家和发展中国家的“选择退出”问题有些担忧,但我怀疑这个问题随着时间会有所缓解,人工智能可以帮助加速这个过程。这个世界不会完美,那些落后的人在前几年内也不会完全赶上。但是只要我们下功夫,就有可能朝正确的方向推进,并且速度会很快。如果能做到这一点,我们就至少能为我们欠每一个地球人应有的尊严和平等的承诺打下基础。
4. Peace and governance 4. 和平与治理
Suppose that everything in the first three sections goes well: disease, poverty, and inequality are
significantly reduced and the baseline of human experience is raised substantially. It does not
follow
that all major causes of human suffering are solved. Humans are still a threat to each other.
Although there is a trend of technological improvement and economic development leading to
democracy and peace, it is a very loose trend, with frequent (and recent) backsliding. At the dawn of the 20th Century, people thought they had put
war
behind them; then came the two world wars. Thirty years ago Francis Fukuyama wrote about “the End
of
History” and a final triumph of liberal democracy; that hasn’t happened yet. Twenty years
ago US
policymakers believed that free trade with China would cause it to liberalize as it became richer;
that
very much didn’t happen, and we now seem headed for
a
second cold war with a resurgent authoritarian bloc. And plausible theories suggest that
internet technology may actually advantage authoritarianism, not democracy as initially believed
(e.g. in the “Arab Spring” period). It seems important to try to understand how powerful AI will
intersect with these issues of peace, democracy, and freedom.
假设前面三个部分的内容都进展顺利:疾病、贫困和不平等有了显著减少,人类体验的基线大幅提高。然而,并不意味着人类遭受痛苦的所有主要原因都得到了解决。人类依然是彼此的威胁. 尽管科技进步和经济发展有助于民主与和平的趋势,但这一趋势相当松散,时常出现(而且是最近的)倒退。在 20 世纪初,人们认为他们已经将战争抛在了身后;但随之而来的是两次世界大战。三十年前,弗朗西斯·福山写过“历史的终结”以及自由民主的最终胜利;但这还没发生。二十年前,美国决策者相信与中国的自由贸易会让中国在变得更富裕时实现自由化;这一点并没有如预期发展,我们现在似乎正走向第二次冷战,面对复兴的威权主义集团。而且一些合理的理论表明,互联网技术实际上可能更有利于威权主义,而非最初认为的民主(例如在“阿拉伯之春”时期)。 似乎理解强大人工智能如何与和平、民主和自由的问题交织在一起是很重要的。
Unfortunately, I see no strong reason to believe AI will preferentially or structurally advance
democracy
and peace, in the same way that I think it will structurally advance human health and alleviate
poverty.
Human conflict is adversarial and AI can in principle help both the “good guys” and the “bad guys”.
If
anything, some structural factors seem worrying: AI seems likely to enable much better propaganda
and
surveillance, both major tools in the autocrat’s toolkit. It’s therefore up to us as individual
actors
to tilt things in the right direction: if we want AI to favor democracy and individual rights, we
are
going to have to fight for that outcome. I feel even more strongly about this than I do about
international inequality: the triumph of liberal democracy and political stability is not
guaranteed, perhaps not even likely, and will require great sacrifice and commitment on all of our
parts, as it often has in the past.
不幸的是,我看不到强有力的理由相信人工智能会像在人类健康和减轻贫困方面那样,在结构上优先推进民主和和平。人类冲突是对抗性的,而人工智能原则上可以帮助“好人”和“坏人”。实际上,一些结构性因素似乎令人担忧:人工智能可能会大大提升宣传和监视的能力,而这都是独裁者工具箱中的主要工具。因此,作为个体行动者,我们必须努力朝正确的方向倾斜:如果我们希望人工智能支持民主和个人权利,我们就必须为这个结果而奋斗。我对此的感受比国际不平等更加强烈:自由民主和政治稳定的胜利并不保证,也许甚至不太可能,这将需要我们所有人付出巨大的牺牲和承诺,正如过去常常所需的那样。
I think of the issue as having two parts: international conflict, and the internal structure of
nations.
On the international side, it seems very important that democracies have the upper hand on the world
stage when powerful AI is created. AI-powered authoritarianism seems too terrible to contemplate, so
democracies need to be able to set the terms by which powerful AI is brought into the world, both to
avoid being overpowered by authoritarians and to prevent human rights abuses within authoritarian
countries.
我认为这个问题可以分成两部分:国际冲突和国家内部结构。在国际方面,当强大的人工智能出现时,民主国家在世界舞台上占据上风似乎非常重要。AI 驱动的专制主义实在太可怕,因此民主国家需要能够决定强大人工智能进入世界的规则,以避免被专制者压倒,并防止专制国家的人权侵犯。
My current guess at the best way to do this is via an “entente strategy”26, in which a coalition of democracies seeks
to
gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain,
scaling
quickly, and blocking or delaying adversaries’ access to key resources like chips and
semiconductor equipment. This coalition would on one hand use AI to achieve robust military
superiority
(the stick) while at the same time offering to distribute the benefits of powerful AI (the carrot)
to a
wider and wider group of countries in exchange for supporting the coalition’s strategy to promote
democracy (this would be a bit analogous to “Atoms for Peace”). The coalition would aim to gain the support of more and
more
of the world, isolating our worst adversaries and eventually putting them in a position where they
are
better off taking the same bargain as the rest of the world: give up competing with democracies in
order
to receive all the benefits and not fight a superior foe.
我现在认为做这个的最佳方法是通过一种“协同战略”26,在这种战略中,一些民主国家的联盟试图通过确保其供应链、快速扩张以及阻止或延迟对手获取关键资源(例如芯片和半导体设备)来获得显著优势(即使只是暂时的)。这个联盟一方面会利用人工智能实现强大的军事优势(制裁),另一方面则会向越来越多的国家分配强大人工智能带来的好处(诱饵),以换取他们支持联盟的推广民主的策略(这有点类似于“和平原子”)。这个联盟的目标是获得越来越多国家的支持,使我们最糟糕的对手孤立起来,最终让他们处于一个更好地接受与世界其他国家相同交易的情况下:放弃与民主国家竞争,从而获得所有的利益,并且不与更强大的对手作斗争。
If we can do all this, we will have a world in which democracies lead on the world stage and have the
economic and military strength to avoid being undermined, conquered, or sabotaged by autocracies,
and
may be able to parlay their AI superiority into a durable advantage. This could optimistically lead
to
an “eternal 1991”—a world where democracies have the upper hand and Fukuyama’s dreams are realized.
Again, this will be very difficult to achieve, and will in particular require close cooperation
between
private AI companies and democratic governments, as well as extraordinarily wise decisions about the
balance between carrot and stick.
如果我们能做到这一切,世界就会变成一个民主国家在国际舞台上主导,拥有足够的经济和军事力量,避免被专制国家侵蚀、征服或破坏的地方,甚至可能将他们的人工智能优势转化为持久优势。乐观地说,这可能会导致一个“永恒的 1991”——一个民主国家占上风、福山的梦想得以实现的世界。不过,要实现这一点非常困难,特别需要私营 AI 公司和民主政府之间的紧密合作,以及在胡萝卜和大棒之间作出极其明智的决策。
Even if all that goes well, it leaves the question of the fight between democracy and autocracy
within each country. It is obviously hard to predict what will happen here, but I do have
some
optimism that given a global environment in which democracies control the most powerful AI,
then AI may actually structurally favor democracy everywhere. In particular, in this
environment
democratic governments can use their superior AI to win the information war: they can counter
influence
and propaganda operations by autocracies and may even be able to create a globally free information
environment by providing channels of information and AI services in a way that autocracies lack the
technical ability to block or monitor. It probably isn’t necessary to deliver propaganda, only to
counter malicious attacks and unblock the free flow of information. Although not immediate, a level
playing field like this stands a good chance of gradually tilting global governance towards
democracy,
for several reasons.
即使一切顺利,这依然留下了民主与专制在每个国家之间斗争的问题。很明显,很难预测这里会发生什么,但我对这种情况有些乐观,因为在一个民主国家控制着最强大的人工智能的全球环境中,人工智能实际上可能在结构上有利于全球的民主。特别是在这种环境下,民主政府可以利用其更强大的人工智能赢得信息战:它们可以对抗专制国家的影响力和宣传行为,甚至有可能通过提供信息渠道和人工智能服务,创造一个全球自由的信息环境,而专制国家缺乏技术能力去阻止或监控。可能不需要传播宣传,只需对抗恶意攻击并打通信息自由流动的渠道。虽然不会立刻见效,但这样的公平竞争环境有很大机会逐渐将全球治理倾向于民主,这有几个原因。
First, the increases in quality of life in Sections 1-3 should, all things equal, promote democracy:
historically they have, to at least some extent. In particular I expect improvements in mental
health,
well-being, and education to increase democracy, as all three are negatively
correlated with support for authoritarian leaders. In general people want
more
self-expression when their other needs are met, and democracy is among other things a form of
self-expression. Conversely, authoritarianism thrives on fear and resentment.
首先,生活质量在第 1-3 节的提高应该,在其他条件相同的情况下,促进民主:历史上,至少在某种程度上确实如此。特别是我预期心理健康、幸福感和教育水平的改善将会增加民主,因为这三者与支持威权领导者有着负相关关系。通常,当人们的其他需求得到满足时,他们会想要更多的自我表达,而民主在某种程度上就是一种自我表达。相反,威权主义则依赖于恐惧和怨恨。
Second, there is a good chance free information really does undermine authoritarianism, as long as
the
authoritarians can’t censor it. And uncensored AI can also bring individuals powerful tools for
undermining repressive governments. Repressive governments survive by denying people a certain kind
of
common knowledge, keeping them from realizing that “the emperor has no clothes”. For example Srđa
Popović, who helped to topple the Milošević government in Serbia, has written extensively
about
techniques for psychologically robbing authoritarians of their power, for breaking the spell and
rallying support against a dictator. A superhumanly effective AI version of Popović (whose skills
seem
like they have high returns to intelligence) in everyone’s pocket, one that dictators are powerless
to
block or censor, could create a wind at the backs of dissidents and reformers across the world. To
say
it again, this will be a long and protracted fight, one where victory is not assured, but if we
design
and build AI in the right way, it may at least be a fight where the advocates of freedom everywhere
have
an advantage.
其次,免费的信息确实有可能削弱专制主义,只要专制者无法对其进行审查。而且,不受审查的人工智能也能为个人提供强大的工具,帮助他们削弱压迫性的政府。压迫性政府通过剥夺人们某种共同知识的权利而生存,保持他们不会意识到“皇帝的新装”。例如,Srđa Popović 帮助推翻了塞尔维亚的米洛舍维奇政府,他广泛地撰写了关于从心理上剥夺专制者权力的技巧,如何打破迷惑并团结反对独裁者的支持。一个超人般有效的人工智能版本的波波维奇(他的技能似乎在智力上有高回报)存在于每个人的口袋里,这种人工智能是独裁者无能为力去阻止或审查的,可能会为全球的异议人士和改革者带来助力。再说一次,这将是一个漫长而艰苦的斗争,胜利并不保证,但如果我们以正确的方式设计和构建人工智能,至少这可能成为一个自由拥护者在各地拥有优势的斗争。
As with neuroscience and biology, we can also ask how things could be “better than normal”—not just
how
to avoid autocracy, but how to make democracies better than they are today. Even within democracies,
injustices happen all the time. Rule-of-law societies make a promise to their citizens that everyone
will be equal under the law and everyone is entitled to basic human rights, but obviously people do
not
always receive those rights in practice. That this promise is even partially fulfilled makes it
something to be proud of, but can AI help us do better?
就像神经科学和生物学一样,我们也可以问,事情怎么才能“比正常更好”——不仅仅是如何避免专制,还要如何让民主制度比现在更好。即便在民主国家, injustices 始终存在。法治社会向公民承诺每个人在法律面前都是平等的,人人都有基本人权,但显然在实践中并不是每个人都能真正享受到这些权利。这个承诺即便部分实现,也值得我们骄傲,但人工智能能够帮助我们做到更好吗?
For example, could AI improve our legal and judicial system by making decisions and processes more
impartial? Today people mostly worry in legal or judicial contexts that AI systems will be a cause of discrimination, and these worries are important and need to
be
defended against. At the same time, the vitality of democracy depends on harnessing new technologies
to
improve democratic institutions, not just responding to risks. A truly mature and successful
implementation of AI has the potential to reduce bias and be fairer for everyone.
例如,AI 能否通过让决策和流程更公正来改善我们的法律和司法系统?如今,人们在法律或司法领域主要担心的是 AI 系统会成为歧视的原因,而这些担忧是重要的,需要加以防范。同时,民主的活力依赖于利用新技术来改善民主制度,而不仅仅是应对风险。真正成熟和成功的 AI 实施有潜力减少偏见,让每个人都能更加公平。
For centuries, legal systems have faced the dilemma that the law aims to be impartial, but is
inherently
subjective and thus must be interpreted by biased humans. Trying to make the law fully mechanical
hasn’t
worked because the real world is messy and can’t always be captured in mathematical formulas.
Instead
legal systems rely on notoriously imprecise criteria like “cruel and
unusual
punishment” or “utterly without redeeming social importance”, which humans then
interpret—and
often do so in a manner that displays bias, favoritism, or arbitrariness. “Smart contracts” in
cryptocurrencies haven’t revolutionized law because ordinary code isn’t smart enough to adjudicate
all
that much of interest. But AI might be smart enough for this: it is the first technology capable of
making broad, fuzzy judgements in a repeatable and mechanical way.
法律系统几个世纪以来一直面临一个两难局面:法律旨在公正,但本质上是主观的,因此必须由有偏见的人来解释。试图让法律完全机械化并没有成功,因为现实世界很复杂,无法总是用数学公式来表达。相反,法律系统依赖一些 notoriously 模糊的标准,比如“残酷和不寻常的惩罚”或“毫无挽回社会价值”,这些标准需要人来解释,而人类的解释往往表现出偏见、偏爱或任意性。加密货币中的“智能合约”并没有彻底改变法律,因为普通代码并不足够智能,无法裁定太多相关的事项。但是,人工智能可能足够聪明来做到这一点:它是第一种能够以可重复和机械化的方式做出广泛模糊判断的技术。
I am not suggesting that we literally replace judges with AI systems, but the combination of
impartiality
with the ability to understand and process messy, real world situations feels like it should
have
some serious positive applications to law and justice. At the very least, such systems could work
alongside humans as an aid to decision-making. Transparency would be important in any such system,
and a
mature science of AI could conceivably provide it: the training process for such systems could be
extensively studied, and advanced
interpretability techniques could be used to see inside the final model and assess it for
hidden
biases, in a way that is simply not possible with humans. Such AI tools could also be used to
monitor
for violations of fundamental rights in a judicial or police context, making constitutions more
self-enforcing.
我并不是说我们真的要用 AI 系统取代法官,但将公正性与理解和处理复杂现实情况的能力结合起来,感觉确实应该在法律和正义上有一些非常积极的应用。至少,这些系统可以作为人类决策的辅助工具并肩工作。任何这样的系统都需要透明度,而成熟的 AI 科学或许能提供这一点:这些系统的训练过程可以被广泛研究,先进的可解释性技术可以用来深入了解最终模型,评估潜在的偏见,这在人类身上根本不可能做到。这些 AI 工具还可以用于监测司法或警务环境中侵犯基本权利的行为,使宪法的自我执行能力更强。
In a similar vein, AI could be used to both aggregate opinions and drive consensus among citizens,
resolving conflict, finding common ground, and seeking compromise. Some early ideas in this
direction
have been undertaken by the computational
democracy
project, including collaborations with Anthropic. A more informed and thoughtful citizenry
would
obviously strengthen democratic institutions.
在这一方面,人工智能可以用来聚合意见,并促成公民之间的共识,解决冲突,寻找共同点以及寻求妥协。一些早期的相关想法是由计算民主项目进行的,包含了与Anthropic 的合作。显然,一个更有见识和深思熟虑的公民群体将增强民主制度。
There is also a clear opportunity for AI to be used to help provision government services—such as
health
benefits or social services—that are in principle available to everyone but in practice often
severely
lacking, and worse in some places than others. This includes health services, the DMV, taxes, social
security, building code enforcement, and so on. Having a very thoughtful and informed AI whose job
is to
give you everything you’re legally entitled to by the government in a way you can understand—and who
also helps you comply with often confusing government rules—would be a big deal. Increasing state
capacity both helps to deliver on the promise of equality under the law, and strengthens respect for
democratic governance. Poorly implemented services are currently a major driver of cynicism about
government27.
人工智能在帮助提供政府服务方面也有很大的机会——比如健康福利或社会服务,这些服务原则上是对每个人开放的,但在实际操作中往往严重不足,某些地方的情况更是糟糕。这包括医疗服务、车辆管理局、税务、社会保障、建筑规范执法等等。拥有一个非常周到和有见识的人工智能,它的任务就是以你能理解的方式为你提供你合法享有的所有政府权益,还帮助你遵守通常让人困惑的政府规则,这会是个大问题。提升政府能力不仅有助于实现法律面前人人平等的承诺,还能增强对民主治理的尊重。目前,服务实施不当是导致人们对政府感到愤世嫉俗的主要原因之一27。
All of these are somewhat vague ideas, and as I said at the beginning of this section, I am not
nearly as
confident in their feasibility as I am in the advances in biology, neuroscience, and poverty
alleviation. They may be unrealistically utopian. But the important thing is to have an ambitious
vision, to be willing to dream big and try things out. The vision of AI as a guarantor of liberty,
individual rights, and equality under the law is too powerful a vision not to fight for. A 21st
century,
AI-enabled polity could be both a stronger protector of individual freedom, and a beacon of hope
that
helps make liberal democracy the form of government that the whole world wants to adopt.
所有这些想法都有点模糊,正如我在这一部分开头所说的,我对它们的可行性没有对生物学、神经科学和扶贫的进展那么自信。它们可能是不切实际的乌托邦。但重要的是要有一个宏大的愿景,愿意大梦一场,尝试各种可能。人工智能作为自由、个人权利和法律平等的保障者的愿景太强大,值得我们为之奋斗。一个 21 世纪的、以人工智能为基础的政治体,既能更好地保护个人自由,又能成为一个希望的灯塔,帮助使自由民主成为全球想要的治理形式。
5. Work and meaning 5. 工作与意义
Even if everything in the preceding four sections goes well—not only do we alleviate disease,
poverty,
and inequality, but liberal democracy becomes the dominant form of government, and existing liberal
democracies become better versions of themselves—at least one important question still remains.
“It’s
great we live in such a technologically advanced world as well as a fair and decent one”, someone
might
object, “but with AI’s doing everything, how will humans have meaning? For that matter, how will
they
survive economically?”.
即使前面四个部分的内容都很顺利——我们不仅缓解了疾病、贫困和不平等,且自由民主成为主导政府形式,现有的自由民主国家也变得更加优秀——至少还有一个重要问题。有人可能会反驳:“我们生活在一个科技如此发达、又公平体面的世界里,真是太好了,但如果都由人工智能来做,那人类的意义在哪里呢?更不用说,他们怎么在经济上生存?”
I think this question is more difficult than the others. I don’t mean that I am necessarily more
pessimistic about it than I am about the other questions (although I do see challenges). I mean that
it
is fuzzier and harder to predict in advance, because it relates to macroscopic questions about how
society is organized that tend to resolve themselves only over time and in a decentralized manner.
For
example, historical hunter-gatherer societies might have imagined that life is meaningless without
hunting and various kinds of hunting-related religious rituals, and would have imagined that our
well-fed technological society is devoid of purpose. They might also have not understood how our
economy
can provide for everyone, or what function people can usefully service in a mechanized society.
我觉得这个问题比其他问题更难。我不是说我对这个问题的看法比其他问题更悲观(虽然我确实看到一些挑战)。我的意思是,这个问题更模糊,也更难以提前预测,因为它涉及到社会如何组织的大问题,而这些问题往往需要时间并且以分散的方式才能得到解决。例如,历史上的狩猎采集社会可能会认为,没有狩猎和各种与狩猎相关的宗教仪式,生活是没有意义的,他们可能会觉得我们这个吃饱穿暖的技术社会是没有目的的。他们也可能不理解我们的经济是如何能满足每个人的需求,或者在一个机械化的社会中,人们可以发挥什么样的作用。
Nevertheless, it’s worth saying at least a few words, while keeping in mind that the brevity of this
section is not at all to be taken as a sign that I don’t take these issues seriously—on the
contrary, it
is a sign of a lack of clear answers.
不过,值得说几句话,虽然要注意这一部分的简短并不是说我不认真对待这些问题——恰恰相反,这正是缺乏明确答案的表现。
On the question of meaning, I think it is very likely a mistake to believe that tasks you undertake
are
meaningless simply because an AI could do them better. Most people are not the best in the world at
anything, and it doesn’t seem to bother them particularly much. Of course today they can still
contribute through comparative advantage, and may derive meaning from the economic value they
produce,
but people also greatly enjoy activities that produce no economic value. I spend plenty of time
playing
video games, swimming, walking around outside, and talking to friends, all of which generates zero
economic value. I might spend a day trying to get better at a video game, or faster at biking up a
mountain, and it doesn’t really matter to me that someone somewhere is much better at those things.
In
any case I think meaning comes mostly from human relationships and connection, not from economic
labor.
People do want a sense of accomplishment, even a sense of competition, and in a post-AI world it
will be
perfectly possible to spend years attempting some very difficult task with a complex strategy,
similar
to what people do today when they embark on research projects, try to become Hollywood actors, or
found
companies28. The facts that (a)
an AI
somewhere could in principle do this task better, and (b) this task is no longer an economically
rewarded element of a global economy, don’t seem to me to matter very much.
在意义的问题上,我觉得相信你所做的事情毫无意义,仅仅因为人工智能能做得更好,这种想法很可能是个错误。大多数人都不是任何领域的顶尖高手,这似乎也并没有特别困扰他们。当然,如今他们仍然能通过比较优势作出贡献,并可能从他们创造的经济价值中找到意义,但人们也非常享受那些没有经济价值的活动。我花很多时间玩视频游戏、游泳、在外面散步和和朋友聊天,这些活动都没有产生任何经济价值。我可能会花一天时间试着提升自己的游戏水平,或者骑自行车上山更快,至于其他地方有些人做得更好,我根本不在乎。总之,我认为意义主要来自人际关系和连接,而不是经济劳动。 人们确实渴望获得成就感,甚至渴望竞争感。在后人工智能的世界里,完全可以花上几年时间去尝试一些非常困难的任务,运用复杂的策略,这和人们今天在进行研究项目、努力成为好莱坞演员或创办公司时所做的类似28。至于(a)某个地方的人工智能在原则上可以做得更好,以及(b)这个任务不再是全球经济中经济上有回报的元素,我觉得这并不太重要。
The economic piece actually seems more difficult to me than the meaning piece. By “economic” in this
section I mean the possible problem that most or all humans may not be able to contribute
meaningfully to a sufficiently advanced AI-driven economy. This is a more macro problem than the
separate problem of inequality, especially inequality in access to the new technologies, which I
discussed in Section 3.
经济这部分对我来说实际上似乎比意义部分更难。这里所说的“经济”是指一个可能的问题,即 大多数或所有 人类可能无法对一个足够先进的人工智能驱动的经济做出有意义的贡献。这是一个比我在第三部分讨论的关于新技术获取的不平等问题更宏观的问题。
First of all, in the short term I agree with arguments that comparative advantage will continue to
keep
humans
relevant and in fact increase their productivity, and may even in some ways level the playing field between
humans. As long as AI is only better at 90% of a given job, the other 10% will cause humans
to
become highly leveraged, increasing compensation and in fact creating a bunch of new human jobs
complementing and amplifying what AI is good at, such that the “10%” expands to continue
to
employ almost everyone. In fact, even if AI can do 100% of things better than humans, but it
remains inefficient or expensive at some tasks, or if the resource inputs to humans and AI’s
are
meaningfully different, then the logic of comparative advantage continues to apply. One area humans
are
likely to maintain a relative (or even absolute) advantage for a significant time is the physical
world.
Thus, I think that the human economy may continue to make sense even a little past the point where
we
reach “a country of geniuses in a datacenter”.
首先,从短期来看,我同意比较优势的论点,它将继续让人类保持相关性,实际上还会提高他们的生产力,并可能在某些方面拉平人类之间的竞争。只要人工智能只在某项工作的 90%上表现更好,那剩下的 10%就会让人类有很大的杠杆作用,增加报酬,实际上还会创造出一大堆新的工作,以补充和放大人工智能擅长的领域,这样“10%”就扩展到几乎雇佣每一个人。事实上,即使人工智能能在 100%的事情上比人类做得更好,但如果它在某些任务上依然低效或昂贵,或者人类和人工智能的资源投入有显著差异,那么比较优势的逻辑仍然适用。人类可能在相当长一段时间内在物理世界保持相对(甚至绝对)优势。因此,我认为即使在我们达到“数据中心的天才国度”之后,人类经济仍然会继续合理。
However, I do think in the long run AI will become so broadly effective and so cheap that this will
no
longer apply. At that point our current economic setup will no longer make sense, and there will be
a
need for a broader societal conversation about how the economy should be organized.
不过,我觉得从长远来看,人工智能会变得非常有效且便宜,届时这种情况就不再适用了。到那时,我们现在的经济结构就不太合理了,会需要更广泛的社会讨论,关于经济应该怎么组织。
While that might sound crazy, the fact is that civilization has successfully navigated major economic
shifts in the past: from hunter-gathering to farming, farming to feudalism, and feudalism to
industrialism. I suspect that some new and stranger thing will be needed, and that it’s something no
one
today has done a good job of envisioning. It could be as simple as a large universal basic income
for
everyone, although I suspect that will only be a small part of a solution. It could be a capitalist
economy of AI systems, which then give out resources (huge amounts of them, since the overall
economic
pie will be gigantic) to humans based on some secondary economy of what the AI systems think makes
sense
to reward in humans (based on some judgment ultimately derived from human values). Perhaps the
economy
runs on Whuffie points. Or perhaps humans will continue to be economically valuable
after all, in some way not anticipated by the usual economic models. All of these solutions have
tons of
possible problems, and it’s not possible to know whether they will make sense without lots of
iteration
and experimentation. And as with some of the other challenges, we will likely have to fight to get a
good outcome here: exploitative or dystopian directions are clearly also possible and have to be
prevented. Much more could be written about these questions and I hope to do so at some later time.
虽然这听起来可能很疯狂,但事实是文明在过去成功地经历了重大的经济转型:从狩猎采集到农业,从农业到封建主义,再从封建主义到工业主义。我怀疑将会需要一些新的、更奇怪的东西,而这是现在没人能很好地想象出来的。可能只是给每个人提供一笔广泛的基本收入,尽管我怀疑这仅仅是解决方案的一小部分。也可能是一个 AI 系统的资本主义经济,然后根据 AI 系统认为该奖励人的一些次级经济,向人类分配资源(数量庞大,因为整体经济总量将是巨大的)。也许经济运行在Whuffie 积分上。或者,或许人类在某种意想不到的方式上继续具有经济价值,超出了传统经济模型的预期。所有这些解决方案都有许多潜在的问题,而且没有大量的迭代和实验,很难知道这些方案是否合理。 和其他一些挑战一样,我们可能不得不努力争取一个好的结果:剥削性或反乌托邦的方向显然也有可能出现,我们必须加以防范。关于这些问题可以写更多,希望将来有机会再讨论。
Taking stock 盘点一下
Through the varied topics above, I’ve tried to lay out a vision of a world that is both plausible
if everything goes right with AI, and much better than the world today. I don’t know if this
world is realistic, and even if it is, it will not be achieved without a huge amount of effort and
struggle by many brave and dedicated people. Everyone (including AI companies!) will need to do
their
part both to prevent risks and to fully realize the benefits.
通过上面各种话题,我试图描绘一个可能的世界,如果一切顺利发展 AI 的话,这个世界比今天的世界要好得多。我不知道这个世界是不是现实的,甚至如果是的话,也需要很多勇敢和奉献的人付出巨大的努力和奋斗才能实现。每个人(包括 AI 公司!)都需要尽自己的一份力,既要防范风险,又要充分实现收益。
But it is a world worth fighting for. If all of this really does happen over 5 to 10 years—the defeat
of
most diseases, the growth in biological and cognitive freedom, the lifting of billions of people out
of
poverty to share in the new technologies, a renaissance of liberal democracy and human rights—I
suspect
everyone watching it will be surprised by the effect it has on them. I don’t mean the experience of
personally benefiting from all the new technologies, although that will certainly be amazing. I mean
the
experience of watching a long-held set of ideals materialize in front of us all at once. I think
many
will be literally moved to tears by it.
但这是一个值得为之奋斗的世界。如果这一切在五到十年内真的发生——大多数疾病的战胜、生物和认知自由的增长、数十亿人摆脱贫困共享新技术、自由民主和人权的复兴——我想在场的每个人都会被它对他们产生的影响所惊讶。我不是说个人从所有新技术中获得好处的体验,尽管那肯定会很令人惊叹。我是指目睹一整套长期以来的理想一下子在我们面前实现的体验。我觉得很多人会被这一幕感动得流泪。
Throughout writing this essay I noticed an interesting tension. In one sense the vision laid out here
is
extremely radical: it is not what almost anyone expects to happen in the next decade, and will
likely
strike many as an absurd fantasy. Some may not even consider it desirable; it embodies values and
political choices that not everyone will agree with. But at the same time there is something
blindingly
obvious—something overdetermined—about it, as if many different attempts to envision a good world
inevitably lead roughly here.
在写这篇文章的过程中,我注意到了一种有趣的紧张感。一方面,这里提出的愿景非常激进:这不是几乎任何人预计在未来十年内会发生的事情,很多人可能会觉得这是个荒唐的幻想。有些人甚至可能认为这并不值得追求;它包含的价值观和政治选择并不是每个人都能认同的。但与此同时,还有一些显而易见的东西——一种被过度决定的东西——就像许多不同的尝试去设想一个美好的世界,最终都不可避免地会指向这里。
In Iain M. Banks’ The Player of Games29, the protagonist—a member of a society called the Culture, which
is
based on principles not unlike those I’ve laid out here—travels to a repressive, militaristic empire
in
which leadership is determined by competition in an intricate battle game. The game, however, is
complex
enough that a player’s strategy within it tends to reflect their own political and philosophical
outlook. The protagonist manages to defeat the emperor in the game, showing that his values (the
Culture’s values) represent a winning strategy even in a game designed by a society based on
ruthless
competition and survival of the fittest. A
well-known post by Scott Alexander has the same thesis—that competition is self-defeating
and
tends to lead to a society based on compassion and cooperation. The “arc of the moral
universe” is another similar concept.
在伊恩·M·班克斯的 游戏玩家29 中,主角是一个名为文化的社会的成员,这个社会的原则和我在这里提到的原则有点相似。他前往一个压迫和军事化的帝国,在这个帝国中,领导权通过复杂的战斗游戏竞争来决定。然而,这个游戏复杂到足以让玩家的策略反映他们自己的政治和哲学观点。主角成功地在游戏中打败了皇帝,显示出他的价值观(文化的价值观)即使在一个由无情竞争和适者生存原则构建的社会设计的游戏中,也代表了一种获胜的策略。斯科特·亚历山大的一个著名帖子持有同样的论点——竞争是自我失败的,往往导致一个基于同情和合作的社会。“道德宇宙的弧线”是另一个类似的概念。
I think the Culture’s values are a winning strategy because they’re the sum of a million small
decisions
that have clear moral force and that tend to pull everyone together onto the same side. Basic human
intuitions of fairness, cooperation, curiosity, and autonomy are hard to argue with, and are
cumulative
in a way that our more destructive impulses often aren’t. It is easy to argue that children
shouldn’t
die of disease if we can prevent it, and easy from there to argue that everyone’s children
deserve that right equally. From there it is not hard to argue that we should all band together and
apply our intellects to achieve this outcome. Few disagree that people should be punished for
attacking
or hurting others unnecessarily, and from there it’s not much of a leap to the idea that punishments
should be consistent and systematic across people. It is similarly intuitive that people should have
autonomy and responsibility over their own lives and choices. These simple intuitions, if taken to
their
logical conclusion, lead eventually to rule of law, democracy, and Enlightenment values. If not
inevitably, then at least as a statistical tendency, this is where humanity was already headed. AI
simply offers an opportunity to get us there more quickly—to make the logic starker and the
destination
clearer.
我认为文化的价值观是一种成功的策略,因为它们是众多小决策的总和,这些决策具有明确的道德力量,通常能让大家团结在同一边。基本的公平、合作、好奇心和自主权的直觉是很难反驳的,并且它们的累积效应往往比我们更具破坏性的冲动要强。我们很容易争辩,如果我们能预防,孩子们就不应该死于疾病,从这点出发,进一步争辩每个人的孩子都应该同样享有这种权利。接下来,我们可以合理地争辩说,我们应该团结一致,利用我们的智慧来实现这个目标。几乎没有人会反对那些无故攻击或伤害他人的人应该受到惩罚,从这里出发,进一步推论惩罚应该在所有人之间保持一致和系统化。人们应该对自己的生活和选择拥有自主权和责任,这一点同样直观。如果将这些简单的直觉推到逻辑的尽头,最终将导致法治、民主和启蒙价值观。 如果说不是必然的话,至少从统计趋势来看,人类本来就是朝这个方向走的。人工智能只是让我们更快到达这个目标的机会——让逻辑更明显,目的地更清晰。
Nevertheless, it is a thing of transcendent beauty. We have the opportunity to play some small role
in
making it real.
然而,这是一种超越美的事物。我们有机会在实现它的过程中扮演一些小角色。
Thanks to Kevin Esvelt, Parag Mallick, Stuart Ritchie, Matt Yglesias, Erik Brynjolfsson, Jim
McClave,
Allan Dafoe, and many people at Anthropic for reviewing drafts of this essay.
感谢凯文·埃斯维尔特、帕拉戈·马利克、斯图尔特·里奇、马特·伊格莱西亚斯、埃里克·布林约尔松、吉姆·麦克莱夫、艾伦·达福和在 Anthropic 的很多人审阅这篇文章的草稿。
To the winners of the 2024 Nobel prize in Chemistry, for showing us all the way.
致2024 年诺贝尔化学奖的获奖者,感谢你们为我们指明了方向。
Footnotes 脚注
-
1https://allpoetry.com/All-Watched-Over-By-Machines-Of-Loving-Grace ↩
-
2I do anticipate some minority of people’s reaction will be “this is pretty tame”. I think those people need to, in Twitter parlance, “touch grass”. But more importantly, tame is good from a societal perspective. I think there’s only so much change people can handle at once, and the pace I’m describing is probably close to the limits of what society can absorb without extreme turbulence. ↩
2我预计会有少部分人的反应是“这太温和了”。我觉得这些人需要在推特上说的“去接触大自然”。但更重要的是,从社会的角度来看,温和是好的。我认为人们能承受的变化量是有限的,而我所描述的节奏可能已经接近社会在没有极端动荡的情况下能够承受的极限。↩ -
3I find AGI to be an imprecise term that has gathered a lot of sci-fi baggage and hype. I prefer "powerful AI" or "Expert-Level Science and Engineering" which get at what I mean without the hype. ↩
3我觉得 AGI 是个不太准确的词,带了很多科幻色彩和炒作。我更喜欢“强大的人工智能”或者“专家级科学与工程”,这样说更贴近我的意思,没有那么多炒作。↩ -
4In this essay, I use "intelligence" to refer to a general problem-solving capability that can be applied across diverse domains. This includes abilities like reasoning, learning, planning, and creativity. While I use "intelligence" as a shorthand throughout this essay, I acknowledge that the nature of intelligence is a complex and debated topic in cognitive science and AI research. Some researchers argue that intelligence isn't a single, unified concept but rather a collection of separate cognitive abilities. Others contend that there's a general factor of intelligence (g factor) underlying various cognitive skills. That’s a debate for another time. ↩
4在这篇文章中,我用“智力”来指代一种可以应用于各种领域的普遍问题解决能力。这包括推理、学习、规划和创造力等能力。虽然我在整篇文章中用“智力”作为简写,但我承认智力的本质在认知科学和人工智能研究中是一个复杂且有争议的话题。一些研究者认为,智力并不是一个统一的概念,而是一组独立的认知能力。其他人则认为,存在一个普遍的智力因素(g 因子)在各种认知技能背后。这是另一个讨论的主题。↩ -
5This is roughly the current speed of AI systems – for example they can read a page of text in a couple seconds and write a page of text in maybe 20 seconds, which is 10-100x the speed at which humans can do these things. Over time larger models tend to make this slower but more powerful chips tend to make it faster; to date the two effects have roughly canceled out. ↩
5这大致上是目前 AI 系统的速度——比如它们可以在几秒钟内阅读一页文本,可能在 20 秒内写出一页文本,这个速度是人类的 10 到 100 倍。随着时间的推移,较大的模型往往会使这个速度变慢,但更强大的芯片会让它变快;到目前为止,这两种效果大致上是相互抵消的。↩ -
6This might seem like a strawman position, but careful thinkers like Tyler Cowen and Matt Yglesias have raised it as a serious concern (though I don’t think they fully hold the view), and I don’t think it is crazy. ↩
6这可能看起来像是一个虚假的立场,但像泰勒·科文和马特·伊格莱西亚斯这样的深思熟虑的思考者提出了这个问题,作为一个严肃的担忧(虽然我认为他们并不完全持有这个观点),而且我觉得这并不疯狂。↩ -
7The closest economics work that I’m aware of to tackling this question is work on “general purpose technologies” and “intangible investments” that serve as complements to general purpose technologies. ↩
7我知道的最接近这个问题的经济学研究是关于“通用技术”和“无形投资”的研究,这些无形投资作为通用技术的补充。↩ -
8This learning can include temporary, in-context learning, or traditional training; both will be rate-limited by the physical world. ↩
8这种学习可以包括临时的、在上下文中的学习,或者传统的培训;这两者都受到物理世界的限制。↩ -
9In a chaotic system, small errors compound exponentially over time, so that even an enormous increase in computing power leads to only a small improvement in how far ahead it is possible to predict, and in practice measurement error may degrade this further. ↩
9在一个混乱的系统中,小的错误会随着时间的推移成倍增加,因此即使计算能力大幅提升,也只会在预测的准确性上有小幅改善,实际上测量误差可能会进一步恶化这个问题。↩ -
10Another factor is of course that powerful AI itself can potentially be used to create even more powerful AI. My assumption is that this might (in fact, probably will) occur, but that its effect will be smaller than you might imagine, precisely because of the “decreasing marginal returns to intelligence” discussed here. In other words, AI will continue to get smarter quickly, but its effect will eventually be limited by non-intelligence factors, and analyzing those is what matters most to the speed of scientific progress outside AI. ↩
10另一个因素当然是强大的人工智能本身可能会被用来创造更强大的人工智能。我认为这可能(实际上,很可能会)发生,但它的影响会比你想象中小,正因为这里讨论的“智力的边际收益递减”。换句话说,人工智能会继续迅速变得更聪明,但其效果最终会受到非智力因素的限制,分析这些因素对于人工智能之外的科学进步速度才是最重要的。↩ -
11These achievements have been an inspiration to me and perhaps the most powerful existing example of AI being used to transform biology. ↩
11这些成就一直激励着我,可能是目前最强有力的人工智能应用于生物学转型的例子。 ↩ -
12“Progress in science depends on new techniques, new discoveries and new ideas, probably in that order.” - Sydney Brenner ↩
12“科学的进步依赖于新技术、新发现和新思想,顺序大概是这样的。” - Sydney Brenner↩ -
13Thanks to Parag Mallick for suggesting this point. ↩
13感谢 Parag Mallick 提到这一点。↩ -
14I didn't want to clog up the text with speculation about what specific future discoveries AI-enabled science could make, but here is a brainstorm of some possibilities:
14我不想在文本中塞满对具体内容的猜测 未来 AI 驱动的科学可能会取得的发现,不过这里有一些头脑风暴的想法 可能性:
— Design of better computational tools like AlphaFold and AlphaProteo — that is, a general AI system speeding up our ability to make specialized AI computational biology tools.
— 设计更好的计算工具,比如 AlphaFold 和 AlphaProteo — 也就是说,一种通用的 人工智能 系统加快了我们开发专业化 AI 计算生物学工具的能力。
— More efficient and selective CRISPR.
— 更高效、更精准的 CRISPR。
— More advanced cell therapies.
— 更先进的细胞疗法。
— Materials science and miniaturization breakthroughs leading to better implanted devices.
— 材料科学和微型化的突破,让植入物变得更好 设备。
— Better control over stem cells, cell differentiation, and de-differentiation, and a resulting ability to regrow or reshape tissue.
— 更好地控制干细胞、细胞分化和去分化,和一个 再生或重塑组织的能力。
— Better control over the immune system: turning it on selectively to address cancer and infectious disease, and turning it off selectively to address autoimmune diseases. ↩
更好地控制免疫系统:选择性地激活它来对抗癌症 感染性疾病,并有选择性地关闭它以应对自身免疫性疾病。 ↩ -
15AI may of course also help with being smarter about choosing what experiments to run: improving experimental design, learning more from a first round of experiments so that the second round can narrow in on key questions, and so on. ↩
15当然,人工智能也可以帮助我们更聪明地选择要进行的实验:改善实验设计,从第一轮实验中学到更多,以便第二轮能够更集中于关键问题,等等。↩ -
16Thanks to Matthew Yglesias for suggesting this point. ↩
16感谢马修·伊格莱西亚斯提出这一观点。↩ -
17Fast evolving diseases, like the multidrug resistant strains that essentially use hospitals as an evolutionary laboratory to continually improve their resistance to treatment, could be especially stubborn to deal with, and could be the kind of thing that prevents us from getting to 100%. ↩
17快速演变的疾病,比如那些多重耐药菌株,基本上把医院当成进化实验室,不断提高它们对治疗的抵抗力,可能特别难以处理,这可能是我们无法达到 100%的原因。↩ -
18Note it may be hard to know that we have doubled the human lifespan within the 5-10 years. While we might have accomplished it, we may not know it yet within the study time-frame. ↩
18注意,在 5 到 10 年内我们可能已经将人类的寿命增加了一倍,但这可能很难被察觉。虽然我们或许已经达成了这一目标,但在研究的时间框架内,我们可能还未意识到这一点。↩ -
19This is one place where I am willing, despite the obvious biological differences between curing diseases and slowing down the aging process itself, to instead look from a greater distance at the statistical trend and say “even though the details are different, I think human science would probably find a way to continue this trend; after all, smooth trends in anything complex are necessarily made by adding up very heterogeneous components. ↩
19在这里,尽管治愈疾病和减缓衰老过程之间的生物差异显而易见,我还是愿意从更远的角度来看待统计趋势,并说:“即使细节不同,我认为人类科学大概会找到继续这个趋势的方法;毕竟,在任何复杂的事物中,平滑的趋势都是通过将各种异质成分相加而形成的。” ↩ -
20As an example, I’m told that an increase in productivity growth per year of 1% or even 0.5% would be transformative in projections related to these programs. If the ideas contemplated in this essay come to pass, productivity gains could be much larger than this. ↩
20举个例子,我听说每年生产率增长 1%甚至 0.5%的提升,对这些项目的预测会有很大的影响。如果这篇文章中考虑的想法能够实现,生产率的提升可能会远远超过这个。↩ -
21The media loves to portray high status psychopaths, but the average psychopath is probably a person with poor economic prospects and poor impulse control who ends up spending significant time in prison. ↩
21媒体喜欢把高阶精神病者描绘得光鲜亮丽,但实际上,大多数精神病者可能只是那些经济前景糟糕、冲动控制差的人,最终会在监狱里待很长时间。↩ -
22I think this is somewhat analogous to the fact that many, though likely not all, of the results we’re learning from interpretability would continue to be relevant even if some of the architectural details of our current artificial neural nets, such as the attention mechanism, were changed or replaced in some way. ↩
22我觉得这有点像这样的事实:我们从可解释性中学到的许多结果,虽然可能不是全部,就算我们现在的人工神经网络的一些架构细节,比如注意力机制,被以某种方式改变或替换,这些结果仍然会继续相关。↩ -
23I suspect it is a bit like a classical chaotic system – beset by irreducible complexity that has to be managed in a mostly decentralized manner. Though as I say later in this section, more modest interventions may be possible. A counterargument, made to me by economist Erik Brynjolfsson, is that large companies (such as Walmart or Uber) are starting to have enough centralized knowledge to understand consumers better than any decentralized process could, perhaps forcing us to revise Hayek’s insights about who has the best local knowledge. ↩
23我怀疑这有点像一个经典的混沌系统——被不可简化的复杂性包围,必须以一种大部分去中心化的方式来管理。不过,正如我在本节稍后所说的,更温和的干预措施可能是可行的。经济学家埃里克·布林约尔夫森对我提出的一个反驳是,大公司(比如沃尔玛或优步)开始拥有足够的集中知识,以比任何去中心化的过程更好地理解消费者,这可能强迫我们重新审视哈耶克的见解,关于谁拥有最好的地方知识。↩ -
24Thanks to Kevin Esvelt for suggesting this point. ↩
24感谢凯文·埃斯维尔特提出这一观点。↩ -
25For example, cell phones were initially a technology for the rich, but quickly became very cheap with year-over-year improvements happening so fast as to obviate any advantage of buying a “luxury” cell phone, and today most people have phones of similar quality. ↩
25例如,手机最初是富人的科技,但随着每年的技术进步迅速,价格很快变得非常便宜,以至于购买“奢侈”手机的优势不复存在,今天大多数人拥有的手机质量都差不多。↩ -
26This is the title of a forthcoming paper from RAND, that lays out roughly the strategy I describe. ↩
26这是 RAND 即将发布的一篇论文的标题,它大致阐述了我所描述的策略。↩ -
27When the average person thinks of public institutions, they probably think of their experience with the DMV, IRS, medicare, or similar functions. Making these experiences more positive than they currently are seems like a powerful way to combat undue cynicism. ↩
27当普通人想到公共机构时,他们可能会想到自己在车管所、国税局、医疗保险或类似机构的经历。让这些经历变得比现在更积极,似乎是对抗过度悲观的有效方法。↩ -
28Indeed, in an AI-powered world, the range of such possible challenges and projects will be much vaster than it is today. ↩
28确实,在一个由人工智能驱动的世界里,这些可能面临的挑战和项目的范围将比现在要广得多。↩ -
29I am breaking my own rule not to make this about science fiction, but I’ve found it hard not to refer to it at least a bit. The truth is that science fiction is one of our only sources of expansive thought experiments about the future; I think it says something bad that it’s entangled so heavily with a particular narrow subculture. ↩
29我正在打破自己不把这事变成科幻的原则,但我发现很难不提一下。事实上,科幻是我们为数不多的关于未来的广阔思维实验来源之一;我认为这与一个特定狭窄的亚文化纠缠得这么深实在没什么好说的。 ↩