Understanding AI Application Architecture
了解 AI 应用程序架构
What does an AI application look like, really? After having built a few, it seems like a good time to break it down into its (many) parts—and explore the operational implications.
AI 应用程序到底是什么样的？在构建了一些之后，现在似乎是将其分解为（许多）部分并探索运营影响的好时机。
We’ve previously explored what’s the same (a lot) and what’s different (a lot) about AI applications. It’s easy enough to define an AI application as “an application that leverages AI, either predictive or generative.”
我们之前已经探讨了 AI 应用程序的相同 （很多） 和不同 （很多） 之处。将 AI 应用程序定义为“利用 AI 的应用程序，无论是预测性的还是生成的性”，都很容易。
That’s true, but for the folks who must build, deploy, deliver, secure, and operate these new kinds of applications, what that does mean?
这是真的，但对于必须构建、部署、交付、保护和操作这些新型应用程序的人来说，这意味着什么？
It means new moving parts, for one. And not just inferencing services. There are other new components that are becoming standard adds to the application architecture, like vector databases. It also means increased reliance on APIs.
首先，这意味着新的活动部件。而不仅仅是推理服务。还有其他一些新组件正在成为应用程序架构的标准添加，例如矢量数据库。这也意味着对 API 的依赖增加。
It is also true that most of these components are containerized. When I set up a new AI application project, I fire up a container running PostgreSQL to use as a vector data store. That’s to support RAG (Retrieval Augmented Generation), which a significant percentage of AI applications are using according to multiple industry reports (this one from Databricks for example). That’s because I can augment a stock AI model using any data I like without having to fine-tune or train a model myself. For a significant number of use cases, that’s the best way to implement an AI application.
这些组件中的大多数都是容器化的，这也是事实。当我设置新的 AI 应用程序项目时，我启动了一个运行 PostgreSQL 的容器，以用作矢量数据存储。这是为了支持 RAG（检索增强生成），根据多份行业报告（例如来自 Databricks 的报告），很大一部分 AI 应用程序正在使用它。那是因为我可以使用我喜欢的任何数据来增强库存 AI 模型，而无需自己微调或训练模型。对于大量使用案例，这是实现 AI 应用程序的最佳方式。
I may also fire up a knowledge graph, typically by running a containerized graph database like Neo4J. Graph databases are particularly useful when working with graph-affine data, like social networks, recommendation engine data, and fraud detection. It turns out they’re also useful in mitigating hallucinations related to policy generation, as we learned early in the year. Inclusion of a graph database can add a new protocol, GraphQL, to the list of ‘new things that need securing.’
我还可以启动知识图谱，通常是通过运行像 Neo4J 这样的容器化图形数据库。图形数据库在处理图形仿射数据（如社交网络、推荐引擎数据和欺诈检测）时特别有用。事实证明，正如我们在今年年初了解到的那样，它们也有助于减轻与政策制定相关的幻觉。包含图形数据库可以将新协议 GraphQL 添加到“需要保护的新事物”列表中。
Then I’m going to decide which inferencing service I want to use. I have options. I could use a cloud provider service or AI as a service (a la ChatGPT) or I could run a local service. For example, my latest tinkering has used Ollama and phi3 on my MacBook. In this case, I’m only running one copy of the inferencing server because, well, running multiple would consume more resources than I have. In production, of course, it’s likely there would be more instances to make sure it can support demand.
然后，我将决定我想使用哪种推理服务。我有选择。我可以使用云提供商服务或 AI 即服务（就像 ChatGPT），或者我可以运行本地服务。例如，我最近的修改在我的 MacBook 上使用了 Ollama 和 phi3。在本例中，我只运行推理服务器的一个副本，因为运行多个副本会消耗比我拥有的资源更多的资源。当然，在生产环境中，可能会有更多的实例来确保它能够支持需求。
Because I’m going to use phi3, I choose phidata as my framework for accessing the inferencing service. I’ve also used langchain when taking advantage of AI as a service, a popular choice, and the good news is that from an operational perspective, the framework is not something that operates outside the core AI application itself.
因为我要使用 phi3，所以我选择 phidata 作为访问推理服务的框架。在利用 AI 即服务时，我还使用了 langchain，这是一个流行的选择，好消息是，从操作的角度来看，该框架并不是在核心 AI 应用程序本身之外运行的东西。
Operational Impact of AI Applications
AI 应用程序的运营影响
I haven’t even written a line of code yet and I’ve already got multiple components running, each accessed via API and running in their own containers. That’s why we start with the premise that an AI application is a modern application and brings with it all the same familiar challenges for delivery, security, and operation. It’s also why we believe deploying AI models will radically increase the number of APIs in need of delivery and security.
我什至还没有写一行代码，我已经运行了多个组件，每个组件都通过 API 访问并在自己的容器中运行。这就是为什么我们从 AI 应用程序是现代应用程序开始的前提开始，它带来了交付、安全性和操作方面所有相同的熟悉挑战。这也是为什么我们相信部署 AI 模型将从根本上增加需要交付和安全性的 API 数量。
运营挑战是相加的，每个新架构都会增加新的挑战或放大现有挑战。
AI applications add another tier to an already complex environment which, of course, increases complexity. Which means AI application architecture is going to increase the load on all operations functions, from security to SRE to the network. Each of these components also has their own scaling profile; that is, some components will need to scale faster than others and how requests are distributed will vary simply because most of them are leveraging different APIs and, in some cases, protocols.
AI 应用程序为本已复杂的环境增加了另一层，这当然会增加复杂性。这意味着 AI 应用程序架构将增加所有运营功能的负载，从安全性到 SRE 再到网络。这些组件中的每一个也有自己的扩展配置文件;也就是说，某些组件需要比其他组件更快地扩展，并且请求的分发方式会有所不同，这仅仅是因为它们中的大多数都利用了不同的 API，在某些情况下，还利用了协议。
What’s more, AI application architecture is going to be highly variable internally. That is, the AI application I build is likely to exhibit different local traffic characteristics and needs than the AI application you build. And more heterogeneity naturally means greater complexity because it’s the opposite of standardization, which promotes homogeneity.
更重要的是，AI 应用程序架构在内部将高度可变。也就是说，我构建的 AI 应用程序可能表现出与您构建的 AI 应用程序不同的本地流量特征和需求。更多的异质性自然意味着更大的复杂性，因为它与标准化相反，标准化促进了同质性。
Standardization has been, for decades, the means through which enterprises accelerate delivery of new capabilities to the market and achieve greater operational efficiencies while freeing up the human resources needed to address the variability in AI application architectures.
几十年来，标准化一直是企业加速向市场交付新功能并实现更高运营效率，同时释放解决 AI 应用程序架构可变性所需的人力资源的方法。
This is why we see some shared services—particularly app and API security—shifting to the edge, a la security as a service. Such a set of shared services can not only better protect workloads across all environments (core, cloud, and edge) but provide a common set of services that can serve as a standard across AI applications.
这就是为什么我们看到一些共享服务（尤其是应用程序和 API 安全）转向边缘，就像安全即服务一样。这组共享服务不仅可以更好地保护所有环境（核心、云和边缘）中的工作负载，还可以提供一组通用服务，作为 AI 应用程序的标准。
Understanding the anatomy of AI applications will help determine not just the types of application delivery and security services you need—but where you need them.
了解 AI 应用程序的剖析不仅有助于确定您需要的应用程序交付和安全服务的类型，还有助于确定您需要它们的位置。
