ProductsOfficialSelected

Exa 借助 AWS Inferentia2/Trainium2 芯片将 Embedding 吞吐提升 47%

Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

Why we picked it

搜索公司 Exa 把 embedding 推理成本和吞吐同时压降了 47%,靠的是在 AWS Inferentia2/Trainium2 上手写 NKI kernel,而不是堆 H100。做 RAG 或大规模索引的开发者值得看看这套降本思路,单位成本能差出一条产品线。

搜索引擎公司 Exa 公开了一项基础设施优化:通过 AWS Neuron 定制 NKI kernel,在 Inferentia2 和 Trainium2 芯片上跑 embedding 推理,吞吐和每百万文档成本均降低约 47%。Exa 的神经网络搜索依赖自研向量数据库,需要对数百亿网页文档做 embedding,推理成本直接决定商业模式能否成立。

Image · AWS Machine Learning Blog
Body

技术上,团队没有停留在框架默认配置,而是用 AWS 提供的 NKI(Neuron Kernel Interface)手写底层 kernel,针对自家 embedding 模型做算子级优化,绕开了通用 GPU 路线的开销。这与常见的 H100 部署方案形成对比:专用 AI ASIC 在大批量、单一模型推理场景下单位成本更低。

对处理海量文档的 RAG 与搜索类创业公司,这是一份可参考的成本工程案例——当推理单价压到足够低,索引全网规模的语料才具备商业可行性。

Original · AWS Machine Learning Blog

Exa boosts embedding throughput by 50% to index billions of documents on AWS AI chips

Exa's neural search engine is backed by an in-house vector database, so the unit economics of embedding tens of billions of web documents drive the business. This post shares how Exa optimized embedding inference on AWS Neuron hardware (Inferentia2 and Trainium2) with custom NKI kernels, reaching a 47% gain in performance and cost per million documents.

Industry Coordinates & Source Verification

This report is categorized under AITOP's "Products" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Exa, AWS, Inferentia2, Trainium2.

Primary source verification: published directly by AWS Machine Learning Blog with first-party authority.