LlamaIndex创始人聊文档处理的新思路，把评估和自动化优化结合起来，适合做复杂文档抽取的人看看。
LlamaIndex创始人Jerry Liu在X上发文，强调评估和hillclimbing在文档处理中的重要性。他认为未来FDE（领域专家）的工作将转向定义业务问题、编码评估标准，并利用自动化优化流程。他提到可以使用Claude Code或Codex来优化工作流，而FDE则专注于确保目标和评估的正确性。该推文还邀请有复杂文档处理需求的团队联系他们。
We're not Palantir, but we do think a lot about evals and hillclimbing w.r.t. document processing. ...
We're not Palantir, but we do think a lot about evals and hillclimbing w.r.t. document processing. If you have really hairy problems around large-scale extraction over complex, real-world document corpuses that require specific constraints around accuracy and/or cost, come talk to us! We'll work closely with your team to make sure that it's well optimized. llamaindex.ai/contact Jerry Liu @jerryjliu0 The future of FDE work seems closely related with all work around evals/posttraining/RL envs. FDEs are effectively responsible for the following: 1. Define the business problem. 2. Codify the business problem into an eval rubric and environment. 3. Hillclimb the environment and output an agent/agentic workflow that solves the business problem. Right now the process of (3) is quite manual - historically FDEs spend hundreds of hours creating bespoke software/workflows that solve the problem. But assuming intelligence is abundant, they can effectively offload (3) to some automated optimization process. This includes RL on the model layer, and using Claude Code/Codex to optimize the harness/workflow. Then the FDE responsibility shifts from implementing the task to defining the right goals and outcomes. In other words, they have access to /goal, and their job is more around making sure the goal, environment, and evals are correct vs. the tactical implementation details. 🔗 View Quoted Tweet 💬 0 🔄 0 ❤️ 0 👀 91 ⚡