Vercel launches agent-desktop for accessibility-tree based desktop automation
Vercel 新出的这个 agent desktop 思路很有意思，https://t.co/ouWngpDjHn 也是一个类似 computer use 的产品，用来自动化控制桌面应用。 假如记得...
The non-obvious point is that accessibility-tree automation trades some visual accuracy for speed and lower token cost, which could make long-running desktop agents far cheaper to operate. Builders experimenting with computer-use agents should test it against vision-based alternatives like Codex's computer use to see where the accuracy-versus-latency tradeoff lands for their workflows.
Vercel has released agent-desktop (agent-browser.dev/desktop), a computer-use style product that automates control of desktop applications. Instead of the screenshot-plus-vision approach used by Codex's computer use and similar tools, it reads the operating system's accessibility tree and returns a JSON snapshot with refs. The agent operates via these refs and re-reads the tree to verify actions, which the author notes should be faster and less token-intensive than vision-based methods. The design follows Vercel's earlier agent-browser tool. The author has not yet benchmarked agent-desktop and plans further testing.
Vercel 新出的这个 agent desktop 思路很有意思，https://t.co/ouWngpDjHn 也是一个类似 computer use 的产品，用来自动化控制桌面应用。 假如记得...
Vercel 新出的这个 agent desktop 思路很有意思， agent-browser.dev/desktop 也是一个类似 computer use 的产品，用来自动化控制桌面应用。 假如记得没错的话，codex 的 computer use 以及一系列类似的产品都是走截图加 vision 识别的操作过程，这样就比较费 token 而且速度比较慢。 agent-desktop 换成了读取操作系统的无障碍树，返回带 refs 的 JSON 快照，Agent 用 ref 操作，再重新读取验证，速度快 而且节省 token，效果怎么样，我还没有测，肯定没有视觉识别准，但是假如速度快也是完全可以试一下的，之后我会做一些测试。 和之前他们出的 agent-browser 思路是一模一样的，agent-browser 我现在还在用，是我很喜欢的一个工具。 💬 16 🔄 9 ❤️ 79 👀 14034 📊 47 ⚡
