技巧精选

用语音和多模态提示与AI代理交互的经验分享

Finally caved in, and I now fully speak to agents as opposed to typing prompts. My first realizatio...

精选理由

有人分享用语音+屏幕录制和多模态提示跟AI代理唠嗑,提示越啰嗦结果越靠谱,还教你怎么录屏加注释,值得试试

AI 摘要

作者完全改用语音而非文字输入与AI代理交互,发现音频描述越详细、越长,代理结果越好。他还开发了屏幕录制、截图、追踪鼠标动作和语音注释功能,帮助代理处理设计和精确开发任务。多模态提示(语音+屏幕+动作)显著提升了代理的可靠性,尽管消耗更多token。作者将这些经验制作为可复用的命令集,插入循环后效果显著改善。

原文 · elvis

Finally caved in, and I now fully speak to agents as opposed to typing prompts. My first realizatio...

Finally caved in, and I now fully speak to agents as opposed to typing prompts. My first realization is that you can just blabber on and tell the agent so many rich details via audio. The longer and the more detailed the audio explanation, the better the results. The most interesting thing about interacting with the agent this way is that I can parallelize more work and enable agents to perform way longer runs, implementing many things at once. In addition, I have developed a new feature where I can record the screen, take screenshots, track mouse actions and movements, annotate, and explain (using voice) to the agent things that it struggles with, like design and precise feature development. My finding is that the richer the prompt modality, the more reliable the agent results are. The noise (if any) doesn't even matter. Yes, it's more expensive (i.e., lots more tokens used this way), but the reliability that you are getting is worth it. I often store those as reusable commands/skills where it applies and inject them into loops. The results are night and day. 💬 11 🔄 5 ❤️ 28 👀 4171 📊 14 ⚡