Andy Hall和Gwern讨论了后AGI世界怎么用AI搞民主，Gwern的守护天使能让每个普通人都拥有一个随时代表自己的AI代理人，还能在几分钟内模拟几百个议员的辩论，挺有启发的思想实验。
Andy Hall在System Check中警告后AGI世界可能走向极权，但AI也是升级民主的最大机会。Gwern的"Guardian Angels"论文详细设计了代表每个人的AI代理，能实时学习用户价值观并模拟国会辩论。该代理需超越数字孪生，在争议问题上主动引导用户思考。文章指出，若依赖单一封闭模型，民主将受制于私人公司，因此开源模型和自主运行代理至关重要。
Interesting.
Interesting. Andy Hall @ahall_research Freedom in the post-AGI world means building political superintelligence with tireless, brilliant political agents who represent us, the people—not governments or companies. In a special July 4th issue of System Check, I get into what this might mean. Several forces tilt the post-AGI world toward totalitarianism: the concentration of resources required to train frontier models, AI's obvious uses for surveillance and control, and existential risks that could justify extreme security states. But AI is also the biggest opportunity to upgrade democracy since the printing press. Most of our governance failures happen because citizens are too busy to pay attention, so a small group of highly motivated wackos drives the process. (See: NIMBYism.) What if that changed? What if AI could give every person a super effective political agent that represents them all the time? @gwern 's new "Guardian Angels" essay is the most serious technical sketch I've seen of that agent—one that learns you deeply, remembers everything, and can carry out "direct democracy on unprecedented scale." His most vivid example: official GAs for every member of Congress, able to simulate a roundtable debate among hundreds of politicians within minutes, or convene an emergency session at 4AM while every human is asleep. I see two big open questions. First: the agent has to be more than a digital twin. It should share your values without freezing your less-considered opinions in amber — willing to push you on topics it has studied more deeply than you have. On contested political questions, AI models don’t seem to possess that capability yet. Second: who governs the guardian angels? Gwern proposes a startup with dual-class founder shares. Sensible for the development phase. But can we build democratic infrastructure on private rails that one company controls forever? Which is why the recent attention back toward open-weight models and orgs that own their own models matters (see the great interview between @satyanadella and @ypatil125 below). The same logic driving firms to want their own models applies to democracy, too. If we're going to own our agents—agents that answer to us, and can't be secretly commanded from afar—we may need models we can run ourselves. The counterarguments to the open model idea (RSI leaving open models behind, safety pressure on open weights) are really big though, and I really have no idea how this is going to play out, or even how it should play out. How are we going to run a democracy if every citizen's agent is built on a single closed model with a single point of control? That's the question to think about this Independence Day. Check out the full piece here: freesystems.substack.com/p/guardian-ang… 🔗 View Quoted Tweet 💬 7 🔄 1 ❤️ 29 👀 16996 📊 7 ⚡