神经符号强化学习方法用于投资组合合规分配
Internalizing Investment Mandates: A Neurosymbolic RL Approach to Portfolio Allocation
机构组合必须遵守投资授权规则，普通RL动不动就违规。这篇论文把六条合规规则学进策略里，违规率从96-98%降到4.8%，不需要事后拦截。做量化、资管或RL合规应用的人都该看看这套蒙特卡洛采样思路。
一篇arXiv论文（arXiv:2611.17329）提出PLPG启发的连续动作分配方法，将投资政策声明中的合规要求内化进强化学习策略。现有神经符号方法如Probabilistic Logic Policy Gradient（PLPG）只能处理有限离散动作空间，限制其在组合分配中的使用。新方法用重参数化蒙特卡洛采样估计策略执行动作满足投资授权的概率，并将其加入训练目标，推理时不设事后拦截机制，合规性由智能体自行学习。
在确定性均值动作、无修正的审计条件下，该策略将六条规则、状态条件授权的违规率从无约束RL的96-98%、奖励惩罚基线的33.4%降至4.8%。结果表明连续动作神经符号策略可以实质性内化符号化授权，为机构资金在真实合规约束下的RL部署提供路径。
Internalizing Investment Mandates: A Neurosymbolic RL Approach to Portfolio Allocation
Institutional portfolios operate under investment policy statements that specify asset allocation constraints, often conditioned on market regime information. Deep reinforcement learning has been shown to optimize financial performance but typically does not enforce rule compliance, making it unsuitable for real-world deployment. Existing neurosymbolic methods such as Probabilistic Logic Policy Gradient (PLPG) offer constraint-satisfying reinforcement learning (RL) via differentiable symbolic supervision but assume a finite, discrete action space. We present a PLPG-inspired continuous allocation method that estimates the probability that the policy's own executed action satisfies the investment mandate, using reparameterized Monte Carlo sampling, and adds it to the training objective. No post-hoc enforcement mechanism intercepts actions at inference time and compliance is learned by the agent. Audited directly on deterministic mean actions with no correction, the policy reduces violations of a six-rule regime-conditional mandate from 96-98% for unconstrained RL and 33.4% for a reward-penalty baseline to 4.8%, showing that a continuous-action neurosymbolic policy can substantially internalize a symbolic mandate.
This report is categorized under AITOP's "Papers" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and AI regulation, compliance frameworks, and sovereign compute policy. Key entities and technical topics under continuous tracking include 强化学习, 神经符号, 量化投资, PLPG.
Source attribution: curated from arXiv cs.LG (@Zeynep Yıldız, Satchit Chatterji, Ercan Atam, Erman Acar).