想给RNN换个思路?这篇论文提出用多个顺序投影挖掘结构依赖,实验显示在复杂数据上更稳,简单数据也不掉链子。
本文提出有序结构依赖假说(OSDH),认为同一观测的多个可接受排序能揭示单一序列无法捕捉的互补结构依赖。为验证该假说,作者提出独立结构专家原则(ISEP),独立训练各投影专属序列模型,再通过融合模型整合表征。具体实现为结构演化RNN(SE-RNN),保持底层循环计算不变,仅增加投影多样性。在三个不同复杂度合成数据集上的实验表明,SE-RNN在存在隐藏结构依赖时持续受益于多投影,在简单数据集上仍具竞争力。该方法独立于底层序列模型,可扩展至其他架构。
Recurrent Neural Networks Beyond Time: Learning from Multiple Ordered Projections
Recurrent neural networks (RNNs) are widely used for sequence learning, yet their application is commonly associated with temporal data, although recurrent computation fundamentally operates on ordered sequences rather than on time itself. Building on this observation, we introduce the Ordered Structural Dependency Hypothesis (OSDH), which proposes that multiple admissible orderings of the same observations may reveal complementary structural dependencies inaccessible through a single sequential organization. To operationalize this hypothesis, we propose the Independent Structural Expert Principle (ISEP), whereby projection-specific sequence models are trained independently before their learned representations are integrated through a dedicated fusion model. As a concrete realization, we present Structural Evolution RNNs (SE-RNNs), which employ conventional RNNs as projection-specific structural experts while preserving the underlying recurrent computation unchanged. Proof-of-concept experiments on three synthetic datasets with substantially different levels of structural complexity demonstrate that the proposed architecture consistently benefits from multiple ordered projections when hidden structural dependencies are present, while remaining competitive on simpler datasets. Since OSDH is independent of the underlying sequence-processing model, the proposed framework naturally extends beyond recurrent networks and may be instantiated using alternative architectures. The results suggest a general computational perspective for exploiting complementary ordered representations across diverse structured learning problems.