英文标题：High-probability guarantees for linear accessibility in feature superposition
作者：Enrico Vompa
arXiv ID：2609.09556 | 分类：stat.ML | 发表：2026-09-09
许可：CC-BY
摘要 神经网络可以利用特征叠加来编码比维度更多的概念，{{NL}}但跨特征干扰限制了同时激活特征的线性可访问性。{{NL}}通过将线性可访问性建模为压缩感知问题，{{NL}}我们推导了在次高斯噪声下固定支撑集的高概率界，{{NL}}证明了充分维度呈线性缩放（{{PT_MATH_1}}），而非先前的最坏情况二次限制。{{NL}}随后我们通过高斯尾部近似在系统参数上验证了这些界。{{NL}}这些结果量化了线性表示假设的几何约束，{{NL}}为评估稀疏自编码器、组合泛化和神经可解释性提供了框架。
Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear accessibility of simultaneously active features. By framing linear accessibility as a compressed sensing problem, we derive high-probability bounds for fixed supports under subgaussian noise, proving the sufficient dimension scales linearly ($d=O_{\varepsilon}(k \log m)$) rather than prior worst-case quadratic limits. We then validate these bounds across system parameters through Gaussian-tail approximations. These results quantify the geometric constraints of the linear representation hypothesis, providing a framework for evaluating sparse autoencoders, compositional generalization, and neural interpretability.
查看完整双语翻译 →
正在跳转到翻译阅读页… 如果没有自动跳转，请点击这里。