论文标题
深度微分森林，对表格数据的关注很少
Deep differentiable forest with sparse attention for the tabular data
论文作者
论文摘要
我们提出了深层森林及其稀疏注意机制的一般建筑。可区分的森林具有树木和神经网络的优势。它的结构是一棵简单的二进制树，易于使用和理解。它具有完全的可不同性，所有变量都是可学习的参数。我们将通过基于梯度的优化方法进行训练，该方法在Deep CNN的培训中显示出强大的力量。我们发现并分析了可区分森林中的注意力机制。也就是说，每个决定仅取决于一些重要特征，而其他决定则无关紧要。注意总是很稀疏。基于此观察结果，我们通过数据感知初始化提高了它的稀疏性。我们使用属性重要性来初始化注意力重量。那么，学习的重量比随机初始化的重量要稀少。我们在一些大型表格数据集上的实验表明，可区分的森林的准确性高于GBDT，GBDT是表格数据集的最先进的算法。源代码可在https://github.com/closest-git/quantumforest上找到
We present a general architecture of deep differentiable forest and its sparse attention mechanism. The differentiable forest has the advantages of both trees and neural networks. Its structure is a simple binary tree, easy to use and understand. It has full differentiability and all variables are learnable parameters. We would train it by the gradient-based optimization method, which shows great power in the training of deep CNN. We find and analyze the attention mechanism in the differentiable forest. That is, each decision depends on only a few important features, and others are irrelevant. The attention is always sparse. Based on this observation, we improve its sparsity by data-aware initialization. We use the attribute importance to initialize the attention weight. Then the learned weight is much sparse than that from random initialization. Our experiment on some large tabular dataset shows differentiable forest has higher accuracy than GBDT, which is the state of art algorithm for tabular datasets. The source codes are available at https://github.com/closest-git/QuantumForest
