论文标题
仔细研究小数据的深度学习
A Close Look at Deep Learning with Small Data
论文作者
论文摘要
在这项工作中，我们在有限尺寸的数据集上进行了各种各样的实验，具有不同的深度学习体系结构。根据我们的研究，我们表明，当每个类别只有几个样本可用时，模型的复杂性是关键因素。与文献不同，我们表明，在某些配置中，可以使用低复杂性模型来改进最新技术。例如，在稀缺训练样本和没有数据增强的问题时，低复杂性卷积神经网络的性能比最先进的体系结构相当或更好。此外，我们表明，即使是标准的数据增强也可以通过大幅度提高识别性能。该结果表明，当数据受到限制时，开发了更复杂的数据生成/增强管道。最后，我们表明辍学是一种广泛使用的正规化技术，即使数据稀缺，也可以保持其作为良好正规化器的作用。我们的发现在流行的CIFAR-10，Fashion-Mnist和SVHN基准的子采样版本上得到了经验验证。
In this work, we perform a wide variety of experiments with different deep learning architectures on datasets of limited size. According to our study, we show that model complexity is a critical factor when only a few samples per class are available. Differently from the literature, we show that in some configurations, the state of the art can be improved using low complexity models. For instance, in problems with scarce training samples and without data augmentation, low-complexity convolutional neural networks perform comparably well or better than state-of-the-art architectures. Moreover, we show that even standard data augmentation can boost recognition performance by large margins. This result suggests the development of more complex data generation/augmentation pipelines for cases when data is limited. Finally, we show that dropout, a widely used regularization technique, maintains its role as a good regularizer even when data is scarce. Our findings are empirically validated on the sub-sampled versions of popular CIFAR-10, Fashion-MNIST and, SVHN benchmarks.
