论文标题
学习一种快速的3D光谱方法，以进行对象细分和在时空上跟踪
Learning a Fast 3D Spectral Approach to Object Segmentation and Tracking over Space and Time
论文作者
论文摘要
我们将视频对象分割作为空间和时间上的光谱图聚类，每个像素的一个图节点和边缘形成本地时空社区。我们声称该视频图中最强的群集代表了显着对象。首先，我们引入了一种基于3D滤波的新颖而有效的方法，用于近似光谱解决方案，作为图形邻接矩阵的主要特征向量，而无需明确构建矩阵。此关键属性使我们可以在GPU上具有快速的并行实现，比计算特征向量的经典方法快的数量级。我们采用光谱时空聚类方法的动机，在视频语义分割文献中独有的是，这种聚类致力于随着时间的推移维护对象一致性，我们使用新颖的分割一致性度量进行评估。此外，我们展示了如何通过多个输入特征通道有效地学习解决方案。最后，我们将方法的公式扩展到分段任务之外，并将其扩展到对象跟踪领域。在广泛的实验中，我们对顶级方法以及结合它们结合的强大合奏表现出显着的改进，并在多个基准上实现了用于跟踪和细分的多个基准。
We pose video object segmentation as spectral graph clustering in space and time, with one graph node for each pixel and edges forming local space-time neighborhoods. We claim that the strongest cluster in this video graph represents the salient object. We start by introducing a novel and efficient method based on 3D filtering for approximating the spectral solution, as the principal eigenvector of the graph's adjacency matrix, without explicitly building the matrix. This key property allows us to have a fast parallel implementation on GPU, orders of magnitude faster than classical approaches for computing the eigenvector. Our motivation for a spectral space-time clustering approach, unique in video semantic segmentation literature, is that such clustering is dedicated to preserving object consistency over time, which we evaluate using our novel segmentation consistency measure. Further on, we show how to efficiently learn the solution over multiple input feature channels. Finally, we extend the formulation of our approach beyond the segmentation task, into the realm of object tracking. In extensive experiments we show significant improvements over top methods, as well as over powerful ensembles that combine them, achieving state-of-the-art on multiple benchmarks, both for tracking and segmentation.
