论文标题
3D荧光笔：通过文本说明在3D形状上的本地化区域
3D Highlighter: Localizing Regions on 3D Shapes via Text Descriptions
论文作者
论文摘要
我们提出了3D荧光笔，这是一种使用文本作为输入将语义区域定位在网格上的技术。我们系统的关键特征是能够解释“不域外”本地化的能力。我们的系统展示了推理在输入3D形状上将非相关概念放置在何处的能力，例如将衣服添加到裸露的3D动物模型中。我们的方法使用神经字段和使用概率加权的混合物对相应的形状区域进行颜色文本描述。我们的神经优化由预先训练的剪辑编码器指导，该剪辑编码器绕过了对任何3D数据集或3D注释的需求。因此，3D荧光笔具有很高的灵活性，一般并且能够在无数输入形状上产生局限性。我们的代码可在https://github.com/threedle/3dhighlighter上公开获取。
We present 3D Highlighter, a technique for localizing semantic regions on a mesh using text as input. A key feature of our system is the ability to interpret "out-of-domain" localizations. Our system demonstrates the ability to reason about where to place non-obviously related concepts on an input 3D shape, such as adding clothing to a bare 3D animal model. Our method contextualizes the text description using a neural field and colors the corresponding region of the shape using a probability-weighted blend. Our neural optimization is guided by a pre-trained CLIP encoder, which bypasses the need for any 3D datasets or 3D annotations. Thus, 3D Highlighter is highly flexible, general, and capable of producing localizations on a myriad of input shapes. Our code is publicly available at https://github.com/threedle/3DHighlighter.
