利用基础模型实现准确的材质分类
Harnessing the Power of Foundation Models for Accurate Material Classification
提出利用视觉语言基础模型进行自动标注和先验蒸馏的流程，可复用于其他数据稀缺的细粒度分类任务；其合成数据生成与自动标注管道为视觉研究提供了数据增广新思路。
原文依据 a robust image generation and auto-labeling pipeline that creates a diverse and high-quality training dataset with material-centric images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论材质分类已成为计算机视觉和图形学中的关键任务，支持为广泛的数字和现实世界应用分配准确的材质属性。虽然传统上被框定为图像分类任务，但由于标注数据的稀缺，该领域面临重大挑战，限制了训练模型的准确性和泛化能力。视觉语言基础模型（VLM）的最新进展为解决这些问题提供了有希望的途径，然而现有利用这些模型的解决方案在材质识别任务中仍表现出不令人满意的结果。在这项工作中，我们提出了一个新颖的框架，有效利用基础模型来克服数据限制并提高分类准确性。我们的方法集成了两个关键创新：（a）一个稳健的图像生成和自动标注管道，创建具有以材质为中心的图像的多样的高质量训练数据集，并通过融合文本提示中的对象语义和材质属性来自动分配标签；（b）一种先验引入策略，用于从VLM中蒸馏信息，结合联合微调方法，优化预训练的视觉基础模型与VLM衍生的先验，在适应材质特定特征的同时保持广泛的泛化能力。大量实验表明在多个数据集上取得了显著改进。我们展示了我们的合成数据集有效捕捉了真实世界材质的特征，并且来自视觉语言模型的先验的集成显著提高了最终性能。
英文原文摘要 1624 字符
Material classification has emerged as a critical task in computer vision and graphics, supporting the assignment of accurate material properties to a wide range of digital and real-world applications. While traditionally framed as an image classification task, this domain faces significant challenges due to the scarcity of annotated data, limiting the accuracy and generalizability of trained models. Recent advances in vision-language foundation models (VLMs) offer promising avenues to address these issues, yet existing solutions leveraging these models still exhibit unsatisfying results in material recognition tasks. In this work, we propose a novel framework that effectively harnesses foundation models to overcome data limitations and enhance classification accuracy. Our method integrates two key innovations:(a) a robust image generation and auto-labeling pipeline that creates a diverse and high-quality training dataset with material-centric images, and automatically assigns labels by fusing object semantics and material attributes in text prompts; (b) a prior incorporation strategy to distill information from VLMs, combined with a joint fine-tuning method that optimizes a pre-trained vision foundation model alongside VLM-derived priors, preserving broad generalizability while adapting to material-specific features.Extensive experiments demonstrate significant improvements on multiple datasets. We show that our synthetic dataset effectively captures the characteristics of real world materials, and the integration of priors from vision-language models significantly enhances the final performance.