以触觉看世界：触觉驱动的材料区域视觉定位
Seeing Through Touch: Tactile-Driven Visual Localization of Material Regions
把触觉信号当作一种「查询模态」，用密集跨模态特征交互生成触觉显著性图，可直接迁移到文本/音频等非视觉查询驱动的区域定位与分割任务；其材料多样性配对策略也为弱信号条件下的跨模态对齐提供了可复用的数据增强思路。
原文依据 We propose a model that learns local visuo-tactile alignment via dense cross-modal feature interactions, producing tactile saliency maps
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文研究触觉定位问题，其目标是识别出与触觉输入具有相同材料属性的图像区域。现有的视触觉方法依赖全局对齐，因而无法捕捉该任务所需的细粒度局部对应关系。现有数据集大多只包含近距离、低多样性的图像，进一步加剧了这一挑战。我们提出一种模型，通过密集的跨模态特征交互学习局部视触觉对齐，生成触觉显著性图，用于以触摸为条件的材料分割。为克服数据集限制，我们引入：(i) 真实场景中的多材料场景图像，以扩展视觉多样性；(ii) 一种材料多样性配对策略，将每个触觉样本与视觉上多样但触觉上一致的图像配对，从而改善上下文定位并提升对弱信号的鲁棒性。我们还构建了两个新的触觉 grounding 材料分割数据集用于定量评测。在新构建的与现有的基准上的实验均表明，我们的方法在触觉定位上显著优于先前的视触觉方法。
英文原文摘要 1159 字符
We address the problem of tactile localization, where the goal is to identify image regions that share the same material properties as a tactile input. Existing visuo-tactile methods rely on global alignment and thus fail to capture the fine-grained local correspondences required for this task. The challenge is amplified by existing datasets, which predominantly contain close-up, low-diversity images. We propose a model that learns local visuo-tactile alignment via dense cross-modal feature interactions, producing tactile saliency maps for touch-conditioned material segmentation. To overcome dataset constraints, we introduce: (i) in-the-wild multi-material scene images that expand visual diversity, and (ii) a material-diversity pairing strategy that aligns each tactile sample with visually varied yet tactilely consistent images, improving contextual localization and robustness to weak signals. We also construct two new tactile-grounded material segmentation datasets for quantitative evaluation. Experiments on both new and existing benchmarks show that our approach substantially outperforms prior visuo-tactile methods in tactile localization.