看见对话：第一人称视频中的交流情境识别
Seeing Conversations: Communication Context Identification in Egocentric Video
把「谁在和拍摄者交流」定义为一个可学习的第一人称视频任务，并用跨个体的长时序注意力建模社交动态；对做第一人称视频、多人交互理解或助听/AR 场景的人，这是一个可直接复用的任务设定与基线模型。
原文依据 We propose CoCoNet, a temporal interaction model for CCI that tracks social dynamics via attention across individuals over long time scales.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在日常对话中，人类借助注视方向或头部朝向等视觉线索毫不费力地识别出交流对象。要在计算机视觉中复现这种社会推理颇具挑战，尤其是在动态的多人场景下。我们引入第一人称视觉中的交流情境识别（Communication Context Identification, CCI）：给定一段第一人称视频序列，判定哪些个体正在与拍摄者进行交流。为支持 CCI，我们采集了一个具有挑战性的大规模数据集，包含在多样化的多人、多对话场景中拍摄的 68.9 小时第一人称视频。我们提出 CoCoNet，一个面向 CCI 的时序交互模型，通过个体间的注意力在长时间尺度上跟踪社交动态。CoCoNet 能灵活处理不同的群体规模，在遮挡情况下维持预测，即使在时序输入有限时也能稳健工作。借助长时序上下文，它在 CCI 上取得 96% 的平衡准确率。性能随群体规模和空间场景布局而变化，凸显了数据集多样性的重要性。本工作推进了基于视觉的对话感知，可支撑助听等应用——利用第一人称视频增强用户对话群体中的个体。
英文原文摘要 1271 字符
In everyday conversations, humans effortlessly recognize communication partners using visual cues such as gaze or head orientation. Replicating this social reasoning in computer vision is challenging, especially in dynamic, multi-person settings. We introduce Communication Context Identification (CCI) in egocentric vision: Given a first-person video sequence, determine which individuals are engaged in communication with the camera wearer. To support CCI, we collected a challenging large-scale dataset comprising 68.9 hours of egocentric video captured across diverse multi-person, multi-conversation scenarios. We propose CoCoNet, a temporal interaction model for CCI that tracks social dynamics via attention across individuals over long time scales. CoCoNet flexibly handles varying group sizes, maintains predictions through occlusions, and performs robustly even with limited temporal input. Leveraging long temporal contexts, it achieves 96% balanced accuracy on CCI. Performance varies with group size and spatial scene layout, highlighting the importance of dataset diversity. Our work advances vision-based conversational awareness, enabling applications in assistive hearing that use egocentric video to enhance individuals in the user's conversation group.