GenMatter：用生成式物质模型感知物理对象
GenMatter: Perceiving Physical Objects with Generative Matter Models
把运动与外观特征层次化地组织成高斯粒子并聚类成可独立运动的实体，配合并行块吉布斯采样做硬件加速推理，为从光流/点轨迹中无监督地恢复可变形物体的3D结构与物体级分割提供了可复用的表示与采样方案。
原文依据 we propose a generative model that hierarchically groups low-level motion and appearance features into particles
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类视觉感知为理解基于运动的场景解释的计算原理提供了有价值的启示。人类能够稳健地检测并分割构成可独立移动物质块的移动实体，无论观察的是稀疏运动点、带纹理表面还是自然场景。相比之下，现有计算机视觉系统缺乏一种能在这些多样设定下通用的统一方法。受人类感知原理启发，我们提出一种生成模型，将低层运动与外观特征层次化地分组为粒子（表示局部物质的小型高斯），再将粒子分组为捕捉连贯且可独立移动的物理实体的簇。我们开发了一种基于并行块吉布斯采样的硬件加速推理算法，以恢复稳定的粒子运动与分组。我们的模型可处理不同类型的输入（随机点、风格化纹理或自然 RGB 视频），使其能在生物视觉成功而现有计算机视觉方法失效的设定下工作。我们在三个领域验证了这一统一框架：在二维随机点运动图上，我们的方法捕捉到人类的对象感知，包括在歧义条件下的分级不确定性；在受格式塔启发的伪装旋转物体数据集上，我们的方法从运动中恢复出正确的三维结构，从而得到准确的二维对象分割；在自然 RGB 视频上，我们的模型追踪构成可变形物体的移动三维物质，实现稳健的物体级场景理解。本工作由此建立了一个植根于人类视觉原理的、基于运动的感知通用框架。
英文原文摘要 1730 字符
Human visual perception offers valuable insights for understanding computational principles of motion-based scene interpretation. Humans robustly detect and segment moving entities that constitute independently moveable chunks of matter, whether observing sparse moving dots, textured surfaces, or naturalistic scenes. In contrast, existing computer vision systems lack a unified approach that works across these diverse settings. Inspired by principles of human perception, we propose a generative model that hierarchically groups low-level motion and appearance features into particles (small Gaussians representing local matter), and groups particles into clusters capturing coherently and independently moveable physical entities. We develop a hardware-accelerated inference algorithm based on parallelized block Gibbs sampling to recover stable particle motion and groupings. Our model operates on different kinds of inputs (random dots, stylized textures, or naturalistic RGB video), enabling it to work across settings where biological vision succeeds but existing computer vision approaches do not. We validate this unified framework across three domains: on 2D random dot kinematograms, our approach captures human object perception including graded uncertainty across ambiguous conditions; on a Gestalt-inspired dataset of camouflaged rotating objects, our approach recovers correct 3D structure from motion and thereby accurate 2D object segmentation; and on naturalistic RGB videos, our model tracks the moving 3D matter that makes up deforming objects, enabling robust object-level scene understanding. This work thus establishes a general framework for motion-based perception grounded in principles of human vision.