OmniFood8K：基于分层频域对齐融合的单图营养估计
OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion
单张 RGB 图像经深度估计后，用频域分层对齐融合 RGB 与深度特征，并用基于掩码的检测头动态选择通道突出关键食材区域；这套「伪深度+频域融合」思路可迁移到任何缺乏深度传感器的细粒度识别任务。
原文依据 we propose the Frequency-Aligned Fusion Module (FAFM) to hierarchically align and fuse RGB and depth features in the frequency domain
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论准确估计食物营养对促进健康饮食习惯和个性化饮食管理至关重要。现有食物数据集大多聚焦西式菜肴，对中国菜覆盖不足，限制了中国餐食的准确营养估计。此外，许多最优营养预测方法依赖深度传感器，限制了其在日常场景中的适用性。为解决这些局限，我们提出 OmniFood8K，一个包含 8,036 个食物样本的综合多模态数据集，每个样本均带有详细的营养标注和多视角图像。此外，为增强模型的营养预测能力，我们构建了 NutritionSynth-115K，一个大规模合成数据集，在保留精确营养标签的同时引入成分变化。我们还提出一个从单张 RGB 图像进行营养预测的端到端框架。首先，我们从单张 RGB 图像预测深度图，并设计尺度偏移残差适配器（SSRA）对其进行细化，以实现全局尺度一致性和局部结构保持。其次，我们提出频域对齐融合模块（FAFM），在频域中分层对齐并融合 RGB 与深度特征。最后，我们设计基于掩码的预测检测头（MPH），通过动态通道选择强调关键食材区域，从而实现更准确的预测。在多个数据集上的大量实验表明，我们的方法优于现有方法。项目主页：https://yudongjian.github.io/OmniFood8K-food/
英文原文摘要 1597 字符
Accurate estimation of food nutrition plays a vital role in promoting healthy dietary habits and personalized diet management. Most existing food datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes, which restricts accurate nutritional estimation for Chinese meals. Moreover, many state-of-the-art nutrition prediction methods rely on depth sensors, restricting their applicability in daily scenarios. To address these limitations, we introduce OmniFood8K, a comprehensive multimodal dataset comprising 8,036 food samples, each with detailed nutritional annotations and multi-view images. In addition, to enhance models' capability in nutritional prediction, we construct NutritionSynth-115K, a large-scale synthetic dataset that introduces compositional variations while preserving precise nutritional labels. Moreover, we propose an end-to-end framework for nutritional prediction from a single RGB image. First, we predict a depth map from a single RGB image and design the Scale-Shift Residual Adapter (SSRA) to refine it for global scale consistency and local structural preservation. Second, we propose the Frequency-Aligned Fusion Module (FAFM) to hierarchically align and fuse RGB and depth features in the frequency domain. Finally, we design a Mask-based Prediction Head (MPH) to emphasize key ingredient regions via dynamic channel selection for more accurate prediction. Extensive experiments on multiple datasets demonstrate the superiority of our method over existing approaches. Project homepage: https://yudongjian.github.io/OmniFood8K-food/