跳转至

基于深度引导的拥挤场景视频目标计数

文章背景与核心概要

在拥挤且存在严重遮挡的动态视频场景中进行精确的目标计数，一直是计算机视觉领域的一大技术难题。传统的视频目标计数方法主要依赖纯 RGB 图像信息，这在面对密集遮挡和复杂空间关系时往往表现出极大的局限性。为了突破这一瓶颈，本文提出了一种全新的深度引导检测器（DG-Det），通过引入多尺度 RGB-D 交叉注意力机制和显式遮挡预测，大幅增强了模型对三维空间结构的理解能力。

除了核心的检测器架构外，本文还创新性地设计了一个统一的去重框架，有效消除了跨帧之间的重复计数冗余，并推出了首个包含深度信息及多目标类别的 RGB-D 视频目标计数数据集。实验结果表明，与现有的基线方法相比，该方法将平均绝对误差（MAE）显著降低了 62.01%，并在均方根误差（RMSE）指标上取得了持续的性能提升，为拥挤场景下的多模态感知研究奠定了坚实的基础。

Depth-Guided Video Object Counting in Crowded Scenes

Summary

This paper introduces a novel approach to video object counting in crowded and occluded environments, moving beyond traditional RGB-only limitations. The authors propose the Depth-Guided Detector (DG-Det), which integrates depth cues using multi-scale RGB-D cross-attention alongside explicit occlusion prediction to enhance spatial understanding. Additionally, the paper presents a unified de-duplication framework to eliminate cross-frame redundant counting and introduces a new RGB-D Video Object Counting dataset. Experimental results demonstrate a significant 62.01% reduction in MAE compared to existing baselines.

Paper Information

Authors

  • Yuanjing Xu
  • Xinyan Liu
  • Weidong Chen
  • Zixuan Zou
  • Linhao Zhang
  • Zhuangzhe Meng
  • Antoni B. Chan
  • Weigang Zhang

Abstract

我们的主要目标是推进拥挤场景下的视频目标计数研究，旨在根据给定的文本或视觉提示，稳健地统计目标类别的所有实例。现有方法依赖于 RGB 信息，这限制了它们在拥挤和遮挡条件下的判别能力。为了解决这一问题，我们提出了一种深度引导检测器（DG-Det）以及通用的后处理流程。通过将深度线索与多尺度 RGB-D 交叉注意力机制和显式遮挡预测相结合，我们的方法增强了空间理解能力，并在拥挤和遮挡场景中实现了稳健的检测。此外，我们引入了一个统一的去重框架，以消除跨帧冗余计数。为了促进未来的研究，我们还发布了一个全新的 RGB-D 视频目标计数数据集，其中包含深度信息以及每个序列的多个目标类别。大量实验表明，与现有基线相比，我们的方法在 MAE 上实现了 62.01% 的减少，并且在 RMSE 上也带来了持续的改进。

Our primary objective is to advance video object counting in crowded scenes, aiming to robustly count all instances of a target category based on given text or visual prompts. Existing methods rely on RGB information, limiting their discriminative ability in crowded and occluded conditions. To address this, we propose a Depth-Guided Detector (DG-Det) along with a general post-processing pipeline. By integrating depth cues with multi-scale RGB-D cross-attention and explicit occlusion prediction, our method enhances spatial understanding and achieves robust detection in crowded and occluded scenes. Furthermore, we introduce a unified de-duplication framework to eliminate cross-frame redundant counting. To facilitate future research, we also release a new RGB-D Video Object Counting dataset featuring depth information and multiple object categories per sequence. Extensive experiments demonstrate that our method achieves a 62.01% reduction in MAE compared to existing baselines, and also produces consistent improvements in RMSE.

License