Abstract

The extraction of 3D human pose and body shape details from a single monocular image is a significant challenge in computer vision. Traditional methods use RGB images, but these are constrained by varying lighting and occlusions. However, cutting-edge developments in imaging technologies have introduced new techniques such as single-pixel imaging (SPI) that can surmount these hurdles. In the near-infrared spectrum, SPI demonstrates impressive capabilities in capturing a 3D human pose. This wavelength can penetrate clothing and is less influenced by lighting variations than visible light, thus providing a reliable means to accurately capture body shape and pose data, even in difficult settings. In this work, we explore the use of an SPI camera operating in the NIR with time-of-flight (TOF) at bands 850–1550 nm as a solution to detect humans in nighttime environments. The proposed system uses the vision transformers (ViT) model to detect and extract the characteristic features of humans for integration over a 3D body model SMPL-X through 3D body shape regression using deep learning. To evaluate the efficacy of NIR-SPI 3D image reconstruction, we constructed a laboratory scenario that simulates nighttime conditions, enabling us to test the feasibility of employing NIR-SPI as a vision sensor in outdoor environments. By assessing the results obtained from this setup, we aim to demonstrate the potential of NIR-SPI as an effective tool to detect humans in nighttime scenarios and capture their accurate 3D body pose and shape.

© 2024 Optica Publishing Group

Deep-learning blurring correction of images obtained from NIR single-pixel imaging

Carlos Osorio Quero, Daniel Durini, Jose Rangel-Magdaleno, Jose Martinez-Carranza, and Ruben Ramos-Garcia
J. Opt. Soc. Am. A 40(8) 1491-1499 (2023)

Res-U2Net: untrained deep learning for phase retrieval and image reconstruction

Carlos Osorio Quero, Daniel Leykam, and Irving Rondon Ojeda
J. Opt. Soc. Am. A 41(5) 766-773 (2024)

Single-pixel imaging using physics enhanced deep learning

Fei Wang, Chenglong Wang, Chenjin Deng, Shensheng Han, and Guohai Situ
Photon. Res. 10(1) 104-110 (2022)

Underwater object detection and temporal signal detection in turbid water using 3D-integral imaging and deep learning

Rakesh Joshi, Kashif Usmani, Gokul Krishnan, Fletcher Blackmon, and Bahram Javidi
Opt. Express 32(2) 1789-1801 (2024)

Probabilistic 3D object recognition and pose estimation using multiple interpretations generation

Zhaojin Lu and Sukhan Lee
J. Opt. Soc. Am. A 28(12) 2607-2618 (2011)