BodyMAP - Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed

Abhishek Tandon*1, Anujraaj Goyal*1, Henry Clever2, Zackory Erickson1
1Carnegie Mellon University, 2NVIDIA
* Equal contribution
arXiv Video Code Data

Abstract

Accurately predicting the 3D human posture and the pressure exerted on the body while resting in bed, visualized as a body mesh (3D pose & shape) with a 3D pressure map, holds significant promise for healthcare applications, particularly, in the prevention of pressure ulcers. Current methods focus on singular facets of the problem---predicting only 2D/3D poses, generating 2D pressure images, predicting pressure only for certain body regions instead of the full body, or forming indirect approximations to the 3D pressure map. In contrast, we introduce BodyMAP, that jointly predicts the human body mesh and 3D applied pressure map across the entire human body using a depth image and its associated 2D pressure image from a pressure-sensing mattress. The 3D pressure map is represented as a pressure value at each mesh vertex, enabling precise localization of high-pressure regions on the body. Additionally, we present BodyMAP-WS, a new formulation of pressure prediction in which we implicitly learn pressure in 3D by aligning sensed 2D pressure images with a differentiable 2D projection from predicted 3D pressure maps. In evaluations with real-world human data, our method outperforms the current state-of-the-art technique by 25% on both body mesh and 3D applied pressure map prediction tasks for people in bed.



BodyMAP leverages a depth and pressure image of a person in bed covered by a blanket, to jointly predict the body mesh and a 3D pressure map of pressure distributed along the human body.

2D Pressure is insufficient

Distinct postures can have similar 2D pressure images. The insets of the 3D pressure map show pressure being applied to different areas demonstrating its use in localizing pressure that is applied on the human body.

Completmenting Modalities

Depth and pressure image complement each other. Image of overlaid modalities depicts the enhanced context available to the model.


More Results


Interpolate start reference image.

BibTeX

@inproceedings{tandon2024bodymap,
      title={BodyMAP-Jointly Predicting Body Mesh and 3D Applied Pressure Map for People in Bed},
      author={Tandon, Abhishek and Goyal, Anujraaj and Clever, Henry M and Erickson, Zackory},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
      pages={2480--2489},
      year={2024}
    }