A Human Body Simulation Using Semantic Segmentation and Image-Based Reconstruction Techniques for Personalized Healthcare

Abstract

The global healthcare market is expanding, with a particular focus on personalized care for individuals who are unable to leave their homes due to the COVID-19 pandemic. However, the implementation of personalized care is challenging due to the need for additional devices, such as smartwatches and wearable trackers. This study aims to develop a human body simulation that predicts and visualizes an individual’s 3D body changes based on 2D images taken by a portable device. The simulation proposed in this study uses semantic segmentation and image-based reconstruction techniques to preprocess 2D images and construct 3D body models. It also considers the user’s exercise plan to enable the visualization of 3D body changes. The proposed simulation was developed based on human-in-the-loop experimental results and literature data. The experiment shows that there is no statistical difference between the simulated body and actual anthropometric measurement with a p-value of 0.3483 in the paired t-test. The proposed simulation provides an accurate and efficient estimation of the human body in a 3D environment, without the need for expensive equipment such as a 3D scanner or scanning uniform, unlike the existing anthropometry approach. This can promote preventive treatment for individuals who lack access to healthcare. © 2024 by the authors.