Problem Statement

The aim of the project was to create a virtual trainer for a Health application on TV that counts the repetition of exercises and tracks the posture of user performing exercises in front of the TV (through a webcam).

‍

Approach

Worked on training pose estimation and activity recognition models on proprietary data based on existing architecture of OpenPose. I also integrated these models into the iOS, Android and Tizen platform to be used in Health and other apps

‍

Key challenges

The major challenge was running the pose estimation model at 4 frame / second (requirement). A speed lower than this led to poor calculation of repetition counting. Most TVs only have a 4GB RAM (much lesser than phones). We implemented an interpolation algorithm that approximates the pose based on previous frame. We were able to run the model on significantly lower resolution image (thus increasing the speed of processing the frame) with the help of the above interpolation algorithm.

‍