Kaizen: Computer Vision Based Interactive Karate Training Platform

Abstract

All types of martial arts consist of several forms of combat used in self-defense, which are deeply rooted in many countries. Of all the martial art types, karate is considered the most well-known out of them all. Due to the pandemic situation in Sri Lanka, karate enthusiasts have lost the opportunity to train in a well-guided environment. As a result, even though virtual training came into play, it has continuously proved its ineffectiveness in evaluating the performance and accuracy of the trainees. The main objective of this proposed system is to virtualize the processes of a physical karate dojo. Kaizen - A Computer Vision-Based Interactive Karate Training Platform is a web-based application that functions as a virtual instructor. The proposed system consists of two main core components for Training and Assessments. The karate training component evaluates the techniques against a set of predefined joint angles. The BlazePose model is used for keypoint detection, and Analytic Geometry is used to extract joint angles. It is also integrated with Amazon Polly, a Deep Learning-based Text-To-Speech (TTS) service to produce real-time audio feedback. The assessment component has the capability to evaluate the trainees through a built-in Smart Evaluator based on a Recurrent Neural Network (RNN). Additionally, the capability to manage the assessments supports the instructors in conducting all the assessments virtually, overcoming the barriers in physical training.