Methodology
Here I detail the process of training the computer vision model and the analytical techniques that power the app’s insightful feedback on sprinting form.
The Model
- Gathering Tailored Data: The model is trained on a set of images I personally captured at different tracks. This approach allowed me to gather a specialized dataset with a broad range of sprinting styles and conditions. By photographing athletes in various positions and angles, I ensured a varied dataset that reflects the complexities of sprinting.
- Fine-Tuning for Sprint Analysis: Leveraging the YOLOv8 model and utilizing Python, I fine-tuned the CNN to specialize in detecting and analyzing crucial aspects of sprinting form, such as limb positioning and torso alignment. This fine-tuning process was essential to shift the model’s focus from general object detection to the specific nuances of athletic movement analysis.
-
Assumed Invariants
Here are some of the things I assumed were always true to make the program simpler and more effective.
- The camera doesn't change position or angle throughout the video.
- The runner is generally consistent with their form regardless of its quality.
- The camera angle is perpendicular to the runner's direction.
- The runner is in their drive stage (top speed) for the entirety of the video.
- The runner is the only person in the video.
- The runner stays on screen for at least 2 seconds.
- The runner goes on screen once, and then doesn't reappear after.
Thought Process
-
Angles
I found that determining angles between body parts from the data generated by the model is relatively simple and extremely insightful. It allows me to extract the specific types of feedback I want for all runners, regardless of size, and without dealing with quantitative measurements which I will get into later.
-
Qualitative Distances
The process of evaluating qualitative distances between body parts relies on the establishment of fixed reference points, or 'anchors.' Instead of measuring specific distances, this approach involves comparing the position of one body part to another based on predefined anchor points. Much like determining angles, this method allows the program to find boolean truths about the runner's form that aren't affected by the variance in size of different runners.
-
Going Forward With Quantitative Distances
While finding quantitative distances are significantly less valuable in giving feedback because of their dependence on the size and proportions of the runner, they can still provide useful data for users who want to do their own analysis. But getting this data is much more complex because users will take their videos from different distances and perspectives.
My solution, which is still being prototyped, is to calculate the distance in pixles between a predetermined pair of keypoints, say, the hip to shoulder, and then compare its distance to the target distances. Then the user could simply input the true distance (they would have to measure this) between the predetermined pair of points to find the target data using ratios.
-
Second Pass Approach
Another idea I've been tinkering with is a second pass approach, where before the main model goes through the video and allows the data to be analyzed, a less sophisticated, but more efficient, model will go through first and determine some preliminary information about the video and the runner. Right now, the only useful info that could be gathered from this approach is the direction that the sprinter is running in (left to right/right to left), so I've streamlined the process by having users enter the direction beforehand.
But if quantitative data were to be collect as mentioned above, finding the average distance between the predetermined pair of keypoints in the first pass might resolve a lot of complexity in the program and could potentially be worth doing.
Results
This program is able to return feedback based on:
- The optimal height of the knees during the sprint.
- The angle of the arms, suggesting improvements for optimal form.
- The lean of the torso, offering suggestions to lean forward or backward for better posture.
- The symmetry and coordination between left and right arm movements.
- Hand positioning relative to the eyes, indicating if arms are being raised too high or not high enough.
- Stride analysis, which examines the angle between the thighs to suggest a wider or narrower stride for efficiency.
- Visual cues indicating the alignment of shoulders and hips for maintaining balance.
If you encounter any issues or have any questions, please feel free to open an issue on this repository or contact me at isaac.saxonov@gmail.com.
Thank you for using StrideScan!