oSATCo App Development:
Data Annotation
A placeholder description and if this description had more words
how would that look?
Aims
- Annotate key points on the body capturing variation of body shape and posture
- Annotate video frames with SATCo scoring, with intention to associate with movement described by the key points
Solution Overview
Data Annotation: Motivation
- Training and verification of AI: Requires a lot of expert examples to work
- Interpretable solution: Can be visually verified by experts and can also lead to knowledge discovery about movement disorders and movement analysis systems
- General solution: Provides a basis for movement analysis in children with movement disability
Why Artificial Intelligence?
Example Annotation (Side)
- Key-points on the body:
- Spine (C7-L1)
- Head (Ear, Eye)
- Arms (5x Shoulder-Finger)
- Thighs (Hip, Knee)
- Key information:
- External support
- Self-support
- Instructions:
- Don’t annotate low-confidence parts
- Quality Assurance
- Expert review + team discussion
Example Annotation (Side & Front)
Example Annotation (Back & Front)
Jobs4Students Annotators (June 2024)
- Students annotated 22,340 frames
- Supervision hours: ~115
- Student annotation hours: ~1,250
Annotation Statistics (June 2024)
- Children: 101
- Cameras (average): 2.94
- Images: 25,510
- Front: 11,369 (44.6%)
- Side: 11,227 (44.0%)
- Back: 2,914 (11.4%)
Annotation Statistics (December 2024)
- Participants 186
- Children: 179 Adults 7
- Frames 48096
- Avg Frames 61.35
- SATCo Scores: 258 ( inc longitudinal)
- GMFM Scores 101
- CMFCS Scores 101
Quality Assurance
- Annotations should be precise and consistent
- Experts (Penny) to validate annotations and team to discuss
- Machine learning models require representative variation
- Range of children (body types, eyewear, clothing, headwear, hair)
- Range of testers (body types, eyewear, clothing, headwear, hair)
- Range of poses – same child
- Different pose – same background
- Different background – same pose
- Natural occlusion
- Natural scene variation (bubbles, toys)
- Anything unusual (edge cases are valuable)
Bespoke Annotation Software
- C++ / OpenCV
- Load / play / step multiple synchronised videos
- Mark frame range
- Cycle frames / key frames
- Click / drag / copy / edit / delete points
