Research
Can body-model augmentations turn a small set of sign videos into a larger, useful training set for sign recognition?
Labeled sign-language video is scarce, so I am building a way to generate extra training clips that keep each sign's meaning. I convert MediaPipe landmarks to 3-D SMPL-X (body), MANO (hands) and a fitted head, perturb them within linguistically grounded limits, and convert them back. The conversion is measured and the first augmentation trials exist. The test that matters, a recognizer trained with and without the generated clips, has not been run.
Log
- Body round trip on 5 pilot clips: 3.52% direct error (shoulder-width units)
- 20-clip round trip: 3.13% body error, 5.10 mm hand mean
- Face geometry added; first augmentation trials found arm changes 3–4× too strong
- nextTrain the same recognizer with and without the augmented clips