video-to-pose tools

This library simplifies installing and using several pose estimation systems.

Download

git clone https://github.com/ZurichNLP/video-to-pose
cd video-to-pose

How to use

To install a specific estimator on your system:

bash install.sh --type openpose [--slurm]

Then to estimate poses for a folder of input videos:

bash videos_to_poses.sh \
    --type openpose \
    --input /path/to/videos \
    --output /path/to/poses \
    [--device cpu|gpu] [--slurm] [...]

Arguments

Parameter
--type	Which pose estimator to use. Available choices: openpose, mediapipe, alphapose, simplest_x, openpifpaf, mmposewholebody, sdpose, sapiens
--input	Path to folder of videos
--output	Path to folder of .pose files
--device (optional)	cpu or gpu. Not all estimators support both; unsupported combinations will fail with a clear error.
--slurm (optional)	Whether to submit estimation jobs to a SLURM queue instead of executing directly. Only supported for openpose and alphapose; all other estimators will fail with a clear error if passed.
...	Any additional named arguments are passed through to the estimator-specific run script

Installing may also differ if --slurm is used, so if you are working on a SLURM cluster, also use --slurm for the installation script above.

For additional, estimator-specific arguments, see the respective estimator's README file (in the estimators sub-folders).

For instance, for the openpose estimator, if --slurm is used, a further argument --chunks can be passed like so:

bash videos_to_poses.sh \
    --type openpose \
    --input /path/to/videos \
    --output /path/to/poses \
    --slurm \
    --chunks 20

to specify the number of chunks to split the input videos into, and the number of resulting jobs to submit.

Pose estimators included

Estimator	Exact version, details	--device support	--slurm support	Requirements
openpose	137-keypoint model	GPU only	Yes	Singularity CE >= 3.x or Apptainer >= 1.x, NVIDIA GPU with CUDA drivers
mediapipe	MediaPipe Holistic, model complexity 2, with iris refinement	CPU only	No	Python 3
alphapose	136-keypoint HALPE_136 whole-body model (default), or 133-keypoint COCO WholeBody	GPU only	Yes	Singularity CE >= 3.x or Apptainer >= 1.x, NVIDIA GPU with CUDA drivers
simplest_x	SMPLest-X-H, 139-keypoint Extended SMPL-X whole-body model (25 body + 21L/21R hand + 72 face + 2 duplicated wrist roots)	GPU only	No	Python 3, NVIDIA GPU with CUDA drivers
openpifpaf	133-keypoint COCO WholeBody model (shufflenetv2k30-wholebody)	CPU or GPU	No	Python 3.9 or 3.10
mmposewholebody	133-keypoint COCO WholeBody model	CPU or GPU (auto-detected)	No	Python 3, NVIDIA GPU with CUDA strongly recommended
sdpose	133-keypoint COCO WholeBody model	CPU or GPU	No	Python 3, NVIDIA GPU with CUDA strongly recommended
sapiens	38-keypoint Goliath model with sapiens_1b_goliath_best_goliath_AP_640	CPU or GPU (auto-detected)	For installation only	NVIDIA GPU with CUDA strongly recommended

Details on output pose format

Estimated poses are converted to the binary .pose format of the pose-format library. All pose-format utilities to store, load, manipulate and visualize the poses can be applied.

SLURM usage

If --slurm is used, then pose estimation jobs are submitted with SLURM commands, assuming a SLURM login node that has sbatch, for instance. These commands are meant as an example and are tailored to a UZH SLURM cluster. They are not expected to run without some modifications on other clusters.

Testing

Estimator-specific tests live in the test/ directory. See test/README.md for details on how to run them.

Acknowledgements

@misc{obrien-et-al-2026video-to-pose, 
    title={Convenience code for installing and using several pose estimation systems},
    author={O'Brien, Catherine and Sant, Gerard and M{\"u}ller, Mathias},
    howpublished={\url{https://github.com/ZurichNLP/video-to-pose}},
    year={2026}
}

@misc{obrien2026comparing,
  title        = {Comparing Pose Estimators for Sign Language Translation},
  author       = {O'Brien, Catherine and Sant, Gerard and M\"{u}ller, Mathias and Ebling, Sarah},
  year         = {2026},
  eprint       = {2604.24609},
  archivePrefix= {arXiv},
  primaryClass = {cs.CL},
  url          = {https://arxiv.org/abs/2604.24609},
}