FreiHAND - Toolbox

FreiHAND is a dataset for evaluation and training of deep neural networks for estimation of hand pose and shape from single color images, which was proposed in our paper. Its current version contains 32560 unique training samples and 3960 unique evaluation samples. The training samples were recorded with a green screen background allowing for background removal. We provide 3 additional sample_versions for each training sample using different post processing strategies yielding a total training set size of 130240 samples.

This repository contains a collection of scripts that show how the dataset can be used. See the project page for additional information.

Basic setup

  1. Download the dataset. See project page for instructions.

  2. Install basic requirements:

    virtualenv -p python2.7 ./venv
source venv/bin/activate
pip install numpy matplotlib scikit-image transforms3d tqdm opencv-python cython

  3. Assuming ${DB_PATH} is the path to where you unpacked the dataset (path to where ./training/ and ./evaluation/ folder branch off).
    This should enable you to run the following to show some dataset samples.
    In my case ${DB_PATH} holds the value ~/FreiHAND_pub_v1/

    python view_samples.py ${DB_PATH}
python view_samples.py ${DB_PATH} --show_eval

The script provides a couple of other parameters you might want to try. Note that for visualization of the hand shape you need to follow the Advanced setup.

Advanced setup (allows for visualization of MANO shape annotation)

  1. Download Models&code from the MANO website

    http://mano.is.tue.mpg.de

  2. Assuming ${MANO_PATH} contains the path to where you unpacked the downloaded archive use the provided script to enable visualization of the hand shape fit.

    python setup_mano.py ${MANO_PATH}

  3. Install OpenDR

    Getting OpenDR installed can be tricky. Maybe you are lucky and the pip install works for your system.

    pip install opendr

    But there is a known issue with a library. On my system the pip install didn't work and I installed the package using:

    bash install_opendr.sh

  4. Visualize samples with rendered MANO shapes

    python view_samples.py ${DB_PATH} --mano

Evaluate on the dataset

In order to ensure a fair and consistent protocol, evaluation of your algorithm on FreiHAND is handled through Codalab.

  1. Make predictions for the evaluation dataset. The code provided here predicts zeros for all joints and vertices.

    python pred.py ${DB_PATH}

  2. Zip the pred.json file

    zip -j pred.zip pred.json

  3. Upload pred.zip to our Codalab competition website (Participate -> Submit)

  4. Wait for the evaluation server to report back your results and publish your results to the leaderboard. The zero predictor will give you the following results

    Keypoint error 70.79cm
Keypoint error aligned 4.87cm
Mesh error 70.84cm
Mesh error aligned 5.12cm
F@5mm=0.0, F@15mm=0.0
F_aliged@5mm= 0.001, F_aligned@15mm=0.029

  5. Modify pred.py to use your method for making shape prediction and see how well it performs compared to the baselines in our leaderboard.

Terms of use

This dataset is provided for research purposes only and without any warranty. Any commercial use is prohibited.
If you use the dataset or parts of it in your research, you must cite the respective paper.

```
@InProceedings{Freihand2019,
  author    = {Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russel, Max Argus and Thomas Brox},
  title     = {FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images},
  booktitle    = {IEEE International Conference on Computer Vision (ICCV)},
  year      = {2019},
  url          = {"https://lmb.informatik.uni-freiburg.de/projects/freihand/"}
}
```

GitHub