This repository contains a grasp generation demo and the training code of Grasping Field: Learning Implicit Representations for Human Grasps
The following code samples hand grasps conditioned on given object meshes.
- pytorch (with GPU)
You can run the following command in the command line to install all the dependencies.
pip3 install -r requirements.txt
The code is tested on Ubuntu 18.04 with python3
Download MANO pickle data-structures
- Go to MANO website
- Create an account by clicking Sign Up and provide your information
- Download Models and Code (the downloaded file should have the format
mano_v*_*.zip). Note that all code and data from this download falls under the MANO license.
- unzip and copy the
modelsfolder into the
- Your folder structure should look like this:
mano/ models/ MANO_LEFT.pkl MANO_RIGHT.pkl ... webuser/ ...
This may take a few minutes.
The demo takes objects from
./input as input, generates 5 samples of grasping hand under
./output/meshes/ contains the raw sdf reconstruction and
./output/mano contains the fitted mano. You can use meshlab to visualize them. Please load the object and hand together for visualization.
The model in
./pretrained_model is trained only on the ObMan dataset.
We include two sample objects from the YCB dataset in
./input. These sample meshes are for demo purpose only, please downlond the full dataset from the website. Note that these objects are also used in the HO3D dataset.
New objects can be given to the model by providing the path to the meshes (
./input) and the list of object (
input.json). The object needs to be reachable when a hand wrist is at the origin and should not be in
[-x,-z] quadrant (see example meshes).
Pre-processing the Data
For data preparation, we modified the point sampling code from DeepSDF to compute the distance to both hand and object, and to find the associate hand-part label for each point. This can be done with the
scripts/sample_points/preprocess_data.py executable. The preprocessing code is in C++ and has the following requirements:
With these dependencies, the build process follows the standard CMake procedure:
mkdir build cd build cmake .. make -j
Once this is done there should be two executables in the
scripts/sample_points/bin directory, one for surface sampling and one for SDF sampling. With the binaries, the dataset can be preprocessed using
More information on the compilation process can be found on DeepSDF
Training a Model
Once data has been preprocessed, models can be trained using:
python train.py -e <experiment_directory>
Parameters of training are stored in a "specification file"(
specs.json) in the experiment directory.
We sincerely acknowledge: Lars Mescheder and Michael Niemeyer for the detailed discussions on implicit function. Dimitrios Tzionas, Omid Taheri, and Yana Hasson for insightful discussions on MANO and the baseline. Partha Ghosh and Qianli Ma for the help with implementation of the VAE. Benjamin Pellkofer for IT/hardware supports.