CloudAAE

This is an tensorflow implementation of "CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds"

Files

  1. log: directory to store log files during training.
  2. losses: loss functions for training.
  3. models: a python file defining model structure.
  4. object_model_tfrecord: full object models for data synthesizing and visualization purpose.
  5. tf_ops: tensorflow implementation of sampling operations (credit: Haoqiang Fan, Charles R. Qi).
  6. trained_network: a trained network.
  7. utils: utility files for defining model structure.
  8. ycb_video_data_tfRecords: synthetic training data and real test data for the YCB video dataset.
  9. evaluate_cloudAAE_ycbv.py: script for testing object 6d pose estimation with a trained network on test set in YCB video dataset.
  10. train_cloudAAE_ycbv.py: script for training a network on synthetic data for YCB objects.

Requirements

Test a trained network

  1. Testing data in tfrecord format is available
  • Download zip file
  • Unzip and place all files in ycb_video_data_tfRecords/test_real/

  1. After activate tensorflow

    python evaluate_cloudAAE_ycbv.py --trained_model trained_network/20200908-204328/model.ckpt --batch_size 1 --target_cls 0

  • --trained_model: directory to trained model (*.ckpt).
  • --batch_size: 1.
  • --target_class: target class for pose estimation.
  • Translation prediction is in unit meter.
  • Rotation prediction is in axis-angle format.
  1. Result
  • If you turn on visualization with b_visual=True, you will see the following displays which are partially observed point cloud segments (red) overlaid with object model (green) with pose estimates. The reconstructed point cloud is also presented (blue).
  • The coordinate is the object coordinate, object segment is viewed in the camera coordinate

0

0_recon

14_recon

14

Train a network

  1. Training data is created synthetically using 3D object model and 6D poses.
  • The 6D pose and class id of target object are in ycb_video_data_tfRecords/train_syn/
  • The data synthesis pipeline takes the target 3D object model and creates a segment of the object in the desired 6D pose. Below is two examples of synthetic segment (red), two real segments (red) are also shown for comparison.

syn_and_real_seg_lamp

syn_and_real_seg_phone

  1. Run script

    python train_cloudAAE_ycbv.py

  2. Log files and trained model is store in log

GitHub

https://github.com/GeeeG/CloudAAE