Geometry Processing with Neural Fields
Pytorch implementation for the NeurIPS 2021 paper:
Most existing geometry processing algorithms use meshes as the default shape representation.
Manipulating meshes, however, requires one to maintain high quality in the surface discretization.
For example, changing the topology of a mesh usually requires additional procedures such as remeshing.
This paper instead proposes the use of neural fields for geometry processing.
Neural fields can compactly store complicated shapes without spatial discretization.
Moreover, neural fields are infinitely differentiable, which allows them to be optimized for objectives that involve higher-order derivatives.
This raises the question: can geometry processing be done entirely using neural fields?
We introduce loss functions and architectures to show that some of the most challenging geometry processing tasks, such as deformation and filtering, can be done with neural fields.
Experimental results show that our methods are on par with the well-established mesh-based methods without committing to a particular surface discretization.
This repository provides a Anaconda environment, and requires NVIDIA GPU to run the
The code is tested in CUDA 10.2, ubuntu 16.04.
The environment can be set-up using the following commands:
conda env create -f environment.yml conda activate NFGP
We offer pre-processed dataset for reproducing the experimental results in the paper.
The instruciton is in section Download Dataset.
If you want to run experiment for your own shape, you need to prepare data in the following two steps:
- Create SDF samples for the shape (See section Create SDF Samples)
- Train a neural field to fit those samples (See section Create Input Neural Fields).
- Depending on the tasks, create the user specified input. For deofmration task, please refer to section Create
Deformation Handles. For sharpening and smoothing, you can directly set it through
the configuration file or the hyper-parameter.
Create SDF Samples
Following command creates SDF ground truth samples for obtaining the Neural Fields that approximates the SDF of the
python scripts/prep_sdf_data.py <mesh_file_name> --out_path data/<mesh_file_name> # Example python scripts/prep_sdf_data.py armidillo.obj --out_path data/armidillo
The data preprocessing pipeline will sample 5M points uniformly withint
[-1, 1]^3 and another 5M near surface (i.e
. surface plus a Gaussian with standard deviation of
We then use the
mesh_to_sdf package to compute the SDF of the points.
These points will be saved to following files:
<out_path>/mesh.obj: a copy of the original mesh.
<out_path>/sdf.npy: a numpy file that contains the sampled points, their SDF, and the original mesh.
Note that this pipeline works the best with watertight mesh.
This processing pipeline takes about 5 – 10 minutes to finish for a mesh with 30k faces.
Create Input Neural Fields
Once you’ve obtained the
sdf.npy file in the previous subsection,
you can use those data to train a neural field:
python train.py configs/recon/create_neural_fields.yaml --hparams data.path=<your_sdf.npy>
You can also create your own config following the examples in folder
Create Deformation Handles
Smoothing and Sharpening
To run our methods on the smoothing or sharpening task, you can use the following configurations:
# Armadillo python train.py configs/filtering/filtering_Armadillo_beta0.yaml # smoothing python train.py configs/filtering/filtering_Armadillo_beta2.yaml # sharpening # Noisy sphere python train.py configs/filtering/filtering_HalfNoisySphere_beta0.yaml # smoothing python train.py configs/filtering/filtering_HalfNoisySphere_beta2.yaml # sharpening # Noisy torus python train.py configs/filtering/filtering_NoisyTorus_beta0.yaml # smoothing python train.py configs/filtering/filtering_NoisyTorus_beta2.yaml # sharpening
You can change the value of
beta by adding
To run it on a different shape, you need to change the
models.decoder to load the appropriate neural fields.