Geometry Processing with Neural Fields

Pytorch implementation for the NeurIPS 2021 paper:

Geometry Processing with Neural Fields

Guandao Yang,
Serge Belongie,
Bharath Hariharan,
Vladlen Koltun


Most existing geometry processing algorithms use meshes as the default shape representation.
Manipulating meshes, however, requires one to maintain high quality in the surface discretization.
For example, changing the topology of a mesh usually requires additional procedures such as remeshing.
This paper instead proposes the use of neural fields for geometry processing.
Neural fields can compactly store complicated shapes without spatial discretization.
Moreover, neural fields are infinitely differentiable, which allows them to be optimized for objectives that involve higher-order derivatives.
This raises the question: can geometry processing be done entirely using neural fields?
We introduce loss functions and architectures to show that some of the most challenging geometry processing tasks, such as deformation and filtering, can be done with neural fields.
Experimental results show that our methods are on par with the well-established mesh-based methods without committing to a particular surface discretization.


This repository provides a Anaconda environment, and requires NVIDIA GPU to run the
optimization routine.
The code is tested in CUDA 10.2, ubuntu 16.04.
The environment can be set-up using the following commands:

conda env create -f environment.yml
conda activate NFGP


We offer pre-processed dataset for reproducing the experimental results in the paper.
The instruciton is in section Download Dataset.
If you want to run experiment for your own shape, you need to prepare data in the following two steps:

  1. Create SDF samples for the shape (See section Create SDF Samples)
  2. Train a neural field to fit those samples (See section Create Input Neural Fields).
  3. Depending on the tasks, create the user specified input. For deofmration task, please refer to section Create
    Deformation Handles
    . For sharpening and smoothing, you can directly set it through
    the configuration file or the hyper-parameter.

Download Dataset

Available soon.

Create SDF Samples

Following command creates SDF ground truth samples for obtaining the Neural Fields that approximates the SDF of the

python scripts/ <mesh_file_name> --out_path data/<mesh_file_name>

# Example
python scripts/ armidillo.obj --out_path data/armidillo

The data preprocessing pipeline will sample 5M points uniformly withint [-1, 1]^3 and another 5M near surface (i.e
. surface plus a Gaussian with standard deviation of 0.1).
We then use the mesh_to_sdf package to compute the SDF of the points.
These points will be saved to following files:

  • <out_path>/mesh.obj : a copy of the original mesh.
  • <out_path>/sdf.npy : a numpy file that contains the sampled points, their SDF, and the original mesh.

Note that this pipeline works the best with watertight mesh.
This processing pipeline takes about 5 – 10 minutes to finish for a mesh with 30k faces.

Create Input Neural Fields

Once you’ve obtained the sdf.npy file in the previous subsection,
you can use those data to train a neural field:

python configs/recon/create_neural_fields.yaml --hparams data.path=<your_sdf.npy>

You can also create your own config following the examples in folder configs/recon.

Create Deformation Handles

Available soon.

Smoothing and Sharpening

To run our methods on the smoothing or sharpening task, you can use the following configurations:

# Armadillo
python configs/filtering/filtering_Armadillo_beta0.yaml  # smoothing
python configs/filtering/filtering_Armadillo_beta2.yaml  # sharpening
# Noisy sphere
python configs/filtering/filtering_HalfNoisySphere_beta0.yaml # smoothing
python configs/filtering/filtering_HalfNoisySphere_beta2.yaml # sharpening
# Noisy torus
python configs/filtering/filtering_NoisyTorus_beta0.yaml # smoothing
python configs/filtering/filtering_NoisyTorus_beta2.yaml # sharpening

You can change the value of beta by adding --hparams trainer.beta=<yourbeta>.
To run it on a different shape, you need to change the models.decoder to load the appropriate neural fields.


Available soon.


View Github