Map Metrics for Trajectory Quality
Map metrics toolkit provides a set of metrics to quantitatively evaluate trajectory quality via estimating consistency of the map aggregated from point clouds.
GPS or Motion Capture systems are not always available in perception systems, or their quality is not enough (GPS on small-scale distances) for use as ground truth trajectory. Thus, common full-reference trajectory metrics (APE, RPE, and their modifications) could not be applied to evaluate trajectory quality. When 3D sensing technologies (depth camera, LiDAR) are available on the perception system, one can alternatively assess trajectory quality --- estimate the consistency of the map from registered point clouds via the trajectory.
Documentation: https://map-metrics.readthedocs.io.
Features
Our toolkit provides implementation of the next metrics:
- Mean Map Entropy (MME), Mean Plane Variance(MPV) [1] [2]
- Mutually Orthogonal Metric (MOM) [3] -- has strong correlation with RPE
Citation
If you use this toolkit or MOM-metric results, please, cite our work:
@misc{kornilova2021benchmark,
title={Be your own Benchmark: No-Reference Trajectory Metric on Registered Point Clouds},
author={Anastasiia Kornilova and Gonzalo Ferrer},
year={2021},
eprint={2106.11351},
archivePrefix={arXiv},
primaryClass={cs.RO}
}
