Surface Reconstruction: A New, Improved Method for an Old Problem

Using neural networks, new method outperforms all existing methods for surface reconstruction

Surface reconstruction refers to methods of digitally representing three-dimensional objects. It has been a research area since the 1990s and has wide-ranging applications for computerized versions of the real world. While many surface reconstruction methods already exist, including some data-driven approaches, CDS researchers have developed a new method for this fundamental geometry processing problem that outperforms well-known existing methods.

CDS faculty members Joan Bruna, Assistant Professor of Computer Science and Data Science, and Claudio Silva, Professor of Data Science, Computer Science, and Engineering, with NYU researchers Francis Williams, Teseo Schneider, Denis Zorin, and Daniele Panozzo, built the new approach. The researchers overfit neural networks to point clouds to generate local charts — two-dimensional parametric representations — which are used to build transitions that ultimately reconstruct three-dimensional objects.

While most existing surface reconstruction methods use point clouds (simplified digital representations of three-dimensional objects from scanners or other measuring equipment) as input to generate digital reconstructions of the original, the new method uses the two-dimensional parametrizations as input to generate surface points of the reconstructed object as output.

This differentiates the researchers’ approach from others since most use point clouds as input, and, consequently, the new approach does not require training data. The researchers compare their method to AtlasNet, another surface reconstruction model, which does require training data and cannot identify pointwise local comparisons to resolve ambiguity. They found that their method outperformed AtlasNet both quantitatively and qualitatively.

They also compared results from their method for specific objects, including a figurine, a gargoyle, an action figure, and an anchor with results from twelve other methods for the same objects. They found that their method outperforms all other methods and as well as the state-of-the-art EAR method, but their new method is better at coping with noisy inputs.

While the authors are pleased with their results, they note that reconstructing an entire surface from local parametrizations is expensive compared to alternatives. In the future, along with reducing the expense, the researchers plan to focus on a theoretical analysis of how the surface geometry informs the neural networks’ architecture. They emphasize that while their method uses a simple approach for the problem of surface reconstruction, it relies on neural networks “overcoming the curse of dimensionality [which] remains a major mystery.”

By Paul Oliver