Neural Rendering: NeROIC creates 3D models from photos

Researchers at the University of Southern California have unveiled a method for 3D printing objects from online image databases. The method, called NeROIC (Neural Rendering of Objects from Online Image Collections), makes it possible to create 3D models of objects and then print them out in a 3D printer – based only on a limited number of images of these objects from the Internet.

According to the researchers, the quality of the generated 3D models is comparable to models captured using sophisticated multi-view camera systems in controlled studio environments. NeROIC is capable of reconstructing objects of high geometric and material quality from a small set of images, even when those images were taken under a wide variety of conditions, they said.

For NeROIC, the scientists use a multi-step process based on neural radiance fields. In a first step, the geometry of the object is reconstructed and the camera parameters are refined. Then, the material properties of the object’s surface and the illumination conditions of each image are estimated.

According to the developers, this modular approach enables the reliable extraction of the 3D geometry as well as the material and lighting information of the imaged object. The data thus obtained can then be used to recreate the object with different textures and illuminations in 3D software. The model can then be printed in a 3D printer.

Initial experiments show that NeROIC is capable of reconstructing a variety of different objects, such as toys, furniture or vehicles, from a few dozen Internet images. According to the researchers, the method also works with objects that would be difficult to scan, such as the bust of Nefertiti in Egypt. The scientists plan to further refine their technology and hope that NeROIC will help digitize and preserve culturally valuable artifacts in the future.

The full paper, called “NeROIC: Neural Rendering of Objects from Online Image Collections,” can be viewed here.