Smartphone camera replicates physical objects for virtual reality
A global team of computer scientists have developed a novel method that enables physical objects to be replicated for the virtual and augmented reality space by using a point-and-shoot camera with a flash.
Previous research showed that capturing and reproducing real-world objects for any virtual environment is complex and time-consuming. However, using a conventional camera with a built-in-flash from any mobile device or off-the-shelf digital camera can simplify this task, according to researchers from South Korea and Spain.
“To faithfully reproduce a real-world object in the VR/AR environment, we need to replicate the 3D geometry and appearance of the object,” says Min H. Kim, associate professor of computer science at KAIST in South Korea and lead author of the research.
“Traditionally, this has been either done manually by 3D artists, which is a labour-intensive task, or by using specialised, expensive hardware. Our method is straightforward, cheaper and efficient and reproduces realistic 3D objects by just taking photos from a single camera with a built-in flash,” he added.
Existing approaches for the acquisition of physical objects require specialised hardware setups to achieve geometry and appearance modelling of the desired objects, with these setups likely to include a 3D laser scanner or multiple cameras, or alternatively a lighting dome with more than a hundred light sources. In contrast, this new technique only needs a single camera, to produce high-quality output.
“Many traditional methods using a single camera can capture only the 3D geometry of objects, but not the complex reflectance of real-world objects, given by the SVBRDF,” notes Kim, referring to spatially varying bidirectional reflectance distribution, which is used to obtain an object’s real-world shape and appearance.
He added: “Using only 3D geometry cannot reproduce the realistic appearance of the object in the AR/VR environment. Our technique can capture high-quality 3D geometry, as well as its material appearance, so that the objects can be realistically rendered in any virtual environment.”
The group had demonstrated their method using a Nikon D7000 digital camera and the built-in camera of an Android mobile phone, which resulted in the novel algorithm which did not require any input geometry of the target object. It successfully captured the geometry and appearance of 3D objects with basic, flash photography and showed consistent results.
Examples that were shown in the experiments included a diverse set of objects that spanned a wide range of geometries and materials, which included metal, wood and plastic. They also included complex shapes like a finely detailed mini-statute of the Egyptian queen Nefertiti.
Kim and his collaborators, Diego Gutierrez, professor of computer science at Universidad de Zaragoza in Spain, and KAIST PhD students Giljoo Nam and Joo Ho Lee, aim to present the new study at SIGGRAPH Asia 2018 in Tokyo next week. The annual conference in Japan’s capital features the most respected technical and creative members in the field of computer graphics and interactive techniques, and showcases leading edge research in science, art, gaming and animation, among other sectors.
The research was published in the paper ‘Practical SVBRDF Acquisition of 3D Objects with Unstructured Flash Photograph’.
Earlier this November, the UK government awarded £1m to a University of Glasgow project which aims to develop immersive virtual reality (VR) classrooms through a public body which distributes funding to translate research into useful products called Innovate UK.
In August, Business Insider, TF Securities analyst Ming-Chi Kuo said he believes Apple is preparing a new augmented reality wearable product to partner with the iPhone, potentially called AR Glass.
