COLLEGE PARK, Md. — University of Maryland researchers have developed jaw-dropping technology that can construct a 3D image based on the reflections in a person’s eyes. They believe the reflective properties of the human eye provide a wealth of unexplored information about the surrounding environment.
In a paper currently undergoing peer review, the team details their innovative process. They capture images of a moving individual’s eyes from different angles, enabling them to reconstruct a 3D scene that extends beyond the camera’s line of sight. The researchers have managed to extract test samples, generating recognizable 3D images of toy aliens and a dog.
The task posed significant challenges, including the complexities of accurately estimating eye poses and untangling the visual interplay between the iris of the eye and scene reflections.
“Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer’s eye iris texture,” the team explains in a statement on their project website.
Given that the geometry of the cornea is nearly identical among all healthy adults, the researchers could compute the exact position of a person’s eyes by counting the pixel size of the cornea in the image.
“Using this insight, we train the radiance field on the eye reflections by shooting rays from the camera, and reflecting them off the approximated eye geometry,” the researchers continue. “To remove the iris from showing up in the reconstruction, we perform texture decomposition by simultaneously training a 2D texture map that learns the iris texture. However, approximating the eye pose just from the image is always very noisy. To address this issue, we perform eye pose optimization which is critical for performance.”
The team tested their technology by using music videos featuring Miley Cyrus and Lady Gaga, with an aim to reconstruct what these artists were observing.
“We manage to reconstruct the object that appears in Miley’s eyes, and we observe an object resembling an upper body from Lady Gaga’s eyes. However, due to the quality of these videos, the correctness of the reconstruction is unclear.”
The technology holds significant potential for various applications, one of which could be a futuristic crime investigation tool, reminiscent of the scene-enhancing photograph from the iconic film “Blade Runner.”
The researchers cannot currently comment on the study because the work is still under review in an academic conference. The findings of their study are published on the pre-print server arXiv.
South West News Service writer Dean Murray contributed to this report.
