UCI computer scientist Ian Harris demonstrates the digital prosthetic as it tracks his natural eye movements. At top, the blue-tinted prosthetic eye looks straight ahead, as all current prosthetics do. At bottom, the pupil of the prototype matches the movement of Harris' brown eye.
Viewed straight on, today’s prosthetic eyes look very realistic. But when the healthy eye moves naturally, the prosthetic eye has no or limited movement, creating a misalignment that many wearers find disfiguring.
Dr. Jeremiah P. Tao, chief of oculofacial plastic and orbital surgery at the UCI Health Gavin Herbert Eye Institute, has helped many patients who have a lost an eye to trauma, cancer or other disabling conditions.
Through a surgery called enucleation, he creates the appearance of an eye by attaching the remaining eye muscles to an orbital implant and later fitting the patient with a custom shell that serves as the artificial eye. The prosthesis, made from a plastic acrylic, is matched in size and color to the patient’s remaining natural eye.
“These devices look great,” Tao says. “But the failure of the eyes to move together has caused many of my patients to feel self-conscious.”
Tapping digital technology
Determined to find a solution, Tao sought out UC Irvine computer scientist Ian G. Harris, PhD, to help design a new type of prosthetic. They hypothesized that digital micro-screen technology — widely used in smart phones and smart watches — could be coupled with algorithms that track pupil movements.
In this way, they could achieve a more dynamic ocular prosthetic, one that moves in synchrony with the healthy eye. Moreover, organic light-emitting diode (OLED) technology allows for single-layer screens that are thin, flexible and lightweight enough to be trimmed and shaped on an ocular shell.
Tao, a professor of ophthalmology at the UCI School of Medicine, and Harris, a professor of computer science at the UCI Donald Bren School of Computer Science, began by capturing ocular rotations of a healthy eye using a small video camera.
Facial landmark detection and camera-based tracking software were used to create a coded algorithm to detect eye movement and pupil size. The data was hardwired to a laptop computer screen where an image of the moving contralateral eye was displayed with minimal delays undetectable by the human eye.
Tracking natural eye movement
The next prototype incorporated spectacles created with a 3D printer with a micro camera embedded in the frame to capture the movements of a healthy eye. Pupil location was wirelessly transmitted to a flat OLED micro screen about the size of a standard eye prostheses.
A crude image of the opposite eye was displayed on the screen and showed the appearance of the eyes working in tandem. The system was limited to replicating slow eye movements in a controlled environment.
Their latest prototype applies what is called a convolutional neural network (CNN) — a type of deep learning artificial neural network used in image recognition and processing — to track pupil location and capture natural fine darting eye movements under various conditions. These algorithms are merged with the micro-screen digital eye prototype.
Synchronizing with the natural eye
“We found that the neural network captured pupil location with high accuracy and allowed the prosthetic eye to move with the same amplitude and velocity as the natural, tracked eye,” says Tao.
Work still needs to be done to make the digital eye look more realistic, but Tao and Harris believe their real-time image processing in tandem with the neural network may lead to a viable next-generation eye prosthesis for patients. Their findings have been accepted for publication in the American Journal of Ophthalmology.
“We’ve made great strides with the software,” says Tao. “I think building all the components for clinical use is possible, but we still need to build a realistic, biocompatible micro-screen that a patient can wear."
About the Gavin Herbert Eye Institute
The UCI Health Gavin Herbert Eye Institute is home to internationally respected ophthalmologists who provide the highest quality eye care for conditions that range from mild myopia to rare retinal ocular disorders. Our team includes pioneers in the development and use of ophthalmic lasers and refractive surgery techniques. Their work formed the foundations of LASIK surgery, the development of the bladeless IntraLase™ laser and the most advanced techniques for corneal transplants and intraocular lens implants. Many of our experts are listed as Best Doctors in America®, a peer-reviewed survey that represents the top five percent of specialists in the nation.
Related stories
