Seeing with Your Ears: Translating the Visual World to Audio

This podcast is about big ideas on how technology is making life better for people with vision loss.

Today’s big idea is seeing with your ears — exploring technology that uses audio to connect people with vision loss to the world around them. Dr. Cal Roberts speaks with Dr. Yonatan Wexler about the OrCam My Eye device. They explore the ideas of whether the brain can recreate a picture using audio alone, how well hearing can replace lost sight, and what are the discovery processes for using audio description technology as a substitute for vision.

The Big Takeaways:

• Developments in technology allow for the use of audio as a substitute for vision for people with no or low vision, sending signals to the brain to create a picture in their mind’s eye.
• The natural gesture of pointing signals the OrCam device to enable audio feedback to the wearer, the same way our selective vision helps us discriminate objects from our surroundings.
• The journey to the development of this device was intricate and involved, highlighting the constant evolution of this technology and future opportunities for development such as the ability for a user to save their experiences.

Tweetables:

“I have an idea of how the ball field is shaped, where the players are standing, and what they do. … It’s not a fully-fleshed out picture, but it … helps me know what everyone is doing at any … moment.” — Karen, a blind baseball fan describing hearing a game play-by-play

“A radio allows you to use your imagination. In some ways, radio is even better.”
— John Sterling

“It happens and you describe it and hopefully you describe it well. I hope it helps the listeners see the game.” — John Sterling, Voice of the New York Yankees

“The beauty of the brain is that it can generate sight using the information from the ears.” — Dr. Wexler

 

Contact Us:

Contact us at [email protected] with your innovative new technology ideas for people with vision loss.

Pertinent Links:

Lighthouse Guild

OrCam

Guest Bios:

John Sterling

John Sterling is an American sportscaster best known as the radio play-by-play announcer of Major League Baseball’s New York Yankees. Sterling called 5060 consecutive Yankees games, beginning in 1989, before taking an 8-day (4-game) break in July 2019.

Dr. Yonaton Wexler

Dr. Yonatan Wexler is an experienced researcher in the field of Artificial Intelligence and Computer Vision. He has conducted research at the Hebrew University, University of Maryland, Oxford University, Weizmann Institute of Science, and Microsoft. His passion is efficient use of visual information that enables exciting new abilities. He is the recipient of the Marr Prize, the highest in the field of Computer Vision. Currently, Yonatan leads the R&D at OrCam, a company that pioneers a generation of clever AI-based assistants that improve people’s daily lives. The OrCam MyEye revolutionizes the lives of the blind and the visually impaired. It clips onto any pair of glasses and verbalizes information they find hard to perceive. The OrCam Focus breaks new grounds for hearing aids. It solves the age-old “cocktail party problem” for the first time and lets the wearer focus on one speaker, discarding any other chatter and noise, hence solving the #1 problem with existing solutions.

Host Bio

Dr. Calvin W. Roberts

Calvin W. Roberts, M.D., is President and Chief Executive Officer of Lighthouse Guild, the leading organization dedicated to addressing and preventing vision loss. Dr. Roberts has a unique blend of academic, clinical, business, and hands-on product development experience. Dr. Roberts is a Clinical Professor of Ophthalmology at Weill Cornell Medical College. He was formerly Senior Vice President and Chief Medical Officer, Eye Care, at Bausch Health Companies where he coordinated global development and research efforts across their vision care, pharmaceutical and surgical business units. As a practicing ophthalmologist from 1982 to 2008, he performed more than 10,000 cataract surgeries as well as 5,000 refractive and other corneal surgeries. He is credited with developing surgical therapies, over-the-counter products for vision care, prescription ocular therapeutics, and innovative treatment regimens. He also holds patents on the wide-field specular microscope and has done extensive research on ophthalmic non-steroidals and postoperative cystoid macular edema. Dr. Roberts has co-founded a specialty pharmaceutical company and is a frequent industry lecturer and author. He currently serves as an Independent Director on multiple corporate boards and has served as a consultant to Allergan, Johnson & Johnson, and Novartis. A graduate of Princeton University and the College of Physicians and Surgeons of Columbia University, Dr. Roberts completed his internship and ophthalmology residency at New York-Presbyterian Hospital/Columbia University Medical Center in New York. He also completed cornea fellowships at Massachusetts Eye and Ear Infirmary and the Schepens Eye Research Institute in Boston.