Seeing AI: Free App Narrating World Around You

Submitted by Charlotte Cushman on Jul 12, 2017
Collage of Seeing AI iOS app

Seeing AI is a free iOS app from Microsoft that is designed to provide audio descriptions of the world to people with visual impairments.  Users point their phone's camera and turn the visual world into an audible experience with descriptions of people, text, and objects, using the power of Artificial Intelligence (AI).

Using VoiceOver, the app enables users to recognize: 
  • Short Text - Speaks text as soon as it appears in front of the camera. 
  • Documents - Provides audio guidance to capture a printed page, and recognizes the text, along with its original formatting. 
  • Products - Scans barcodes, using audio beeps to guide you; hear the name, and package information when available. (works with iPhone 6 and later)
  • People - Saves people’s faces so you can recognize them, and get an estimate of their age, gender, and emotions. 
  • Scenes (early preview) - Hear an overall description of the scene captured. 
  • Images in other apps - Just tap “Share” and “Recognize with Seeing AI” to describe images from Mail, Photos, Twitter, and more. 

 

Watch this introductory overview of Seeing AI:

 

Other video tutorials include:

  • Short Text Demo:  Learn how to instantly hear short snippets of text
  • Document Demo:  A guide on how to make the most of the Document channel
  • Product Recognition Demo:  Listen to audio cues to locate barcodes and identify products
  • Person Demo:  Learn how to use Seeing AI to engage with people around you
  • Scene Demo:  Learn more about this experimental channel that describes what’s in front of you
  • Currency Demo:  A guide for using the app to read currency bills (coming soon)
  • Recognize Images In Other Apps:  Describe images in your photo gallery and other apps including Mail, Twitter, WhatsApp and more

 

Download from iTunes.

Learn more from Microsoft.

 

Seeing AI app collage

  • Facebook
  • Forward
  • Twitter

Comments

Submitted by Paul Yang on Jul 15, 2017

We just tried the amezing app today. Every channel works so great. Thank you so much for developing this app for blind people. The only thing we experience difficult so far is to switch channels when the voiceover is on. Should we use any finger combination to do the switch or need to turn off voiceover to switch?
Best Regards,
Paul

Submitted by Charlotte Cushman on Jul 16, 2017

Hi Paul,

The expected gesture would be to flick right and left; however when listening to the VoiceOver hint, it says to flick up or down to switch channels. 

Good luck!

 

Submitted by Ruth Weber on Aug 28, 2017

Thanks for the tutorials you’ve already done.

Problem: I’ve been blind for 40 years and am just now learning the iPad and how to use voiceover. I need more help on which voiceover techniques to use when engaged with seeing eye.

For example: How do I get to each of the five different programs? When I first open the app, how do I get to DOCUMENTS or to SCENES? How do I move back and forth among them?

Is there a menu, and how do I access it?

In other words, I need more help with the basics. If you know other tutorials that I can access through my computer, please tell me about them.

Thanks!

Submitted by Diane Brauner on Aug 28, 2017

On the Home screen of the Seeing AI, you can right swipe multiple times to find the 5 categories or simply drag your finger around the bottom of the screen.

Once you are the first category, which is Short Text, if you listen to the VoiceOver hint, it tells you, “Short Text, adjustable. Swipe up or down with one finger to adjust the value.” This up or down swipe surprised me too! I’m not sure why Seeing AI did not follow the standard VoiceOver protocol (swiping right or left) to navigate between categories.

The Menu button is located in the top left corner. When you open the Home Screen, VoiceOver usually starts on the Menu button. When on the Menu button, double tap will activate it.

Here is a Paths to Technology post that includes the Seeing AI video tutorials for each category. These tutorials will talk you through each category option.

VoiceOver, like any screen reader, is a bit overwhelming to get started with on your own. If possible, I suggest connecting with a strong user who can help you. If you like to figure things out on your own, Paths to Technology has lots of information about using the iPad with VoiceOver, including an easy curriculum. Here are two posts to get you started!

This was last updated for iOS 9.2, but is still very appropriate. Chapter 3 is all about learning VoiceOver with explanations and step-by-step activities.

Have fun!

Diane