Seeing AI logo
In March 2016, Microsoft showed off a prototype of its Seeing AI app, which looked very promising at the time but required further development. In mid July 2017 the company released an enhanced and updated version, as a free app on iOS.
The Seeing AI iOS app uses computer vision to describe the world for the visually impaired by using an iPhone's camera to tell the user about objects in the field of view. It identifies things in the immediate environment of the user including people, objects, and even emotions — to provide important context for what's going on around you. Users simply point their smartphone's camera, select a channel, and hear a description.
There are other uses for the Seeing AI app too, such as working in groups. It can tell you whether or not people are really listening to what you're saying, and it can describe the general age and gender of the people around you, as well as their emotions via advanced facial-recognition software.
The app recognises ‘saved friends’ and speaks out their name and distance from the camera when it is pointed at them. It also reads text out loud as printed text comes into view — for example, the name and address of a letter envelope, information on a sales receipt, or the details on a credit card.
When paying by cash, the app is able to identify the value of currency, so you don't end up getting out the wrong value notes to pay. When looking for something in your kitchen pantry or in a store the user can use the barcode scanner with audio cues to help find the product you want, along with being able to hear additional product details from the outside of the can or packet, like preparation instructions or cooking details.
There are 'experimental features' that can be explored, including 'handwriting' audio reading and 'Scene' descriptions, which can be accessed now. Microsoft said in a statement that further refinements are in the pipeline and the app will soon be available on Android as well.
As of mid July 2017, the app is only available in the US, Canada, India, Singapore, Hong Kong, and New Zealand — with Australia among a second wave of countries to have access to the app towards the end of 2017. For more information go to the Seeing AI website and watch a Seeing AI video.
Top of page