One of the most revolutionary changes to come to iOS in some time is VoiceOver recognition, an AI powered method to provide information about controls which have not been properly labeled to VoiceOver. Read on to find out more about how to configure and use this new feature.
Please note that VoiceOver Recognition only works on the iPhone x models and up. Limited support may be provided on older devices and iPads, but the full experience requires hardware which is only available in the x and newer models. I know that I have had some success with improved OCR on the iPad, but the section does not exist in settings, nor does the ability to recognize the screen of inaccessible apps.
VoiceOver recognition encompasses several different features, each of which works slightly differently. Before using any of them, however, they need to be configured. These features are not on by default.
Go to Settings>Accessibility>VoiceOver. On supported devices running iOS 14 or later, you will find a new section in the VoiceOver settings called "VoiceOver recognition." In this section you will find a disclaimer which explains what VO Recognition does and advises that it should not be used in high-risk situations, for navigation, or for diagnosis or treatment of medical conditions. Following this, you will find toggles for the various sections of VO Recognition.
- Image Descriptions: Images within apps or on the web are described. Ex: you will sometimes get a description of the images of the food in promotional photos as you scroll through restaurants in delivery apps or have screenshots and promotional artwork described in app store pages.
- Screen recognition: iOS will attempt to recognize items on the screen and their purpose, even in inaccessible apps.
- Text Recognition: text found in images will be read aloud.
After this will be a place to choose between spoken, sound, or no feedback when these services are engaged.
adding screen recognition to the rotor
Compatible devices will have a new rotor option available called screen recognition. Add it to the rotor in the VoiceOver rotor section of the settings. To enable it, set the rotor to screen recognition and swipe up or down. When enabled, VoiceOver will use on-device intelligence to attempt to create an accessible rendering of the app, using context to generate interactable elements such as buttons and text, where there were unlabeled buttons or simply inaccessible elements before. This is very much still a work in progress, but early testing has demonstrated promising results with several inaccessible apps and games.
using these features
When you encounter an image, put VoiceOver focus on the image, either by swiping to it or finding it with explore by touch. Wait about 1 second, and any information VO Recognition was able to grab will be read out. Occasionally, it is necessary to re-focus the image a few times in order to receive a recognition result.
Each of these options alone would have been a wonderful improvement to Voiceover, but used in conjunction, they do a great deal to enhance the accessibility of iOS. Are they perfect? No, unfortunately. It can be difficult to get recognition to commence, and some games developed in unity which I expected to work with the new screen recognition feature still don't seem to, but there is a lot of promise in these features, and even in these early days they represent drastic improvements to iOS and VoiceOver
Category: NewsNo one has commented on this post.
You must be logged in to post comments.
Or Forgot username or password?
Register for free
Tangela Mahaffey is a barely reformed English Major currently residing in Colorado. She reads entirely too many fantasy books and takes almost nothing seriously, including herself. She loves technology, music, writing, puns, and cats. She can be reached on twitter @tmmahaff
