Publication: Loop interface

Published today is a defensive disclosure of an invention of mine, which I call the “Loop Interface.” Such an interface may be particularly useful to people who are blind or who have low vision and need speech output in order to use technology. The abstract from the disclosure:

Disclosed is a user interface method that allows for quick, efficient exploration and usage of user interfaces by people who cannot see. The interface method is a circular direct-selection interface where many user interface elements are arranged around the edges of a touch panel or touch screen. This makes a circular list of elements that users can touch to get speech output and then activate when they find the desired element. This arrangement can be explored easily but also allows users to efficiently and directly select elements on familiar interfaces.

The problem is that current methods of using screen readers are inadequate. People may use swiping gestures or keystrokes to navigate from element to element, which can be inefficient. Alternatively, on some touchscreen devices, a screen reader might allow a person to tap or drag a finger around the screen to explore the screen via text to speech. This exploration can be challenging because onscreen elements may be anywhere on the screen in any arrangement and might be missed when dragging a finger on the screen.

A visual depiction of the Loop Interfaces in use.
Using the Loop Interface by tracing the edges of the screen with a finger and listening to the text-to-speech feedback.

With the Loop Interface, a person can simply trace the edges of a screen with a finger and get to all elements on the screen. When they become familiar with a particular application, then they can directly touch the desired element without having to navigate through all the intervening options. More details about the Loop Interface are available in the disclosure [PDF]. In the future, I plan to do user testing on a basic version of the Loop Interface.

The defensive disclosure was published today in the IP.com prior art database (IP.com database entry, requires subscription for full content). The publication was reviewed and sponsored by the Linux Defenders program. It will soon be published in freely-available form on the Publications page of the Defensive Publications web site.

The pressure-sensitive screen on the Apple Watch

Apple Watch.
A rendering of the Apple Watch by Justin14 [CC-BY-SA-4.0], via Wikimedia Commons.
Joseph Flaherty at Wired published an article today suggesting that the Apple Watch’s pressure-sensitive touchscreen might be a bigger deal than the Apple Watch itself. While I’ll let the market in the future determine the ultimate success of the product, I thought that Flaherty raised an interesting point with regards to the touchscreen user interface. Flaherty states that being able to sense a press with the so called “flexible Retina display” would allow for context-specific menus—much like the powerful “right-click” interaction with many computer interfaces. This could allow for a potential decluttering of the touchscreen interface because the user has a way of calling up menus that does not involve tapping on menu buttons or toolbars.

This decluttering could be useful for many, but may also be problematic for some. Context menus hide functionality, which some users may now not be able to find. In a poorly designed interface, people may have to “hard-press” many types of onscreen objects and elements to see which ones have the right hidden menu. In a well-designed interface, users would know just by looking (or convention) which onscreen elements have associated context menus.

An Accessibility Use of Pressure-sensitive Touchscreens

Having a pressure-sensitive touchscreen also allows for a very interesting and useful method of access for people who are blind and who have low vision: pressing firmly to activate an onscreen element. Currently, iOS and Android have built-in screen reading software. One way of using these respective screen readers is in an “exploration mode” where a user touches elements on the screen to have them read through text to speech. Because the system cannot different between a touch that means “what is this?” and a touch that means “activate this,” a separate gesture is needed to activate the desired element one found. On VoiceOver in iOS for example, the person double taps anywhere on the screen to activate the item that was last touched. This second gesture can be somewhat awkward and involves extra motions.

With a pressure-sensing touchscreen, this “activation gesture” would be much more intuitive. Instead of having to double-tap the screen, press an Enter button, draw a circle on the screen, or any other complex gesture, the user could simply press harder. In the touchscreen exploration mode, a light or regular touch means “what am I touching?” and a firm touch would mean “activate this.” This would be very powerful and intuitive to people who cannot see touchscreens well enough to read them.

Publication: Virtual jog wheel

People who use screen readers frequently need to navigate between different elements on a screen. As the user navigates to an element, information about that element is provided in speech output. On touchscreen devices, screen reading software typically uses swiping/flicking gestures for navigation. For example, one swipes to the left to go to the next element and to the right to go to the previous element when using VoiceOver on iOS by Apple (see more details about iOS VoiceOver). The problem is that these gestures are relatively slow and fatiguing. Using buttons to control navigation may be easier, but is still relatively slow.

I recently invented and described a software technique for quickly and easily navigating a user interface on a touchscreen device. The user activates a virtual jog wheel mode and then makes arc or circular gestures on it. By just making a single circular gesture, the user can navigate through a number of elements to get to the one they want.

Virtual jog wheel in use.
The virtual jog wheel in use. The person makes a generally circular gesture on the virtual jog wheel to navigate through the elements on the screen.

More details are available in the virtual jog wheel disclosure [PDF]. This invention is free for anyone to use, but those using it will still need to ensure that the technology does not infringe any other patents.

The defensive disclosure was published today in the IP.com prior art database (IP.com database entry, requires subscription for full content). The publication was reviewed and sponsored by the Linux Defenders program through the Defensive Publications web site.