Web Posted on: January 19, 1999

Eye-Controlled Computing: The VisionKey Experience

Presenters: Jan Heynen, Gail Snuggs
Authors: D.A. Kahn, J. Heynen & G.L. Snuggs
H.K. EyeCan Ltd., Ottawa, Canada

Introduction For people with severe physical disabilities making the simplest communications a challenge, the use of the eyesas a pointer is well known. It is conceptually a simple step to automate reading the eyes and develop a switch with multiple selection options that could be used as a keyboard to operate a personal computer. This was the premise behind VisionKeyÔ which started clinical trials five years ago and began to be marketed in 1996.

As a computer is a general purpose machine, the ability to access it opens up endless possibilities not only for basic communication but also for creativity, learning, employment, control of the environment and entertainment. It enables a level of independence and quality of life that users have testified to making all the difference to the tolerability of their situations.

The technology Two basic types of eye-controlled communicators have emerged. One uses a standard video camera to view the eyes as they observe a keyboard image appearing on a computer screen. It requires a computer to process the video signals and display the keyboard. A second computer may be required to run standard software applications.

The other type employs a head-mounted miniature viewer that integrates an eyetracker with a keyboard display. VisionKey is of the latter type and uses a lightweight 2oz Viewer mounted on standard glasses frames in front of one eye, leaving the other eye a clear view of the surroundings. This approach requires a stable mounting and a fitting customized to the individual user. A separate Control Unit manages the Viewer display and converts the eye tracker signals into keyboard selection signals, text on an LCD display and (optionally) a voice output.

The advantages are that the user has freedom of orientation and position, the device can be used without a computer and has a high degree of portability for use at different locations or while traveling, and where a computer is being controlled, standard software applications can be run with no on-screen keyboard to subtract from the display area. There is no requirement for any special application software.

VisionKey was designed for use either as a stand-alone device for basic communications or as a substitute for a computer keyboard. In the stand-alone mode, it operates without a computer, using either the 20 character alphanumeric display or the optional built-in voice synthesizer. As a keyboard substitute, it operates with any type of PC or Macintosh computer, connecting through a serial port and using transparent adaptation software such as SerialKeys within Windows '95. A recent option allows VisionKey to connect through the keyboard port and work with any computer operating system with no need for adaptation software. This simplifies the system setup and allows users of other languages to use keycharts with special characters such as accents.

The user has a high degree of control of VisionKey features. He/she can choose any of nine selection speeds, start a recalibration, turn the BEEP on or off, turn the Viewer lighting on or off in the sleep mode and store these choices for automatic adoption at the next session. There is also a repeating BEEP alarm feature.

Character selection The simplest and therefore most desirable method of eye- controlled character selection is to simply gaze at the required character for a minimum dwell period. However, it is essential that the machine should not continue to select characters when the user is pausing to think or to consider information appearing on the computer display. This requires some easy means of suspending selections, a function sometimes called a clutch mechanism. The VisionKey selection method [US Patent 5,844,544] achieves this by making the entire central 5 x 5 portion of the keyboard image containing the alphabet insensitive to selection after a character has been selected. To make a new selection, the user must first gaze at the edge of the keychart.

The VisionKey ABC character chart shows 49 keys in a square 7 x 7 arrangement which subtends about 26 degrees to the eye. Each key can take any one of three cases, Upper (or shifted) and Lower as with a normal keyboard, and Command. This gives a total of 147 possible selections, a number which allows every computer keyboard key to be allocated. The Command case accesses all the computer keys such as the F1 to F12 keys, Insert, Home etc.

Figure 1 Basic ABC-chart Ó

The keychart characters are grouped in blocks of four keys occupying top left, top right, bottom left and bottom right positions. At any one time, the user can select from no more than 16 positions, each corresponding to a block. To select a character in the top right position in the block, the user must first gaze at the top right corner of the keychart and then at the required character. Similar remarks apply to characters located at other positions in the block.

During selection, when the eye is steady, the selected key becomes highlighted. This confirms to the user that the machine is following the eye. After the user has become familiar with the keychart layout, an operating speed of about 1 character per second is achievable.

Field experience With users and trials in over a dozen countries, a wealth of experience has been amassed with a variety of disabilities. As a result, various improvements and new features have been added both to the equipment and to the fitting, training and support processes.

These include the development of the basic ABC-chart, an intermediate chart for learning purposes that serves as a simple typewriter with capital letters, numbers, and a few punctuation marks. An improved nose bridge was developed for added comfort and easier setting of position. There is now an option for the caregiver to set the selection speed.

The largest user community is formed by those with ALS. These people are often already computer literate but are unable to continue using their hands or voice. As people with ALS often experience a high fatigue level, VisionKey should ideally be introduced in the early stages even before the loss of speech. This makes learning and training much easier and ensures that when the disease becomes more severe, an effective communications system is in place.

Other good candidate indicators are Muscular Dystrophy, athetoid Cerebral Palsy (CP), high level Quadriplegia (C1 and C2), Brainstem Stroke and Locked-in-Syndrome. CP may induce severe involuntary body movements which could disturb the Viewer and impede effective operation. Some with CP, those recovering from a stroke, and others may have impaired eye control. A special version of VisionKey was developed for these users. VisionKey 6 requires only coarse vertical eye movements and uses a sequential scan selection procedure. This version is slower but retains all the other features of the standard version.

Potential users often ask about a mouse capability. Fortunately, almost all software applications have keyboard alternatives to mouse operation. An exception is Netscape Navigator and Microsoft Explorer is therefore the preferred web browser.

The Windows 95 SerialKeys feature required for use with the serial port is defective and the work-around is difficult for caregivers inexperienced with computers. We therefore developed the option to connect through the keyboard port.

Careful attention from an optometrist or optician is required to ensure the correct frame and Viewer settings and fitting. At all stages, the company works closely with the user's family, caregivers and rehabilitation team. Before-and-after purchase assistance and support is provided by e-mail, fax and toll-free telephone at almost any hour. VisionKey is supplied with the EyeSpy training aid and a training video. The corporate support capability includes the adaptation of the equipment to special situations. One user was fitted with a custom mounting enabling her to retain a ventilation mask over her nose.

The portability is highly valued. The internal rechargeable battery gives over eight hours of use and VisionKey is well matched with compact notebook computers. It can also be powered from a 12V auto lighter socket and used with the voice synthesizer while travelling. Similarly valued is the freedom to position the user to optimize comfort, with no requirement to be sat upright.

Users often spend many hours every day with VisionKey which is used not only for communicating with caregivers, friends and family but also as a window to the wider world. One user has written a book of sea chanteys; a computer science student has created his own website; another user is preparing Power Point presentations for his work. Many use e-mail and surf the net, while others simply use the machine as a typewriter or to create conversational messages on a screen or on the voice synthesizer.

Conclusions

The groundwork for the success of this technology has been thorough and the results are demonstrated by the growing volume of satisfied users around the world. The company continues to develop ways to ease the path to success for new users. It is also developing with partners, value-added features and new products harnessing technology advances in response to changes in customer needs resulting from the rapid evolution of the computing environment.