NEW DIMENSIONS IN HAPTICS

Wunji Lau, Peter Hanson, and Dave Schleppenbach
Purdue University VISIONS Lab
1393 BRWN Box 88
West Lafayette, IN 47906
Phone: (317) 494-8718
FAX: (317) 494-0239
Email: ryoohki@expert.cc.purdue.edu (Wunji Lau)
Email: phanson@ecn.purdue.edu (Peter Hanson)
Email: engage@purdue.edu (Dave Schleppenbach)
WWW:
http://www.chem.purdue.edu/facilities/sightlab/index.html

Web Posted on: December 12, 1997

The Purdue University VISIONS (Visually Impaired Students' Initiative ON Science) Lab is dedicated to improving opportunities for the visually impaired to study advanced scientific, mathematical, and engineering (SEM) material at the college level. It is composed of three separate divisions: a production facility that produces educational materials for the visually impaired, a research group whose purpose is to design and develop new and better technologies for use by both the blind and the sighted, and a development group, to help disseminate new solutions into the adaptive community. The research lab is in the process of developing several different means of making pictures, diagrams and other graphical information available to the blind.

One of the primary techniques for the interpretation of scientific information is visualization. Ordinarily, a student uses his or her eyes to receive data and appropriate that data into a cognitive network of related information. This process requires skills such as pattern recognition, data filtering, the detection of critical elements of the data, and, most importantly, predictive generalization. For the visually impaired, and the blind in particular, SEM students must rely on haptic input (input related to the sense of touch) to perform many of these complex tasks. Interestingly, VISIONS Lab studies have shown that while some aspects of haptic input are inferior to those of visual input, others can afford the blind advantages over the sighted. For example, the filtering out of background noise (such as the XY-grid behind a graph) is a simple visual task. Sighted students learn to interpret noisy data at an early stage, and can do so even when the signal-to-noise ratio is quite low. On the other hand, data filtering in a haptic environment can be quite difficult and generally requires a high signal-to- noise ratio. However, Purdue's blind students generally learn stereochemistry (the three dimensional nature of molecular structure) more readily than the sighted, perhaps because the haptic environment of molecular models and tactile graphics is more informative than two-dimensional images drawn on a page.

While it is certain that haptic input is necessary for blind students to interpret complex graphical images, the methods for the creation of haptic environments are often complex and confusing. The focus of this paper is to describe the multiple techniques for creating haptic scientific information, and the advances that the VISIONS Lab has made into making new methods of tactile graphic generation. Basically, haptic interfaces can be created in two ways: "on-line" and "off-line".

The so-called "off-line" methods for the generation of haptic environments is generally referred to as tactile graphic creation. Tactile graphics are the most widely used form of haptic interface for the blind today, excluding braille, which deals only with textual information and not graphics. These "off- line" techniques include embossers, tactile image paper, and "lo- tech" solutions.

One "off-line" technique central to the VISIONS Lab's production operations is Braille printer embossing, which is used to output textbook materials for our students. The printers also have the ability to make "dot art," or pictures made up of closely-spaced Braille dots. For example, many interpoint embossers such as the Basic-D and Everest (sold by Sighted Electronics) can create single-sided tactile graphics with a mere 1.6 millimeters between dots. Although this is quite good for "dot art", there are limitations. These images are unfortunately quite bulky, and difficult to create with standard software. Also, the resolution is not high enough for many tasks such as the creation of tactile spectra, for example. A new kind of embosser, called the TIGER, is under development (contact John Gardner of Oregon State University for details). This embosser prints "dot art" as well as raised lines, which allows for the creation of even finer resolution tactile graphics. Still, this may not be enough for some tasks.

Another method for producing tactile graphics which is used extensively by the VISIONS Lab is to use tactile image paper, which is specially treated so that inked areas on the paper will raise when heated in a special machine. The best example of this technology is the Flexi-Paper (TM) and Tactile Image Enhancer (TM) by Repro-Tronics. This method allows more detail than the "dot art" method, but is equally bulky, and rather expensive. Nevertheless, this paper has proven to be the most cost- effective, complete, and quality-consistent method currently available to the VISIONS Lab for providing complex chemistry and biology diagrams to blind students.

Working closely with the end-users, the VISIONS Lab has extensively researched the limits of resolution and detail that the paper is capable of. The result is a system of standards for such things as line thickness and shape size that we have applied to our on-campus work, and which we hope to see integrated into a common standard for all Flexi-Paper graphics. The standards provide numerical values for things like normal, bold and double- bold line thicknesses, line spacing, braille-graphic spacing, and many others, as well as providing test information on the limits of the human tactile distinguishing ability.

A project related to the VISIONS Lab's use of tactile image paper is the refinement of image processing routines to reduce complex drawings or photos to a line drawing representing the most prominent edges. While this project is under development, the VISIONS Lab has engaged the services of a scientific illustrator for the purpose of simplifying and redrawing especially complex or large images so that they can be transferred to the tactile image paper. While this software based solution will not completely remove the need for human intervention in the development of tactile images, it significantly reduces the amount of work involved in converting primarily visual information into tactile format for scientific illustrators. For example, one blind student at Purdue has taken a biology course involving dissections and photomicrographs. For this, the VISIONS Lab scientific illustrator laboriously re-drew many of the photomicrographs while the VISIONS Lab model-building staff created 3-D, anatomically correct models. A software solution to this problem would be tremendously more cost-effective and much less time-intensive.

This image processing project is inspired by work done by other groups, such as the excellent work done at the University of Delaware's Applied Science and Engineering Lab (ASEL). Basically, the VISIONS Lab approach is to create a "plug-in" filter module, compatible with many major graphics packages such as Adobe Photoshop (TM) and COREL Photopaint (TM). This filter allows a sighted user to use the pre-built user interface of his or her favorite graphics package and makes use of the wide variety of input files available to most major graphics programs. In addition, this filter transforms the image into a raised line graphic suitable for enhancing via the Tactile Image Enhancer (TM).

A final option for the creation of tactile graphics is the "lo- tech" solution. This includes things such as the Sewell Raised line drawing kit and the Swail Dot Inverter (both available from the American Printing House for the Blind). Both of these inexpensive solutions allow the user to manually create raised lines on paper. Perhaps an even better solution is the Thermo-Pen (TM) of Repro-Tronics, which allows a blind user to create raised lines manually on Flexi-Paper by direct drawing. This tool is equally useful for both the blind and the sighted. Finally, several two-dimensional slate-and-stylus devices exist for the creation of both braille and graphics (contact Judy Dixon of the Library of Congress for more information).

While the combination of Braille-printer text and tactile image paper graphics has served the adaptive community well for many years, the need for a "paperless" approach to haptic interfacing is becoming more and more important. The advent of the information age combined with the complexity of modern science requires both sighted and blind students to interpret vast amounts of data in short periods of time. Thus, a complementary system of image generation that is both portable and highly mutable is a crucial need as the blind prepare for the 21st century. While the technologies mentioned earlier are regarded as "off-line" interfaces (i.e. not directly computer-automated in the final image form), this complementary project will involve an "on-line" interface, by which the images are generated, refreshed, and maintained by computer control.

One fascinating approach to this problem is that provided by SensAble Devices, Inc. SensAble has developed the PHANToM system, which creates a force-feedback haptic environment. This device fools the user into believing that he or she is "touching" virtual surfaces. The effect is amazingly realistic, and can mimic surfaces of different texture, springiness, or even stickiness. In addition, the PHANToM allows the visually impaired or sighted user to experience three-dimensional environments in an incredibly realistic way. Although quite expensive, the PHANToM will undoubtedly be a major tool for the visually impaired SEM student in the years to come.

A second "on-line" approach is the creation of an actual surface that the visually impaired user can feel. The goal of the VISIONS Lab project is the construction of a new type of refreshable dot display that has several advantages over the current crop of such devices. In this computerized age of mass data transfers and storage, it is rather unfeasible to expect a blind individual to carry around several hundred pounds of Braille documentation for any particular application. A refreshable Braille display linked to a computer is a far more compact and efficient method of storing and accessing an equivalent, or larger, amount of information. Unfortunately for many, the technology behind these displays is extremely delicate and expensive. A small display costs more than a top-of-the-line desktop computer; larger displays are more expensive than automobiles.

The reason for this expense is the way the pins are raised and lowered. Current Braille displays use piezoelectric crystals to raise and lower the dots. These crystals emit an electric charge upon physical deformation, and vice versa. These crystal actuators, while small and reliable, are both expensive to produce and quite fragile. All commonly marketed refreshable Braille displays use this technology.

The VISIONS Lab is developing a refreshable pin display based on an alternate technology that should prove cheaper and more durable. The relative cost-effectiveness of our technique enables us to make available refreshable arrays of thousands of dots, far more than the current financially feasible maximum of about five hundred. This new type of display will thus be able to display not only standard Braille characters, but larger drawings and digitized images, as well. One exciting potential use of the device is to allow blind computer users to use graphical user interfaces like Windows (R) by "feeling" a reproduction of the computer screen on a large pin array. A little skillful programming will even enable the device to show animated images.

The small size of the individual elements in our display enables us to make the spacing between dots quite close, although for the prototype the dots are not as close as required by standard Braille. This, of course, is the same difficulty that braille embossers have in producing simultaneous high-resolution graphics and braille. Nevertheless, the Braille generated by the device is quite readable. We plan to produce multiple types of pin displays; some will be spaced according to the requirements of Braille (i.e. in rows of eight-dot cells) while others will be spaced closely and evenly to allow for refreshable generation and viewing of tactile images.

The device is attached to an electronics board that is linked, in turn, connected to a computer via a serial port. We are in the process of writing software to control the device for a variety of applications. Braille conversion software is planned for common applications such as WordPerfect (R) and the various Microsoft (R) products, as well as image conversion and creation interfaces for graphics display. Future developments in this area by the VISIONS Lab include research and development of a three- dimensional refreshable display that will be able to create tactile shapes with height as well as length and width.

In sum, the future of haptic interfaces is very promising. Many cost-effective techniques already exist for the production of high-quality tactile graphics. As visually impaired students become more used to this educational medium, newer, computer- controlled devices will be introduced to allow for even better, faster, and cheaper tactile graphics. Eventually, even fully three-dimensional haptic environments may be only a few mouse-clicks away from reality.