Speech : Screen Reader and Machine Understandability

Jun Ishikawa
Japan(University of Shizuoka )

1.Why a sociologist works as a programmer

I have two professions. I'm a sociologist and I'm an assistive technology developer.
I've been professional sociologist for two decades.I've been working with assistive technology field more than a decade.
Ever since I have lived the double life of sociologist by day and programmer by night.

2. Desire of Making Tools

I started studying programming language late twenties. I have to say I was a slow starter. But I had a strong dream.
Making a cool Screen Reader that would make it possible for the blind to access mainstream software.
The screen reader is the most important and essential software to the people who cannot see the screen and are to operate the computer.
Humans have always made tools in order to overcome their bodily limitations in their history.
Blind people were one of those and so did I. The main reason of making tools for me is the experience of the taste of joy in being able to do something today that was not possible yesterday.
But it wasn't just that. Actually, the process of making something itself was interesting. I like making.
To create software, you have to write a program using an editor. Once the program is written, you put it through a compiler, which is software to make software.
This process is called compiling. If there are any grammatical errors in the program, the compiler interrupts the program and shows the error on the screen.
The programmer looks at this and modifies the program by correcting the error.
I needed a screen reader to make a screen reader and an editor to make an editor.
In other words, at the beginning, I was in the situation of needing the software that I want to make in order to be able to make it.
The development of the screen reader needs high-level technical skills.
By the time I was finally satisfied with the level of my Japanese MS-DOS screen reader (called Grassroots) the MS-DOS era was going to close. It took too long.
I did try making a screen reader for Windows.
In cooperation with the Dutch company ALVA, I initiated a project to transplant their Windows screen reader, outSpoken, into a Japanese environment.
That was year 1999.
Anyhow, I've developed many software for the blind.
They include screen readers, Self voicing internet browsers, mail software, editors and now developing GPS navigation software.

3. Fighting with JAWS

It was this February that I received an E-mail from Freedom Scientific in U.S.A.
They asked me to localize their product, JAWS, to Japanese.
At that time, IBM Japan had been dealing with that localization, however, Freedom Scientific wanted to break the relationship and get remarried to me.
I felt this work involved a high degree of risk.
Moreover, I would have to carry heavy responsibility.
However, my policy has been "We develop the tools we need".
So I soon accepted their proposal.
My team started localizing JAWS to Japanese from this April.
The team consisted of 5 people and 4 people of them were completely blind engineers and power users.
This team might seem to be week, but it was best actually.
I thought there was no one else except my team and IBM who could localize huge screen reader, JAWS, to Japanese.
JAWS was not only monster but also black box.
We had no information indicating its entire structure.
Therefore, we understood how JAWS behaves little by little as if we solved mystery novel scenarios. By making hypotheses and testing repeatedly, we gradually got to understand the essence of JAWS.
The screen reader had to be made adaptive to each application software one by one using every beautiful or nasty, cool or tricky strategies, such as screen hook, object model, MSAA, java access bridge.
Therefore, the structure of JAWS was extremely complicated.
The expression of "spaghetti" was not enough to show its characteristic.
However, we released Japanese version of JAWS only in 4 months.
Powerful screen reader like JAWS is fundamental tool for the visually impaired persons for job access and for everyday life.
JAWS is an abbreviation of Job Access With Speech.
Becoming skillful user of JAWS and office application remarkably increases work opportunities of blind persons.
Recently, we have customized JAWS to adapt to some office application used at hospitals and libraries.
Using the buildin scripting function of JAWS makes it possible to read the application that cannot be read by standard JAWS.
In fact, Japan's job training and support system for pwd is still not sufficient.
Many employees with disability are provided meaningless jobs or no job.
They go to their office, stay there during work time and return home. To empower such people to be capable workers, we have been providing JAWS training program and professional JAWS instructors to companies.

4 GUI, AUI and BUI

Computers display information visually on screen, and people select using a pointing device.
Interaction between people and computers today is based much on this repetition.
This interaction is made possible using an interface called a GUI.
However, the visually impaired need the information supplied with speech or Braille.
Yet, there is very little software that supports speech or Braille output.
Consequently, there is a need for software that converts GUI to AUI(auditory) and BUI.
This is today's GUI screen reader.
Efforts to improve GUI screen reader such as JAWS are very important, but very difficult.
Screen readers can not get enough information to show a perfect performance.
It is almost inevitable that screen readers are not stable, not perfect and difficult to manipulate. It is a gigantic patchwork.
Principally thinking, accessibility should be achieved ideally through the joint efforts of design for all and assistive technology.
If the user interface becomes more machine understandable and machine scriptable, screen reader performance will improve dramatically, and consequently, so will accessibility and usability.
Recently, there has been much interest in web accessibility.
Web page accessibility improves by making web pages machine understandable.
This is the same not only for web applications, but also for all applications.
In that sense, I have to have strong request to next generation operating systems from this perspective, namely machine understandability.