Shin Shibata
President, Shinshu Associates

1. Development of a GPS navigation system six years ago

After working for a manufacturing company, I am now in semi-retirement in the countryside. I begin each day by walking around sunrise. However, pigmentary degeneration of my retinas, a condition which I have had for a long time, advanced and it became difficult for me to walk alone due to the deterioration of my sight. There is no treatment for this disease, and it progresses with age, so there are many older people with this disability.

Due to this, I began to develop a voice navigation device for myself using GPS, so that I would be able to walk by myself even if I became unable to see. I completed it in around two years, in collaboration with friends from my student days. I walk a four-kilometre course each morning using this navigation system and a cane.

However, even when I recommended it to others, they seldom used it. It apparently did not provide good enough performance or functionality for walking in the city, where there are many obstacles. I was able to gain a variety of opinions about the conditions necessary to enable widespread use of a walking support device from these experiences.

2. From GPS to tactile paving

As we are all aware, GPS involves a certain degree of error; moreover, it cannot be used inside. On the other hand, tactile paving for the blind has already been installed in public places. With this paving, there is no error in location.

However, with tactile paving alone, users do not know where they are or where they are going, and it is difficult to guide users along multiple paths. The idea sparked by this was encoded tactile paving. This works by putting markers on tactile paving which can be read by a smartphone, synthesizing appropriate sentences to guide the user by comparing this information with the data in the smartphone, and providing spoken guidance.

Based on my experiences described above, I proposed simpler encoded tactile paving, and gave this the name “Panda Navi”.

3. The basic principles of Panda Navi

Tactile paving comes in two types: tiles with lines on, and tiles with dots on, which are placed at key points. Tiles with dots on are also called “warning tiles”, and usually have 25 raised dots on them.

With Panda Navi, 8 of these 25 raised dots are selected and black rings fitted around them, with these arrangements creating optical distinctions between each different tile.

The way in which the rings are fitted shows the direction of the tile, and a total of 455 different patterns can be created. By filming these tiles with the smartphone camera, the app decides which of the 455 patterns is on this tile and how many degrees the tile is rotated from the standard direction using image processing logic.

The black rings stand out clearly, like a panda’s eyes, and can be placed accurately on points where the lines intersect, so they can be recognized definitively even in poor conditions.

4. Explanation of the operation of Panda Navi

The left-hand side of Photo 1 was taken facing the tile in the standard direction (north, for example). In this situation, the following tile data is registered in the database.

Pattern ID (assigned automatically) 146

Latitude and longitude (assigned automatically) 36.8750,139.3213

Name of current location (voice input)　Trifurcation of routes to the West Exit ticket gates

To the front (voice input) Towards lines 2 and 3

To the right (voice input) Automatic ticket gate

To the rear (voice input) Towards line 1

To the left (voice input)　－

If an image like the right-hand side of Photo 1 is displayed on the smartphone of the person walking, on the other hand, the app concludes that “A tile with pattern 146 is rotated 180 degrees clockwise = the smartphone is facing south”. Taking this into account, the app tells the user that “line 1” is in front of them, not “lines 2 and 3”. With this, the user knows where they are and which direction they are facing.

It should also be noted that by comparing the latitude and longitude of the position of the tile and those of the user’s device, the app can distinguish between tiles with the same pattern placed in distant locations.

Photo 1 Rings fitted onto tiles with dots. Left: facing north, right: facing south.

5. The potential of Panda Navi

Let us consider the potential of Panda Navi in the light of what was learned through the development of the GPS navigation system introduced above.

(1) Is it sufficiently useful for visually impaired people?

Panda Navi was exhibited at the Seeds and Needs Matching Event held by the Association for Technical Aids in Tokyo in December 2021. It attracted a strong degree of interest and anticipation from visitors with disabilities themselves, as well as from their supporters. An opportunity for around 20 visually impaired people to experience it was also held in Kofu City the same month (organized by Yamanashi Prefecture Association for the Welfare of the Visually Impaired). It received positive feedback there, including “It is very helpful for me to know my current location and direction”, and “I am happy to find a special purpose machine that can be used with a single touch”.

(2) How to operate

We suggest the following two ways of using it, according to the user.

 Partially sighted or sighted people who are used to smartphones can download the app and tile data to their own smartphones and use this. For other users, we offer a small special purpose machine that can be operated simply by pressing a button on the screen. In this case, it can be used as though you took a torch out of your pocket to walk along a road at night (Photo 2).

Photo 2 Special purpose navigation machine using a small smartphone (prototype)

(3) Installation cost

The existing tactile paving infrastructure can be used, reducing installation and maintenance costs.

(4) Useful for everyone

Even sighted people get lost in major stations and other transport hubs. It would be very helpful if they could read the floor tiles to find out their current location and direction. Moreover, by supplying the guidance sentences for the tile data in multiple languages and setting the speech output to match the operating system of the smartphone, it would be possible to guide foreigners in their native languages.

In the future, managing the data for all the tiles together on a cloud server and enabling it to be used throughout the world is surely not just a dream.

6. From the “technological perspective” to the “user perspective”

Various techniques are proposed in order to compensate for a lack of visual information. In recent years, different types of sensor technology aimed at self-driving cars, image recognition technology using AI, and so are making rapid progress. However, it is not true that “If we use advanced technology, we will solve mobility problems”.

I have put together a list of some of the points which can tend to be overlooked, based on my experience as an end user.

• If the user cannot see the device, it is difficult to set up or run
  Support devices which have become relatively widespread are single-function, palm-sized machines which can be operated with one touch of a button.
• The majority of visually impaired people do not use smartphones
  For people who lost their sight in middle or old age, using a smartphone is probably an enormous hurdle to clear.
• Mobility support devices present unique difficulties
  • The user holds a cane in their right hand, so cannot operate it with both hands.
  • It is hard for the user to continue holding the device in their left hand when walking.
  • If the batteries run out, the user cannot move any further.

In these ways, there are major differences between devices for installation in vehicles and portable devices for visually impaired people when walking. If using a device is a nuisance relative to its benefits, people will end up saying “I’ll ask the people around me for help, so I don’t need it”. When developing a support device, the best approach is to develop it together with the people who will use whenever possible, starting by listening carefully to what they need.

7. Towards social implementation of Panda Navi

 I am already moving ahead with a patent application. From now on, I hope to work together with interested parties to make Panda Navi widely used and improve the quality of life of people with disabilities.

◆ Panda Navi website
http://www.shinshu-a.com/pn/ (Japanese only)