Web Posted on: August 24, 1998
Technological Solutions to Autonomous Robot Control
Axel Gräser
Institute of Automation, University Bremen, Kufsteiner Straße NW1, D 28359 Bremen
Tel.:++49-421-218-7326, Fax.: ++49-421-218-4596
E-Mail:AG@IAT.UNI-BREMEN.DE
http://siemens.physik.uni-bremen.de
1. Summary
The paper discusses the possibilities of rehabilitation robots for elderly and disabled people under the important aspect of (semi) autonomous robot control, which plays a key role in ac-ceptance and economic profitability. Due to the financial restrictions in this area, the user in-terface, the wheelchair, the robotarm, the necessary sensors and the control units are quite different compared with industrial robots. The whole control structure has to be rede-signed under the needs of the target group and the financial restrictions. In the paper typical scenarios for the application of these ro-bots and new scientific results are presented, esp. for:
- The control of redundant robots ( a wheelchair with a robot arm has at least 9 DOF )
- The robot arm control based on computer vision and image algebra to identify typical ob-jects and visual servoing, to move the gripper to the object
- The structure of a control unit and a possible new programming method, programming by demonstration.
The paper will also identify key problem areas and necessary developments to enlarge the usability to a wider circle of users.
| Top |
2. Introduction
To design Service Robots for elderly and disabled people is a very demanding task. There are fundamental differences in the necessities, compared with industrial robot applications, as well as to several other service robot applications. The tasks vary in wide areas, depending on the actual situation of the user and they cannot be pre programmed as in other robotic appli-cations. While all service robots must be designed for operation in clustered, a priory un-known and variable environments, new demands arise for the support of the target group. The interaction with the user and the variety of the supporting tasks may be mentioned here. On the other hand no total autonomy is demanded. It is rather important to use the cognitive abilities of the users and support compensate they only in areas where physical or cognitive impairments exist. Another limitation is the number of units which may be sold. Closely re-lated to the small number compared with other industrial products is the price of the today available components. The design of a robotic aid must consider these limitation but also pos-sibilities which arise due to the specific users. For the near future, the primary target group are disabled people with serious physically handicaps but only small mental disabilities if at all. Opposite to other service robot applications the cognitive abilities can and must be used to support and simplify the controller design, esp. for environment identification and scene in-terpretation. The remaining automation of the tasks is still very demanding, but the sensor data interpretation is eased if the user support the automation system.
The paper will discuss the design philosophy which results from this approach. An electric wheelchair with an added robotic arm serves as an example. The main target group of this unit are paraplegics ore people with similar handicaps.
| Top |
3. Scenario and Overall Control Structure
Figure 1 shows the typical scenario and a principal control structure for a wheelchair with a coupled robot arm with 6 DOF (Degrees of Freedom).The mobile basis has 3 DOF and the robot arm (MANUS e.g. ) has another 6 DOF. Together with a lift unit the overall system has 10 DOF. This type of system is called a redundant robot, which means that a infinite number of solutions for the inverse kinematic problem may be calculated. Inverse kinematic is used to calculate the angles or positions of the robot joints
Typical scenarios for a paraplegic user with serious restrictions of the upper extremities are e.g.:
- Drink - Grasp a bottle, pouring a drink in a glass, grasp the glass and transfer it to the mouth of the user, move the glass in a way, that the user can drink.
- Prepare a ready to serve meal - take the meal from the refrigerator, move it to the mi-crowave oven, operate the microwave, take the meal to a table, operate the packaging etc.
- Eat - Grasp the spoon, fork etc, use the cutlery to get the food, move it to a position so that the user can eat the meal, repeat the same motion several times with minor changes in locations of the food.
Figure 1: Scenario and principal control structure for a wheelchair with coupled robot arm
If we look to the structure of the automation system to fulfil the tasks mentioned above some critical areas can be identified:
- Specification of the tasks on a very high level and the automatic division of the tasks in a sequence of subtasks
- Identification of the necessary objects, their location and the assigned operational instructions
- Movement of the gripper with respect to obstacles, the number of joints ( DOF ) of the whole system and with respect to the environment
- Grasping and handle objects with respect to the shape of the objects and the form of the gripper
In the following chapters new results for various problems mentioned above will be discussed and the present status and the pros and cons considered.
| Top |
4. Sensor Based Control
Due to the different handicaps of the target group a wide variation of user interfaces is neces-sary. The EU has supported within TIDE the development of several high level user interfaces for elderly and disabled people. The advantages and disadvantages of the different concepts are well known and for the problems considered here the availability of user interfaces for every handicap can be assumed. But the input of a high level command is only the first but necessary step. The division of a high level command into a sequence of low level commands can be done in several ways, the simplest is the assignment of a organised list to each high level command and the possibility for the user to reorganise the list or even complete it in some circumstances. Nevertheless the low level commands need always sensor information, because due to the variable environment and the mobile basis the specific characteristics of a task are never the same. For the mobile basis several different sensor systems have been con-sidered in the past. Most of all in connection with the development of semiautonomous vehi-cles, semiautonomous wheelchairs for example. In re-lation to ( semi ) autonomous robot arm control only little results are published. A detailed look on the re-sults for mobile systems and a comparison with the demands from robot arm control show that visual in-formation will play the key role, because only the visual system will be able to handle the complex informa-tion of unknown environments. On the other hand, scene interpretation is still an unsolved scientific problem. To overcome these difficulties the user with his cognitive abilities must support the system and specify within a captured picture the objects to be han-dled. Based on this information the system can analyse the picture and generate additional information about the objects. To support the system, markers may be used to characterise the objects. Barcodes are one possible example, but in near future chip card processors placed on the objects may become the main information carrier. Due to the fast developing technology these chip card processors may also contain - locally stored, this mean at the objects- infor-mation about possible handling strategies, as well information about size etc. The advantage of such a distributed information structure is the simplification of the software system which generate the necessary information to control the wheelchair and robot arm.
| Top |
5. Vision Based Motion Control
As mentioned above, vision based control is for this application the top choice. In relation with the robot arm control, depth information is important. Vision based depth measurement is possible with two cameras (stereo system), one camera with variable zoom ore focus or with new developed systems which generate grey scale pictures and depth information in par-allel. Based on prize, size, reliability and the existence of powerful solutions, 3D calcu-lation from a stereo camera system seems the best choice. On the other hand the calibration of the camera systems with all the internal and external parameters (11 are mentioned in some publications) constitute a serious challenge. In industrial applications much effort is spent for calibration. But for mobile service robots like wheelchairs with robot arms even frequent cali-bration is no solution at all, because of the possible disturbances on the camera mountings due to the mobile basis. Thes disturbances which will change the relative positions of the cameras with respect to the robot arm base. Calibration robust, visual control, a method published by Hager [HCM95] represent a basis for a possible solution. The basic idea is the position control based on the image information in both cameras. If the reference point at the gripper is at a specific target point in both images, the gripper is at the target position in 3D space. The target point normally is a significant point at an object, easily detectable by the image software.
Figure 2: Stereo camera images with imaginary points for approach and grasping and obstacle between gripper and object
This method has been extended [T_G96,T98, L97, L98] to include obstacle avoidance and the movement of imaginary reference points. The extension is necessary to move the gripper in locations which are not on the target but close by, to carry out a handling strategy determined by the geometry of the gripper and the object. With the new method the whole handling process - approach to the object including obstacle avoidance, calculation of the gripping points, approach to the grip-ping points and finally grasping the object - is performed in image plane only. The whole process becomes robust against calibration errors. Only a very draft measurement of the basic co-ordinates is necessary. The maintenance of the whole control system is determined by this calibration independence.
| Top |
6. Control of Robots with more than 6 DOF
In service robot applications obstacle avoidance, security of the user and autonomy plays the main role. The speed of the movement is of secondary importance. The calculation of a path for the gripper leads to the specification of the cartesian co-ordinates for the gripper, 6 values for location and orientation. Based on these target co-ordinates, the joint co-ordinates of the roboter has to be calculated. This is also known as inverse kinematics. For regular robots with 6 DOF, the inverse kinematics can be solved analytically. The advantage of an analytical so-lution is, that the on-line computing time is minimal, all configurations are calculated, the results are of high accuracy and the solution may be calculated without difficulties even near singularities. In the case of more than 6 DOF no unique solution can be calculated. Additional conditions must be formulated to calculate unequivocal solutions. A new method which uses r=s-6 imaginary links to reduce the system to 6 DOF is presented in [I_G97, I_G98]. The advantage of this new method is, that it leads to analytical solutions for the inverse kinematic problem , but with insight in the structure of the solution and the influence of the imaginary links onto the solution.
Figure 3: Principle design of a robot arm with mobile basis.
Figure 4: Wheelchair with MANUS robot arm as an real example for a mobile ba-sis with robot arm
In service robot design the method has the other big advantage, that the computational power which must be available on-line will be reduced drastically. This is again important because of the price restrictions and the limited power available at wheelchairs.
| Top |
7. Programming by Demonstration
Service robots are quite new concepts for the support of disabled and elderly people. The ac-ceptance will depend on several conditions. Important factors will be the cost of maintenance and the costs which arise for the daily adaptation to the environment. If each change in the programming requires special trained people with detailed knowledge of robot programming, this may rather deter possible users. Programming by demonstration is a new method to re-duce the necessary knowledge and skills of the programmer. Equipment like a glove and pose sensor measure the location and the movement of the arm and hand of a user, who execute a specific task. The basic idea is that a computer program analyses the actions of a demonstra-tion and reproduce the action by assigning known basic actions to the demonstration [K98, R97].
Figure 5: Principal structure of Programming by demonstration, shown here in an industrial robot envi-ronment
Some results are published, but for the type of service robots considered here still a lot of re-search is necessary. Analysing the demonstration lead to two main difficulties. The intention of the demonstrator during the demonstration must also be recognised. Besides, the demon-strator itself is together with the used objects part of a closed loop control system. This may best be explained by a typical scenario: pouring a drink from a bottle in a glass. The liquid level in the glass is measured by the optical and cognitive system of the person who demon-strates the task. The typical movements as well as the duration depends from parameters like the shape and liquid level in the bottle and the glass.
A one by one transfer of the movements measured at demonstration to slightly different situation is impossible. To reduce the depend-ency from these parameters also closed loop operation is necessary. The automatic generation of the closed loop and the regulator from demonstration only is still an open problem.
| Top |
8. Outlook
The main components for a service robot to support paraplegics are available from industry and as scientific results, as shown above. But the presently used degree of automation is to small to get the right support for the people who really need such units. This is esp. true for the robot arm control. So, the components have to be completed by high level control units. The research results for automation are available to be integrated and to form a complete service system. But a challenging problem arises from the severe price limitations for support units for disabled people. The complete automation unit as described above may presently not sold in large quantities. Small quantities on the other hand prevent price reductions. The fu-ture design must take these limitations into consideration. A possible marketing strategy could be the implementation of small automation packages in an open software structure. The goal of the separate packages is the extension of the capabilities step by step. While the specifica-tion of these small software units is easy manageable, the specification of the overall software structure will lead to serious problems. The manufacturer in this market segment are mostly small companies, which fight for there marketing share and revenue. The software design and specification esp. the definition of the interfaces should be done by an independent organisa-tion but with the support of these companies. The design must follow a unit construction prin-ciple, where each new part extends the possibilities for the user. The support of these stan-dardisation process by TIDE is recommended here, to be able to reach a better market share step by step.
| Top |
9. Literature
[HCM95 ] Hager, G.D. (Dept. of Comput. Sci., Yale Univ., New Haven, CT, USA); Wen-Chung Chang; Morse, A.S.: Robot Hand-Eye Coordination Based on Stereo Vision, IEEE Control Systems Magazine (Feb. 1995) vol.15, no.1, p.30-9
[T_G97] Trittin, Gräser: Visual feedback in automation. Eufit 96, Aachen,1996
[T98] Tritin: Bildbasierte Roboterregelung für Serviceaufgaben, Presentation, GI- Fachausschuß 4.3, 3.3.1998 Bremen
[R97] Ruchel Nils: Programmieren durch Vormachen, 20. Automatisierungstechnisches Kolloquium Salzhausen, 1997
[L97] Lang, Oliver: Tiefengewinnung mit Zoomkameras, 20. Automatisierungstechnisches Kolloquium Salzhausen, 1997
[L98] Lang, Oliver: Visuelle Roboterregelung mittels Zoomkameras, Presentation, GI- Fachausschuß 4.3, 3.3.1998 Bremen
[K98] Kossow, Matthias: Greifen beliebiger Objekte auf Basis von 3D Sensordaten, Presentation, GI- Fachausschuß 4.3, 3.3.1998 Bremen
[I_G97] Ivlev O., Gräser A. An Analytical Method for the Inverse Kinematics of Redundant Robots. Proc. of the Third ECPD International Conference on advanced Robots, Intelligent Automation and Active Systems. Bremen, September 15-17, 1997, S. 416-421.
[I_G98] Ivlev O., Gräser A. Resolving Redundancy of Series Kinematic Chains through Imaginary Links. CESA'98 IMACS Multiconference: Computational Engineering in Systems Applications. Nabeul-Hammamet, Tunisia, April 1-4, 1998, (submitted and accepted for publication)
| Top | | TIDE 98 Papers |