Control of Neuralprostheses I: Sensor Fusion

Brian J. Andrews, Richard Williamson, Narce Oulette, Andrew Koles Department of Biomedical Engineering, University of Alberta, Edmonton, Canada. http://www.bme.med.ualberta.ca

BACKGROUND

Preliminary results suggest new possibilities for extracting information from sensors for use in various components of the neuroprosthetic system including: interpretation of user commands; detection of user intentions. discrimination locomotor events and phases see companion paper by Williamson et al.. feedback data for control loops; data for hand crafted rule based controllers. data for self adaptive controllers see companion paper by Thrasher et al.. data for system diagnostics. patient compliance and functional outcomes. supplementary sensory feedback to the user. In general, the sensors in a neuralprosthesis may be considered as windows to the biomechanical state of a patients neuro-muscular-skeletal system in which inadequate motion is being assisted by FES and unwanted muscle action (spasticity, clonus) is being inhibited by electrical blocking of the peripheral nerve. Each individual sensor is limited in what it can sense and the information it can provide. This can be thought of as a decomposition of the whole neuro-muscular-skeletal status information into its components by the sensors. In the field of pattern recognition and decision theory this is termed sensor (caused) fission [1]. The information fragmentation resulting from this unavoidable fission process can be partially reconstructed by a sensor (information) fusion process.

Here we present preliminary data on sensor fusion based on supervised machine learning to obtain a comprehensive set of signals for neuroprosthetic control from a suite of unobtrusive sensors. This modular sensor system is being developed for lower limb application in particular, a hybrid FES system to assist transfers, standing, obstacle negotiation and short range locomotion in paraplegia. The hybrid system comprises sensors, a control computer, a 22 channel implanted and an ankle foot orthoses [2].

OBJECTIVE Can machine learning technology be used to map a suite of sensors to a comprehensive set of biomechanical variables required for control of neural prostheses?

METHODS

Multi-Sensor Suite The sensor suit was chosen heuristically on the basis of minimizing encumbrance; availability of miniature, robust and low cost sensors; cosmesis reliability and ease of use. A more systematic method of selecting and positioning sensors, referred to as sensor simulation has been previously described [3]. The sensors are restricted to and integrated within the AFO's and a waistband. The sensors presently being investigated are: micro-machined accelerometers with a dc response (type ADXL05, Analog Devices Inc.); strain gauges; a custom electromagnetic position and angle sensor. Accelerometers were selected because they are rich in information on inclination and inertial motion. Strain gauges were selected to provide information on brace loading, in particular, to predict incipient knee buckling [5]. The electromagnetic sensor system was chhosen to provide information complementary to the accelerometers.

Four accelerometers are distributed in the waistband and three in each AFO cluster. The strain gauges were bonded to the anterior surface of the AFO's. The electro-magnetic position-angular sensor comprises three 12 kHz transmitters distributed in the waistband and three receivers in each of the AFO clusters. Each transmitter coil is activated in turn and the corresponding signal strengths from each receiver were sampled to provide information on relative angle and position of the motions between the transmitters and receivers. A single multicore cable was used to connect each AFO sensor cluster to the waistband. The methods described below do not require precise alignment of the sensors with any anatomical structures and no attempt was made to linearise or calibrate the outputs. The set is overdetermined and may be reduced depending on the FES control tasks e.g. for gait event detection the accelerometers in the AFO are sufficient see the companion paper [4] in these proceedings.In order to illustrate the technique we illustrate the approach with two examples:

There is often a requirement to monitor specific biomechanical variables that are not directly sensed. Here we illustrate this with two cases: by extracting the knee angle (typical of a signal used in closed loop control) during the standing up and sitting down into an office chair of an able bodied volunteer; by extracting the forward velocity of the foot (typical of a signal that may be used for biofeedback) of an able bodied individual walking on a powered treadmill.

In the hybrid FES system described in [2] the floor reaction orthosis stabilizes the knee without muscle activation provided the ground reaction vector is ahead of the knee joint. This allows the electrical stimulation to be turned off for most of the time thus avoiding FES induced fatigue. However should the vector shift behind the knee the extensor muscles must be immediately activated to avoid collapse [5]. We therefore need a knee buckle indicator or even better, an indicator of incipient buckling so that control action can begin before the knee begins to flex. Here we demonstrate that prediction of knee buckling is possible even without the use of the AFO strain gauges.

Knee angle during sit-stand-sit maneuvers.

In this example the waistband sensors and the right leg AFO sensor cluster were used. A reference knee angle signal was obtained from a flexible goniometer (Penny & Giles Ltd. UK) attached across the knee. All the signals were sampled at 100Hz. A neural network (employing the delta-bar-delta back-propogation rule, 11 input neurons and 6 hidden neurons) [6] was then trained to map the sensor inputs to the teacher signal from the Penny and Giles Goniometer. The raw sensor signals input to the ANN are shown in figure 1. (only four of the six electromagnetic signals were used - mainly reflecting the saggital motion of the right limb). The attributes input to the ANN were simply the signal amplitudes. After training the knee angle can then be predicted from the ANN as illustrated in figure 2.

Figure 1. Typical set of sensor signals. top to bottom: 4 electromagnetic; 4 waistband accelerometers; 3 accelerometers on the right AFO.

Figure 2. The predicted knee angle versus that measured by the Penny & Giles goniometer.

Foot velocity during treadmill walking

Here only the accelerometers on the right shank were used as input to a neural network. The derived attributes input to the ANN were; the sensor amplitudes; simple differences; a binary stance/swing signal. The ANN was generated using the Predict algorithm using a Kalman filter [6]. The reference forward velocity data was determined by affixing a retroreflective marker on the foot and measuring its displacement using a TV-computer system. After training the ANN was then able to predict the foot velocity as shown in figure 3.

Figure 3. The predicted foot velocity and the associated error during training and on unseen data.

Detecting Knee Buckling

Here the same sensor set shown in figure 1 was sampled at 100 Hz and input to the Datalogic R rough sets algorithm [7]. Figure 4 shows a typical knee buckle as indicated by the Penny and Giles flexible goniometer signal. The corresponding binary detection signal is shown superimposed.

DISCUSSION

A sensory system for lower limb FES is presented. Sensor fusion techniques using machine learning suggest that a broad range of signals can be extracted from this sensor suit. This means that biomechanical variables can be measured with "virtual sensors" from a sensor suite selected for ergonomic reasons. This technique may have more general application than FES e.g. whenever difficult to measure variables need to be monitored but the application does not allow traditional instruments to be used.

Figure 4. Knee buckling detector using Rough Sets. The teacher signal, knee angle, as measured by a Penny & Giles flexible goniometer is shown superimposed on the detector output.

References

/1/ Dasarathy B.V. (1994) Decision Fusion, IEEE Comp. Soc. Press, USA.

/2/ Davis R, Houdayer T, Andrews B, Patrick J, Mortlock A. (1995) Paraplegia: Hybrid Standing with the Cochlear FES-22 Stimulator and Andrews FRO Brace, Proc 2nd Sendai FES Internat. Symp., pp 206-210, Oct. 23-26, Japan.

/3/ Andrews B.J. (1995) On the Use of Sensors for FES Control, Proc. 5th Vienna Inter. Workshop on FES, pp 263-266, Aug. 17-19, Austria.

/4/ Williamson R, et al. (1996) Control of Neuralprostheses II: Gait Event Detection Using Accelerometers, in these RESNA proceedings

. /5/ Andrews BJ, Baxendale RH (1986) A Hybrid Orthosis Incorporating Artificial reflexes for SCI Patients, J. Physiol., 38: 19, 1986.

/6/ --- (1995) Trade literature on Predict and Neural Works, NeuralWare Inc, Pittsburgh, USA.

/7/ Rapaport J (1995) Rough Sets: The boundaries of Knowledge, PC AI, March 1995.

Acknowledgments This work was supported in part by the AHFMR, MRC and NSERC of Canada.

Brian J. Andrews Ph.D., Fax:(403)492 8259

Control of Neurapprostheses I:  Sensor Fusion