Neural Networks in Control Systems

The state of the art in the area of neural networks in control systems is reflected in this special issue. The need to meet demanding control requirements in increasingly complex dynamical control systems under significant uncertainties makes neural networks very attractive, because of their ability to leam, to approximate functions, to classify pattems and because of their potential for massively parallel hardware implementation. Neural networks do appear to be able to implement many functions essential to control systems with higher degree of autonomy.

This is the second special issue of the IEEE Control Systems Magazine devoted to Neural Networks in Control Systems; the first special issue was two years ago, in April 1990 [1]. In fact, this is the fourth issue of the Magazine with a collection of articles specifically aimed at neural networks in control [1]-[3].

Two years ago the area of neural networks in control systems was at its early stages of development. There were many hopes for the field, and fewer accomplishments. Over the past two years the field has been developing, but not by surprising leaps and bounds. Rather it has been evolving through steady progress. Certain views and approaches have now emerged to become accepted and popular. The field is also moving away from blind applications of large neural networks to applications on more specific problems. The standards are high and publication of results in the area requires proper justification of the particular approach taken and proof of the claims that are made. I will try here to give an overview of the main approaches.

The type of neural network most commonly used is the feedforward multilayer neural network, where no information is fed back during operation. There is, however, feedback information available during training. Supervised learning methods, where the neural network is trained to learn input/output patterns presented to it, are typically used. Most often, versions of the backpropagation algorithm are used to adjust the neural network weights during training; this is generally a slow and very time consuming process as the algorithm usually takes a long time to converge. The individual neuron activation functions most often are sigmoidal functions, but they also may be signum or Gaussian functions.

One property of multilayer neural networks is central to most applications to control. Such networks can generate input/output maps which can approximate, under mild assumptions, any function with any desired accuracy. One may have to use a large number of neurons, but any desired approximation, if it can be accomplished at all, it can be accomplished with a multilayer network with only one hidden layer of neurons or two layers of weights.

To model the inputloutput behavior of a dynamical system, the neural network is trained using input/output data and the weights of the neural network are adjusted most often using the backpropagation algorithm. Because the typical application involves nonlinear systems, the neural network is trained for classes of inputs and initial conditions. The underlying assumption is that the nonlinear static map generated by the neural network can adequately represent the system 's behavior in the ranges of interest for the particular application. There is of course the question of how accurately a neural network, which realizes a static map, can represent the inputloutput behavior of a dynamical system. For this to be possible one must provide to the neural network information about the history of the system - typically delayed inputs and outputs. How much history is needed depends on the desired accuracy. There is a tradeoff between accuracy and computational complexity of training, since the number of inputs used affects the number of weights in the neural network and subsequently the training time. One sometimes starts with as many delayed input signals as the order of the system, and then modifies the neural network accordingly; it also appears that using a two hidden layer neural network - instead of a one hidden layer network - has certain advantages. The number of neurons in the hidden layer(s) is typically chosen based on empirical criteria and one may iterate over a number of networks to determine the neural network that has a reasonable number of neurons and accomplishes the desired degree of approximation.

When a multilayer neural network is trained as a controller - either an open or closed loop controller - most of the issues are similar to the above. The difference is that the desired output of the neural network - the controller generated appropriate control input to the plant - is not readily available, but has to be induced from the known desired plant output. For this, one either uses approxima tions based on the mathematical model of the plant if available, or a neural model of the dynamics of the plant or even of the dynamics of the inverse of the plant. In the latter case the assumption is that the inverse dynamics can be represented by a neural network.

Neural networks may be combined to both identify and control the plant, thus imple menting an adaptive controller. It is also pos sible to adaptively change the neural controller based on an additional training signal, which is an indication of how well the system is performing - using, that is, a critic to help adjust the neural controller parameters.

Neural networks can also be used to detect and identify system failures, and to help store information for decision making - thus provid ing the ability to decide when to switch to a different controller among a finite number of controllers, or to classify pattems etc.

In general, there are potential applications of neural networks at all levels of hierarchical intelligent controllers that provide higher degrees of autonomy to systems [4]. Neural networks are useful at the lowest execution level - where the conventional control algo rithms are implemented via hardware and software - through the coordination level, to the highest organizational level, where decisions are being made based on possibly uncertain and/or incomplete information. One may point out that at the execution level - the conventional control level - neural network properties such as the ability for function ap proximation and the potential for parallel im plementation appear to be most important. In contrast, at higher levels, abilities such as pattern classification and the ability to store information in a, say, associative memory appear to be of most interest.

Theoretical developments are of course of great importance. Today we have greater understanding of the fundamental neural network characteristics compared to two years ago. However, in a control system which contains neural networks it is hard to prove typical control system properties such as stability. The main reason is the mathematical difficulties associated with nonlinear systems controlled by highly nonlinear neural network controllers. In view of the mathematical difficulties encountered in the past in the adaptive control of linear systems controlled by linear controllers, it is hardly surprising that the analytical study of nonlinear adaptive control using neural networks is a difficult problem indeed. Some progress has been made in this area and certain important theoretical results have begun to emerge, but clearly the overall area is still at its early stages of development. The encouraging news is that there are successful applications of neural networks in control systems that work, and this certainly provides clues and guidelines for the corresponding theoretical development.

This special issue contains seven articles all of which were presented, in some form, at conferences in 1991; at the American Control Conference, the IEEE Robotics and Automation Conference, or at the IEEE 7th International Symposium on Intelligent Control. In selecting these papers the aim was to present as varied and current picture of the research in the field as possible. This is an emerging technology with a great variety of ideas being applied to new and old applications, and it is important to represent this variety. It is also a fast paced technology, and so it is essential to provide timely coverage when capturing the state of the art. The coverage is by necessity restricted due to editorial deadlines and length constraints and this Special Issue provides but a window through which to view the current state of the art. Certainly the approaches presented here on neural networks in control systems are not the only ones. Collected together, however, they provide a good picture of the trends in the area.

The first article by K. S. Narendra and S. Mukhopadhyay titled "Intelligent Control Using Neural Networks," deals with controlling a system when structural failures occur. A two-level hierarchical neural network controller is used. The higher level detects and classifies a failure as a member of a particular class of failures; it then activates a prestored fixed controller known to stabilize the system under this particular class of failures. At the lower level, an adaptive neural network controller is then used to improve the response of the system on line; this is accomplished by identifying the new values of the system parameters and updating the parameters of the controller.

In "Reinforcement Learning is Direct Adaptive Optimal Control," by R. S. Sutton, A. G. Barto and R. J. Williams, Q-learning is used as a method to implement direct adaptive optimal control to nonlinear systems. Generally, direct methods are less computationally intensive than indirect and it is important to be able to use direct methods in problems which are very demanding computationally, such as optimal control of nonlinear systems. In this paper Q-learning, a reinforcement learning method is discussed as an on-line dynamic programming method to perform, in a relatively computationally inexpensive way direct optimal adaptive control.

CMAC (Cerebellar Model Articulation Controller) neural networks are capable of learning nonlinear functions very quickly due to the local nature of weight updating. Higher order CMAC neural networks which can learn to approximate both functions and function derivatives are introduced in the next paper "Theory and Development of Higher-Order CMAC Neural Networks" by S. H. Lane, D. A. Handelman, and J. J. Gelfand. For this, B-Spline receptive field functions are used in conjunction with more general weight addressing schemes. Learning methods for these networks are also discussed.

Visual feedback information is used to control a robotic manipulator in "Self-Organizing Visual Servo System Based on Neural Networks" by H. Hashimoto, T. Kubota, M. Kudou and F. Harashima. The task is to move the manipulator end-effector in a position where gripping of an object can easily be performed. Two neural networks - global and local - are used to learn the nonlinear mapping between image data and joint angles; the first network learns the appropriate control signals for longer and the second network for shorter object distances.

The next paper "Hierarchical Neurocontroller Architecture for Robotic Manipulation," by L. C. Rabelo and X. J. R. Avula, discusses how neural networks may be utilized at various stages of controlling the motion of a 2-link robot arm. A hierarchical neural network controller is used where the higher level deals with motion analysis issues - there are three distinct neural networks addressing delineation of the robot arm workspace, coor dinate transformation and the motion decision making process - while the lower level provides the appropriate control law using an emulator of the arm dynamics.

The paper "Disturbance Pattern Classifica tion and Neuro-Adaptive Control" by D. J. Cooper, L. Megan and R. F. Hinde, Jr. discus ses an adaptation strategy to cope with load disturbances in process control. The strategy is based on an analysis of patterns found in past error and control input variables. It uses a neural network to learn a mapping from the error signal of a poorly tuned controller to the nature of the source of the problem - step component in an otherwise oscillatory distur- bance that may cause the process charac- teristics to change - and hence to a modification of the controller.

In the last paper titled "Implementations of Learning Control Systems using Neural Net- works," by M. A. Sartori and P. J. Antsaklis, a particular neural network and a direct method to assign its weights are introduced to sys- tematically incorporate prior knowledge about the system's behavior when using neural networks to design subsystems in a control system. Note that in general sig- nificant prior information about a variety of system characteristics is typically available to the designer and its systematic use is very important as it may lead to significant com- putational savings, when compared to direct application of learning methods. Here the prior information is assumed to be in certain form - input/output data points together with additional specifications for the behavior be tween the given points - and an algorithm to design an appropriate neural network which incorporates this information is given. Learn ing methods can subsequently be used if ap propriate to refine the design. The approach is then used in dynamical system (plant and con troller) modeling, in failure detection and identification, in information extraction problems and in control scheduling.

I am indebted to the authors and referees for all their efforts put forth in developing this Special Issue and to the Magazine Editor Herb Rauch for all his suggestions and help.

[1] P.J.Antsaklis, Guest Ed., Special Issueon Neural Netwgrks for in Control Systems, IEEE Control Syst. Mag., vol. 10, no. 3, pp. 3-87, Apr. 1990.
[2] Special Section on Neural Networks for Control Systems, IEEE Control Syst. Mag., vol. 9, no. 3, pp. 25-59, Apr. 1989.
[3] B. Bavarian, Guest Ed., Special Section on Neural Networks for Systems and Control, IEEE Control Syst. Mag., vol. 8, no. 2, pp. 3-31, Apr. 1988.
[4] P.J. Antsaklis, K.M. Passino, and S.J. Wang, "An introduction to autonomous control systems", IEEE Control Syst. Mag., vol. 11, pp. 5-13, June 1991.

Panos J. Antsaklis received his Diploma in Mechanical and Electrical Engineering from the National Technical University of Athens (NTUA), Greece, in 1972 and his M.S. and Ph.D. degrees in Electrical Engineering from Brown University in 1974 and 1977, respectively. He is currently professor of electrical engineering at the University of Notre Dame. He has held faculty posi tions at Brown University, Rice University, and Imperial College, University of London. During sabbatical leaves, he has been senior visiting scientist at LIDS of MIT in 1987 and at Imperial College in 1992; he also was visiting professor at NTUA in 1992 and at the Technical University of Crete, Greece in 1993. His research interests are in Systems and Control Theory, with emphasis in the control of hybrid and discrete event systems, in autonomous, intelligent and learning control systems, in neural networks and in m