Learning How to Control

Dept. of Electrical Engineering
University of Notre Dame


National Science Foundation
June 15, 2023- June 14, 2026

Description: Internet-of-Things (IoT) enabled manufacturing systems (a.k.a. Manufacturing 4.0) form a particularly important class of cyberphysical system (CPS). IoT-enabled manufacturing systems have a physical fabric woven from a heterogeneous mix of machines carrying and processing materials across the factory floor. The cyber fabric for these systems is a heterogeneous mix of wired and wireless digital communication networks enabling the global visibility of numerous data streams used by cloud-based planners and edge devices to manage the physical fabric's workflows. These IoT-enabled systems are complex CPS with a great deal of modeling uncertainty. The physical and cyber fabrics are open to an external environment that can shift in an abrupt and unpredictable manner. Such shifts may be due to changes in customer work orders or due to environmental changes that increase interference in the cyber fabric's wireless (RF) networks. The dynamics of both fabrics are coupled since congestion in th physical fabric may create congestion in the cyber fabric and vice versa. This complexity and uncertainty stand as major obstacles to the broader acceptance of IoT technologies by U.S. manufacturers. To lower the risk in adopting IoT technolo- gies, this project proposes developing methods that learn how to control complex CPS in Manufac- turing 4.0 applications. This project addresses that need by developing methods that learn how to control complex CPS found in IoT-enabled manufacturing. This project proposes doing that by developing a meta-learning framework for control based on a novel machine learning model that appears to address robustness issues found in competing deep reinforcement learning for control. This project will develop algorithms and software implementations of this novel approach to meta-learning. The project will benchmark the method's performance on a testbed capturing the complex interaction's between an IoT-manufacturing system’' physical and cyber fabrics.
Products:
Intellectual Merit: The intellectual merit of this proposal centers on meta-learning algorithms for the control of complex and uncertain CPS performing a range of tasks. The first novel aspect of this work is its adoption of a new type of machine learning model called a behaviorally ordered abstraction (BOA). This model has greater cross-task generalization capacity, better sample efficiency, and greater interpretability than other deep learning methods. The project's second novel aspect is its potential to address issues regarding the robust stability of deep reinforcement learning. The third novel aspect of this work is its embedding of meta-learning in a generalized regulator in which the framework learns “how” to configure controller synthesis across all tasks. The project's fourth novel aspect of is its benchmarking of meta-learning methods in IoT-enabled manufacturing.
Broader Impacts: The proposed research, if successful, would address critiques regarding the robustness of deep reinforcement learning. The proposed research, if successful, would transform the use of IoT technologies in complex CPS. The proposed research would also lower the barriers that US industries face in adopting IoT technologies, thereby improving U.S. industrial global competitiveness. Regarding curriculum development, this project will be integrated into a professional MS program with a concentration in CPS and IoT systems engineering. Educational and training opportunities will be provided for women through a joint program between Notre Dame and Saint Mary's College. The project will leverage existing programs such as EngRET@ND for K-12 outreach.