1. Hardware/Software Co-Exploration for Neural Architectures. [slides]

As the increasing number of applications (ranging from mobile and connected home to security, surveillance and automotive) that combine edge computing and machine learning, implementing neural networks on edge devices become ever more important; however, with limited hardware resources on the edge devices and the restricted timing specification, neural networks that attain state-of-the-art accuracy will not fit. We establish that in order to achieve the highest accuracy and hardware efficiency on the edge, it is best to co-explore hardware design and neural architecture (the above figure), rather than taking state-of-the-art approaches that either explore neural architecture only assuming fixed hardware, or explore hardware design only assuming fixed neural architecture. The proposed co-exploration framework opens up the hardware design freedom, which can significantly push forward the Pareto frontiers between hardware efficiency and test accuracy, and we can obtain much better design trade-offs than the existing approaches. Our contribution can be seen as an analogy to the concept of hardware-software co-design, which has prevailed due to its superior.

We aim to develop a full-stack tool to support co-exploration of neural architecture and hardware design from (1) optimization of fixed neural network on FPGAs; (2) FPGA-implementation aware NAS; (3) co-explore neural architectures on multiple FPGAs; (4) co-explore neural architecture on network-on-chip-based scalable hardware.

Fig.1: Illustration of co-exploring neural architectures and hardware designs.

The publication list is as follows:

2. On-Device Personalized AI.

Artificial Intelligence and Machine Learning (AI/ML) techniques have been deployed in an increasing number of local devices at the user end for better privacy-preserving and quick response. The applications range from autonomous driving, personal health assitance, to comprehensive vital signs monitoring. In the real-world application scenarios, all these integrated advanced techniques are targeting an overarching goal: enhancing personalized experiences. This involves providing precise detection for each individual subject, as well as personalized plans and preventative actions based on the ambient environments. However, the inter- and intra-subject variability, changing system requirements as well as unstable conditions make it difficult to ensure optimal service across users. The AI/ML models need to be properly personalized for better user experiences. On the other hand, the limited hardware resources (in terms of memory capacity, computing power, energy budget, etc) of local devices greatly hinder the development of advanced AI/ML approaches on the local end.

We aim to develop a algorithm-system co-design framework to address the aforementioned challenges to support the development of personalized AI/ML on resource-constrained local-end hardware platforms. The framework ranges from 1) Optimizations on tiny deep model generalization; 2) Prior-incorported regulated deep model personalization; 3) AIGC-based self-supervised personalization; and 4) TinyML design paradigm.

Fig.2: AIGC-based self-supervised personalization framework for implantable monitoring devices.

The publication list is as follows:

3. Unsupervised Medical Image Computing.

The success of deep learning heavily depends on the availability of large labeled training sets. However, it is hard to get large labeled datasets in the medical image domain because of the strict privacy concern and costly labeling efforts. Therefore, it is desirable that the DNNs can leverage the numerous unlabeled medical data to achieve higher performance with limited annotations. Contrastive learning, as a self-supervised learning (SSL) method, has shown great success in learning image-level features from large-scale unlabeled data. The main idea of contrastive learning is to contrast the similarities of sample pairs in the representation space through contrastive loss, pulling the representations of similar pairs (a.k.a. positive pairs) together and pushing the representations of dissimilar pairs (a.k.a. negative pairs) apart. After pre-training, the trained encoder can be used as the initialization to improve downstream tasks such as object detection and image segmentation. We aim to leverage the underlying structural information in medical image for SSL and propose several techniques (e.g., contrastive image synthesis, positional contrastive learning, BYOL-TracIN, etc.) to enhance its representation learning ability in the medical image field.

Fig.3: Positional Contrastive Learning.

The publication list is as follows: