PROJECTS

1. Machine Learning for Medical Applications.


The volume and complexity of diagnostic imaging are increasing at a pace faster than the availability of human expertise to interpret it. Recently machine learning especially AI has shown great potential in medical data analysis, medical image processing, and automatic diagnosis and surgery assistance with improved quality of healthcare and largely reduced cost.

There are three main challenges in machine learning for medical applications. 1) plenty of sources in multiple formats, e.g., MRI, CT, ultrasound, file; 2) great anatomy and structure variance; 3) training difficulty: a few training data due to expensive labeling and a large size of each training data (e.g., the size of a CT image is about 512*512*(200-400));

We propose to combine AI and traditional computer vision methods to solve the above problems and implement clinic-acceptable automatic segmentation and diagnosis methods/frameworks/systems. Recently, we focus on a very challenging medical application, cognitive heart disease (CHD). We have published several papers in top AI conferences and the first segmentation dataset of CHD to the public.

Fig.1: Examples of our dataset with serious structure variations in CHD.

The publication list is as follows:

  • Xiaowei Xu, Tianchen Wang, Yiyu Shi, Haiyun Yuan, Qianjun Jia, Meiping Huang, and Jian Zhuang, "Whole-Heart and Great Vessel Segmentation in Congenital Heart Disease using Deep Neural Networks and Graph Matching," in Proc. of Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, 2019 (acceptance rate 31%).
  • Tianchen Wang, Jinjun Xiong, Xiaowei Xu, Meng Jiang, Yiyu Shi, Haiyun Yuan, Meiping Huang, and Jian Zhuang, "MSU-Net: Multiscale Statistical U-Net for Real-time 3D Cardiac MRI Video Segmentation," in Proc. of Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, 2019 (acceptance rate 31%).
  • Xiaowei Xu, Meiping Huang, Qianjun Jia, Hailong Qiu, Haiyun Yuan, Yuhao Dong, Jian Zhuang and Yiyu Shi, "Accurate Congenital Heart Disease Model Generation for 3D Printing," in Proc. of IEEE International Workshop in Signal Processing Systems, Nanjing, China, 2019. (Invited Paper)
  • Zihao Liu, Xiaowei Xu, Tao Liu, QI Liu, Yanzhi Wang, Yiyu Shi, Wujie Wen, Meiping Huang, Haiyan Yuan and Jian Zhuang, "Machine Vision Guided 3D Medical Image Compression for Efficient Transmission and Accurate Segmentation in the Clouds," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, 2019 (acceptance rate 25%).
  • Xiaowei Xu, Qing Lu, Lin Yang, Sharon Hu, Danny Chen, Yiyu Shi, "Quantization of Fully Convolutional Networks for Accurate Biomedical Image Segmentation," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt lake city, Utah, 2018 (acceptance rate 29.6%)
  • Xiaowei Xu, Qing Lu, Tianchen Wang, Jinglan Liu, Cheng Zhuo, Sharon Hu, Yiyu Shi, "Empowering Mobile Telemedicine with Compressed Cellular Neural Networks," in Proc. of IEEE/ACM 2017 International Conference On Computer-Aided Design, CA, 2017 (Invited Paper).
  • 2. Hardware/Software Co-Exploration for Neural Architectures.


    As the increasing number of applications (ranging from mobile and connected home to security, surveillance and automotive) that combine edge computing and machine learning, implementing neural networks on edge devices become ever more important; however, with limited hardware resources on the edge devices and the restricted timing specification, neural networks that attain state-of-the-art accuracy will not fit. We establish that in order to achieve the highest accuracy and hardware efficiency on the edge, it is best to co-explore hardware design and neural architecture (the above figure), rather than taking state-of-the-art approaches that either explore neural architecture only assuming fixed hardware, or explore hardware design only assuming fixed neural architecture. The proposed co-exploration framework opens up the hardware design freedom, which can significantly push forward the Pareto frontiers between hardware efficiency and test accuracy, and we can obtain much better design trade-offs than the existing approaches. Our contribution can be seen as an analogy to the concept of hardware-software co-design, which has prevailed due to its superior.

    We aim to develop a full-stack tool to support co-exploration of neural architecture and hardware design from (1) optimization of fixed neural network on FPGAs; (2) FPGA-implementation aware NAS; (3) co-explore neural architectures on multiple FPGAs; (4) co-explore neural architecture on network-on-chip-based scalable hardware.

    Fig.2: Illustration of co-exploring neural architectures and hardware designs.

    The publication list is as follows:

  • Weiwen Jiang, Lei Yang, Edwin Sha, Qingfeng Zhuge, Shouzhen Gu, Yiyu Shi and Jingtong Hu, "Hardware/Software Co-Exploration of Neural Architectures," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (under review).
  • Weiwen Jiang, Xinyi Zhang, Edwin H.-M. Sha, Qingfeng Zhuge, Lei Yang, Yiyu Shi and Jingtong Hu, "Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search," in Proc. of IEEE/ACM Design Automation Conference, 2019 (Nominated for Best Paper Award) (acceptance rate 25%)
  • Weiwen Jiang, Edwin Sha, Xinyi Zhang, Lei Yang, Qingfeng Zhuge, Yiyu Shi and Jingtong Hu, "Achieving Super-Linear Speedup across Multi-FPGA for Real-Time DNN Inference," IEEE/ACM International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), New York, NY, 2019 (Nominated for Best Paper Award) (acceptance rate 29%).
  • Lei Yang, Weiwen Jiang, Weichen Liu, Edwin Sha, Yiyu Shi, and Jingtong Hu, “Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence”, in Proc. of the Asia and South Pacific Design Automation Conference, ASP-DAC, 2019.
  • Qing Lu, Weiwen Jiang, Jingtong Hu and Yiyu Shi, "On Neural Architecture Search for Resource-Constrained Hardware Platforms," in Proc. of IEEE/ACM International Conference On Computer-Aided Design, Westminster, CO, Nov. 2019. (Invited Paper)
  • Xinyi Zhang, Weiwen Jiang, Yiyu Shi and Jingtong Hu, "When Neural Architecture Search Meets Hardware Implementation: from Hardware Awareness to Co-Design," in Proc. of IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Miami, FL, Aug. 2019. (Invited Paper)
  • 3. Uncertainties in Neural Networks.


    One of the foremost challenges in adopting deep neural networks (DNNs) to real-world mission-critical applications is the lack of competency-awareness. Humans make final decisions based on not only the most likely prediction but also the associated estimation of self-evaluated competence or confidence. For example, human doctors will conduct further investigations whenever they are in doubt for a diagnosis even when their best guess is a simple flu. In contrast, modern DNNs fails to associated its predictions with accurate competency estimation. In this project, we aim to improve the competency-awareness of DNNs so that they provide not only accurate predictions but also accurate quantitative measures of confidence. We develop competency-aware optimization methods for DNNs as well as reliable and informative evaluation metrics in different use cases.

    Fig.3: Competency-aware neural networks.

    The publication list is as follows:

  • Yukun Ding, Jinjun Xiong, and Yiyu Shi. "Uncertainty-aware Training of Neural Networks for Selective Medical Image Segmentation". Under Review
  • Yukun Ding, Jinglan Liu, Jinjun Xiong, and Yiyu Shi. "Evaluation of Neural Network Uncertainty Estimation with Application to Resource-Constrained Platforms". Under Review
  • 4. Statistical Convolutional Neural Network (SCNN) for Acceleration.


    Various convolutional neural networks (CNNs) have been developed that achieved accuracy comparable with that of human beings in computer vision tasks such as image recognition, object detection and tracking, etc. Most of these networks, however, may not fully utilize the temporal and contextual correlation typically present in multiple channels of the same image or adjacent frames from a video, thus limiting the achievable throughput. This limitation stems from the fact that existing CNNs operate on deterministic numbers.

    We propose to develop a general framework called statistical convolutional neural network (SCNN), which extends any existing CNN architectures but operates directly on correlated distributions rather than deterministic numbers. By introducing a parameterized canonical model to model correlated data and defining corresponding operations as required for CNN training and inference, we show that SCNN can process multiple correlated features more effectively, hence achieving significant speedup over existing CNN models.

    Fig.4: Statistical Convolutional Neural Network.

    The publication list is as follows:

  • Tianchen Wang, Jinjun Xiong, Xiaowei Xu, Meng Jiang, Yiyu Shi, Haiyun Yuan, Meiping Huang, and Jian Zhuang, "MSU-Net: Multiscale Statistical U-Net for Real-time 3D Cardiac MRI Video Segmentation," in Proc. of Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, 2019 (acceptance rate 31%).
  • Tianchen Wang, Jinjun Xiong, XIaowei Xu and Yiyu Shi, "SCNN: A General Distribution based Statistical Convolutional Neural Network with Application to Video Object Detection," AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019 (acceptance rate 16%).
  •