|
Douglas Thain Professor and Associate Chair Computer Science and Engineering University of Notre Dame 384C Fitzpatrick Hall |
New Online Textbook
|
Douglas Thain, Introduction to Compilers and Language Design, Second Edition, 2020. http://compilerbook.org |
DISC Online Course
Research
I lead the Cooperative Computing Lab,
where my team creates software
that allows people to easily harness hundreds
to thousands of computers from clusters, clouds, and grids. Our software
is used around the world to solve large scale problems in many fields
of science and engineering.
I am always looking for excellent students to join the CCL Team. To learn more, watch this movie, read my blog, or visit my research lab.
Courses
- CSE 30341 - Operating Systems Principles
- CSE 40771/60771 - Distributed Systems
- CSE 40822/60822 - Cloud Computing
- CSE 40243/60243 - Compilers and Language Design
- DISC - Data Intensive Scientific Computing (EdX Edge)
- CSE 40677 - Open Source Software Development
- CSE 66771 - Foundations of Distributed Systems
- CSE 20211 - Fundamentals of Computing
- CSE 60641 - Graduate Operating Systems
News in the CCL
- Accelerating Function-Centric Applications via Reusable Function Context in Workflow Systems
- A New Visualization Tool for TaskVine Released
- Integrating TaskVine with Merlin
- Introducing Shepherd: Simplifying Integration of Service Workflows into Task-Based Workflows
- TaskVine at ParslFest 2024
- Predicting Resources of Tasks in Dynamic Workflows with Bucketing Algorithms at IPDPS 2024
- TaskVine at the HEP Analysis Grand Challenge
- CCTools 7.8.0 released
- Distant Futures at SC 2023
Distributed Computing Blog
- Getting Beyond Stack Overflow for CS Students
- Compilers Book, First Edition
- Graduation Address for Catholic Engineers
- Build a Compiler for AlbaCore in Spring 2018
- Talk at ScienceCloud Workshop
- Online Course in Data Intensive Scientific Computing
- Writing a Compilers Textbook
- NunyaOS: An Experimental OS Kernel
- Sandboxes, Distributed Computing, and Closures
Recent Publications
- Barry Sly-Delgado and Ben Tovar and Jin Zhou and Douglas Thain,
Reshaping High Energy Physics Applications for Near-Interactive Execution Using TaskVine
(to appear) ACM/IEEE Supercomputing, pages 1-11, 2024. - Saiful Islam and Douglas Thain,
Shepherd: Seamless Integration of Service Workflows into Task-Based Workflows through Log Monitoring
Workshop on Workflows at ACM Supercomputing, pages 1-8, 2024. - Thanh Son Phung and Colin Thomas and Logan Ward and Kyle Chard and Douglas Thain,
Accelerating Function-Centric Applications by Discovering, Distributing, and Retaining Reusable Context in Workflow Systems
ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2024. DOI:10.1145/3625549.3658663 - Thanh Son Phung and Douglas Thain,
Adaptive Task-Oriented Resource Allocation for Large Dynamic Workflows on Opportunistic Resources
IEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 12, 2024. - Thanh Son Phung and Ben Clifford and Kyle Chard and Douglas Thain,
Maximizing Data Utility for HPC Python Workflow Execution
SC23 Workshop: High Performance Python for Science at Scale (HPPSS), 2023. DOI:10.1145/3624062.3624136 - Barry Sly-Delgado and Thanh Son Phung and Colin Thomas and David Simonetti and Andrew Hennessee and Ben Tovar and Douglas Thain,
TaskVine: Managing In-Cluster Storage for High-Throughput Data Intensive Workflows
18th Workshop on Workflows in Support of Large-Scale Science, 2023. DOI:10.1145/3624062.3624277
Slides From Recent Talks
- VC3: Virtual Clusters for Community Computation at DOE Collaborative Projects PI Meeting, September 2019.
- Scalable Application Design: Pitfalls and Possibilities, CVMFS Workshop at CERN, June 2019.
- Reduction of Workflow Resource Consumption Using a Density-based Clustering Model, WORKS Workshop at Supercomputing 2018.
- VC3: A Virtual Cluster Service for Community Computation, PEARC: Practice and Experience in Advanced Research Computing, 2018.
- Challenges in Delivering and Deploying Software on Large Clusters, Workshop on Runtime and Operating Systems for Supercomputers, June 2018.
- Seamless Scientific Computing from Laptops to Clouds,
keynote talk at ScienceCloud Workshop at HPDC, 2017.
My Ph.D. Graduates
- Dr. Nicholas Hazekamp, Engineer at Atomic Object.
- Dr. Chao (Charles) Zheng, Engineer at Alibaba.
- Dr. Peter Ivie, Engineer at VidAngel.
- Dr. James Sweet, postdoc at ND Center for Research Computing.
- Dr. Haiyan Meng (Engineer at Google, Inc). 2017.
- Dr. Patrick Donnelly (Engineer at Red Hat, Inc), 2016.
- Dr. Haipeng Cai, 2015, (Postdoc at Virginia Tech)
- Dr. Peter Sempolinski (Postdoc at Notre Dame), 2016.
- Dr. Dinesh Rajan, 2015, (Engineer at Amazon, Inc)
- Dr. Li Yu, 2013. (Engineer at Bloomberg, Inc)
- Dr. Peter Bui, 2012. (Faculty at Notre Dame, previously University of Wisconsin - Eau Claire)
- Dr. Hoang Bui, 2012. (Faculty at Western Illinois U, previously postdoc at Rutgers)
- Dr. Christopher Moretti, 2010. (Faculty at Princeton University)
- Dr. Jeffrey Hemmes, 2009. (Faculty at Auburn, previously Air Force Institute of Technology)
- Dr. Kyle Wheeler, 2009. (Micron, Inc, previously Sandia National Labs)
Research Funding
- VC3: Virtual Clusters for Community Computation, Douglas Thain, Robert Gardner, and John Hover, Department of Energy, 2016-2019.
- SI2-SSE: Scaling up Science with the Cooperative Computing Tools, Douglas Thain, National Science Foundation, April 2016-2019.
- SI2-SSE: Connecting Cyberinfrastructure with the Cooperative Computing Tools, Douglas Thain, National Science Foundation, April 2012-2015.
- dV/dt: Acclerating the Rate of Progress Toward Extreme Scale Collaborative Science, Miron Livny, Ewa Deelman, Douglas Thain, William Allcock, and Frank Wuerthwein, Department of Energy, September 2012-2015.
- REU Site: Data Intensive Scientific Computing, Douglas Thain and Kevin Lannon, National Science Foundation, February 2016-2019.
- DASPOS: Data and Software Preservation for Open Science,Michael Hildreth, Jaroslaw Nabrzyski, Mark Neubauer, Douglas Thain, and Robert Gardner, National Science Foundation, August 2012-2015.
- CDI-Type II: Open Sourcing the Design of Civil Infrastructure, Tracy Kijewski-Correa, Ahsan Kareem, Gregory Madey, Douglas Thain, August 2009-2013.
- CRI: Distributed Research Testbed (DiRT), Douglas Thain, National Science Foundation, August 2009-2012.
- "CSR-AES: Troubleshooting Large Scale Computing Grids with Machine Learning Techniques", Nitesh Chawla, Xiaohui Song, Shaowen Wang, and Douglas Thain, National Science Foundation, August 2007-2008.
- "The Notre Dame Extended Research Community", Mitchell Wayne, Thomas Loughran, Douglas Thain, Daniel Karmgard, Anna Goussiou, National Science Foundation, GK-12 Program, Sep 2007-2012.
- "CAREER: Data Intensive Grid Computing on Active Storage Clusters", Douglas Thain, National Science Foundation, Faculty Early Career Development Program, May 2007-2012.
- "HECURA: Deconstructing Clusters for High End Biometric Applications", Douglas Thain and Patrick Flynn, National Science Foundation, High End Computing University Research Activity, March 2007-2009.
- "SGER: Enabling Electronic Self-Defense with Dynamic Identities", Douglas Thain, National Science Foundation, Cybertrust Program, September 2005-2007.
- "An Experimental Approach to Integrative Research for Sensor-Rich Collaborative Teams" Christian Poellabauer, Nitesh Chawla, and Douglas Thain, Department of Defense, Defense University Research Instrumentation Program, April 2006-2007.
Professional Service
- Associate Editor, IEEE Transactions on Cloud Computing.
- Steering Committee, ACM High-Performance Parallel and Distributed Computing
- Program Chair, ACM High-Performance Parallel and Distributed Computing, 2011.
- Program Vice Chair, IEEE/ACM Cluster, Cloud, and Grid Computing 2013
- Workshops Chair, ACM High-Performance Parallel and Distributed Computing, 2010.
- Recent Program Committees:
- ACM High Performance Distributed Computing (HPDC), 2008-2021.
- TheWebConf (WWW), 2021.
- IEEE/ACM Supercomputing (SC) 2011-2021.
- Practice and Experience in Advance Research Computing,2019-2021
- IEEE International Conference on e-Science, 2015-2021.
- International Conference on Scientific and Statistical Database Management,2019-2021..
- IEEE/ACM Cluster, Cloud, and Grid Computing (CCGrid) 2011-2021.
- IEEE International Conference on Massive Storage Systems and Technology (MSST), 2014.
- IEEE International Conference on Networking Architecture, and Storage (NAS), 2014.
- Workshop on Workflow Models Systems, Services, and Applications in the Cloud, 2012.
- PDSW: Parallel Data Storage Workshop, 2011
- IEEE International Conference on Cloud Computing Technology and Science (CloudCom), 2011, 2010.
- IEEE Grid Computing, 2011, 2007, 2006, 2005.
- High Performance Computing and Communications, 2011.
- Euro-Par, 2010.
- Cloud Computing and its Applications, 2009, 2008.
- IEEE International Conference on Distributed Computing Systems, 2008, 2007.
- Middleware for Grid Computing Workshop, 2007, 2006.
- IEEE TCSC Doctoral Symposium, 2007.