CSE 40822/60822 - Cloud Computing - Fall 2014

Instructors

  • Prof. Douglas Thain
    dthain at nd dot edu
    Office Hours: Tue 2-4PM, 382 Fitzpatrick Hall
  • Prof. Dong Wang
    dwang5 at nd dot edu
    Office Hours: Thu 2-3:30, 214B Cushing Hall
  • TA: Chao "Charles" Zheng
    czheng2 at nd dot edu
    Office Hours: Mon 2-4PM, Wed 4:30-6:30PM, 212 Cushing Hall

    • Getting Help

  • Piazza Discussion Page - The best place for technical questions like "What does this error message mean?"
  • Office Hours - One of the instructors is available each afternoon Mon-Thu.
  • Email - Contact Prof. Thain for questions about grades, course policies, etc.
  • Recorded Lectures are available on Sakai. Recordings are good for reviewing what happened, but not as good as being there in person!
  • Grades are available in Sakai.

    • Course Overview

    This class is an introduction to the theory and practice building large scale computer systems that harness hundreds or thousands for machines to attack problems of enormous scale. Such distributed systems are necessary to solve problems of such large size that they cannot complete in any reasonable time on a single machine. In recent years, these system have been known as clouds but have a much lnger history known as distributed systems.

    Cloud computing encompasses a variety of modes of computing, including infrastructure and data center management, high throughput computing, distributed programming models, No-SQL storage, and more. We will take a tour of many of these topics by alternating with a high-level discusison of the principles, followed by a case study with a current technology.

    Each assignment will involve designing a program or system that scales up to a large number of machines, using a variety of technologies. This will be a highly practical class, and should be enjoyable to any student who likes to write lots of code and make real systems work. Many students who take this class end up using these tools in their daily work. The class is open to juniors, seniors, and graduate students.

    Course Documents

  • Syllabus
  • Class Mailing List
  • A0 - Warm Up Assignment
  • A1 - High Throughput Ray-Tracing with Condor
  • A2 - Parallel DNA Analysis with Work Queue
  • A3 - Web Data Analysis Using Hadoop
  • A4 - Data Processing Using Amazon AWS
  • Final Project
  • Project Presentation

    • Tentative Schedule

    Week Lecture Materials
    25 August The Cloud Landscape Reading: A View of Cloud Computing
    Reference: 2013 Hype Cycle
    1 September Principles of Distributed Computing Reading: A Note on Distributed Computing
    A0: Warm Up Assignment
    8 September Case Study: Condor Reading: Condor Overview Paper
    Reference: Condor at Notre Dame
    Reference: Condor 8.0 Manual
    A1: High Throughput Ray-Tracing with Condor
    15 September Cloud Programming Models
    22 September Case Study: Makeflow and Work Queue Lecture: Makeflow Lecture Slides
    Reference: Makeflow Web Page
    Lecture: Work Queue Lecture Slides
    Reference: Work Queue Web Page
    Tutorial: In-Class Tutorial
    A2: Parallel DNA Analysis with Work Queue
    29 September Case Study: Hadoop Reading: Google Map-Reduce Paper
    Reference: Data Intensive Text Processing with Map-Reduce
    Reference: Hadoop Project Web Page
    Reference: Hadoop at Notre Dame
    Lecture: Hadoop Lecture Slides Part 1
    Lecture: Hadoop Lecture Slides Part 2
    A3: Web Data Analysis with Hadoop
    6 October The Hadoop Stack Reading: Pig Latin Paper (Apache Pig)
    Lecture: Pig Latin Slides
    Reading: BigTable Paper (Apache HBase)
    Lecture: HBase Slides
    13 October The Hadoop Stack Continued Reading: Spark Paper (Apache Spark)
    Lecture: Spark Slides
    Project Proposals due October 17
    20 October Fall Break
    27 October Case Study: Amazon AWS
    		 Reference: Amazon AWS Docs
    Lecture: Amazon AWS Slides
    Midterm Exam on Friday, October 31
    3 November Scaling Up Web Applications Reference: Amazon Architecture Center
    Reference: Memcached Project
    Lecture: Scaling Up Notes
    A4: Data Processing Using Amazon AWS
    10 November The CAP Theorem Reading: Perspectives on the CAP Theorem
    Reading: Eventually Consistent
    Lecture: CAP Lecture Slides
    Project Updates During Office Hours
    17 November Coordination and Configuration Reference: Puppet Docs
    Lecture: Puppet Lecture Notes
    Reading: Zookeeper Paper (Apache Zookeeper)
    Lecture: Zookeeper Lecture Notes
    Reading: Mesos Paper (Apache Mesos - Docker)
    Lecture: Mesos and Docker
    24 November Case Study: Open Stack Reference: Open Stack (Section 1)
    Thanksgiving Break Wed-Fri
    1 December Project Presentations
    8 December Project Presentations
    Final Project Due Wednesday, December 10th
    17 December4:15PM in 125 DeBart Final Exam