CSE 40822/60822 - Cloud Computing - Fall 2018

MWF 10:30-11:25, Debartolo 319

Prof. Douglas Thain (dthain at nd dot edu)
TA: Haihui Tan (htan2 at nd dot edu)

Course Web Page

http://www.nd.edu/~dthain/courses/cse40822/fall2018

Course Overview

This class is an introduction to the theory and practice building large scale computer systems that harness hundreds or thousands for machines to attack problems of enormous scale. Such distributed systems are necessary to solve problems of such large size that they cannot complete in any reasonable time on a single machine. In recent years, these system have been known as clouds but have a much longer history known as distributed systems.

Cloud computing encompasses a variety of modes of computing, including infrastructure and data center management, high throughput computing, distributed programming models, scalable storage, database systems, coordination and configuration, and more. We will take a tour of many of these topics by alternating with a high-level discusison of the principles, followed by a case study with a current technology.

Each assignment will involve designing a program or system that scales up to a large number of machines, using a variety of technologies. This will be a highly practical class, and should be enjoyable to any student who likes to write lots of code and make real systems work. Many students who take this class end up using these tools in their daily work. The class is open to juniors, seniors, and graduate students.

Lectures and Readings

The course structure will alternate between a discussion of the broad principles of distributed computing and detailed case studies of specific cloud technologies. I have not yet found an excellent, broad, textbook about cloud computing, and so the lectures will be supplemented with a variety of articles from ACM proceedings, software websites, and so forth. Note that the readings for each week are background material that will help you to be ready for the lecture and the discussion. The readings are not a substitute for the lectures.

Assignments

For each case study, there will be an assignment in which you will use the software or system to accomplish a task using large scale computing. This will give you some hands-on experience with the technology, and cause you to wrestle with the fundamental challenges so that you understand the strengths and weaknesses of each system.

Graduate students enrolled in 60822 will have some additional assignments to develop a scholarly bibliography of related work, culminating in a technical paper.

For the final project, small groups will propose to build and evaluate a software system that relies on cloud computing in order to scale up. Example projects might include a scalable filesystem, a social networking system, or a scientific computing portal.

Resources

We will provide access to several large computing systems here at Notre Dame which are no cost to you: our 10K-core Condor pool, and a 180-TB Hadoop cluster. Later in the semester, we will make use of commercial cloud providers, which will require that you create an account coupled with a credit card on those systems. A reasonable credit (usually $100) is available for academic use of those systems, so that you will be able to experiment at reasonable scale at no additional cost. However, if you go over the limit, you will be responsible for the additional costs. We will discuss how to monitor your usage at appropriate points during the semester.

Prerequisites

Course Outcomes

Students successfully completing this course will be able to:
  • Describe and compare the architecture and operation of various cloud computing systems.
    Presented in course readings and class discussions, evaluated in exams.
  • Describe how distributed systems are fundamentally different from standalone systems in matters such as naming, invocation, synchronization, and fault tolerance.
    Presented in course readings, evaluated in exams.
  • Construct, test, and evaluate programs in a cloud computing environment.
    Presented, practiced, and evaluated in the practical assignments.
  • Communicate technical results orally and in writing.
    Practiced in assignment writeups and final project talk.

    • Attendance

    Students are expected to attend all class meetings and exams, and turn in all assignments before the due date and time. Exceptions will only be made for the grave circumstances outlined in the duLac student life handbook. Students who must be absent due to participation in a University activity must confer with the instructor at the earliest possible date.

    Grading

    All assignments are due at the beginning of class on the date due. Late assignments will receive no credit. This includes assignments submitted after class has begun.

    Paper assignments should be brought to class. Programming assignments will be turned in electronically by copying all required files to a "dropbox" directory. You are free to turn in assignments multiple times before the deadline expires. It would be a good habit to turn in an incomplete but working assignment on a daily basis. Thus, there is no excuse for failing to turn in an assignment: everyone should turn in something long before the deadline. Exceptions will be made only in grave circumstances.

    Grades will be weighted as follows:
    Assignments 50 % (A0, A1, A2, A3, A4)
    Final Project 30 % (Presentation, Final Paper, and Code)
    Midterm Exam 10 %
    Final Exam 10 %

    Collaboration

    Take-home assignments are to be completed individually. The final project is to be completed in pairs. In-class exams are to be completed alone.

    Students are encouraged to seek out and exploit reference manuals, books, websites, and other documentation that will help you to complete each take-home assignment, provided that you indicate what sources you have used. However, all software development, experimental work, and writing of results must be by your own hands, in your own words.

    Notre Dame students are expected to abide by the Academic Code of Honor Pledge: "As a member of the Notre Dame community, I will not participate in or tolerate academic dishonesty."

    A Note About the Assignments

    To undertake the class assignments, you will have to spend some time learning how to use each of these systems. The lectures will explain the general principles and direct you towards reference materials. However, you will need to figure out some of the details on your own by reading manuals and experimenting on your own. Do not expect to succeed on the first try. Plan to spend tmie looking up options, experimenting with features, and puzzling out error messages. You may need to consult with the instructors during class, office hours, or via email to solve problems. Thus, start working on each assignment right away, and do not leave it until the last minute.