CSE 40822/60822 - Cloud Computing - Fall 2018
MWF 10:30-11:25, Debartolo 319
Prof. Douglas Thain (dthain at nd dot edu)
TA: Haihui Tan (htan2 at nd dot edu)
Course Web Page
This class is an introduction to the theory and practice
building large scale computer systems that harness hundreds or thousands
for machines to attack problems of enormous scale. Such distributed
systems are necessary to solve problems of such large size that they
cannot complete in any reasonable time on a single machine.
In recent years, these system have been known as clouds
but have a much longer history known as distributed systems.
Cloud computing encompasses a variety of modes of computing,
including infrastructure and data center management, high
throughput computing, distributed programming models, scalable storage,
database systems, coordination and configuration, and more.
We will take a tour of many of these topics by alternating
with a high-level discusison of the principles, followed by a case
study with a current technology.
Each assignment will involve designing a program or system that scales
up to a large number of machines, using a variety of technologies.
This will be a highly practical class, and should be enjoyable to any student
who likes to write lots of code and make real systems work. Many students
who take this class end up using these tools in their daily work. The class is open to juniors, seniors, and graduate students.
Lectures and Readings
The course structure will alternate between a discussion of the broad
principles of distributed computing and detailed case studies of specific
cloud technologies. I have not yet found an excellent, broad, textbook
about cloud computing, and so the lectures will be supplemented with
a variety of articles from ACM proceedings, software websites, and so forth.
Note that the readings for each week are background material
that will help you to be ready for the lecture and the discussion.
The readings are not a substitute for the lectures.
For each case study, there will be an assignment in which you
will use the software or system to accomplish a task using large
scale computing. This will give you some hands-on experience with
the technology, and cause you to wrestle with the fundamental
challenges so that you understand the strengths and weaknesses
of each system.
Graduate students enrolled in 60822 will have some additional assignments
to develop a scholarly bibliography of related work, culminating in a
For the final project, small groups will propose to build and
evaluate a software system that relies on cloud computing in
order to scale up. Example projects might include a scalable
filesystem, a social networking system, or a scientific computing portal.
We will provide access to several large computing systems here
at Notre Dame which are no cost to you: our 10K-core Condor pool,
and a 180-TB Hadoop cluster. Later in the semester, we will
make use of commercial cloud providers, which will require
that you create an account coupled with a credit card on those systems.
A reasonable credit (usually $100) is available for academic use
of those systems, so that you will be able to experiment at reasonable
scale at no additional cost. However, if you go over the limit,
you will be responsible for the additional costs.
We will discuss how to monitor your usage at appropriate points during the semester.
- Junior, senior, or graduate standing.
- Proficiency in the C programming language and scripting languages such as Perl or Python.
- Proficiency in using and troubleshooting Linux command line tools.
Students successfully completing this course will be able to:
Describe and compare the architecture and operation of various cloud computing systems.
Presented in course readings and class discussions, evaluated in exams.
Describe how distributed systems are fundamentally different from standalone systems in matters such as naming, invocation, synchronization, and fault tolerance.
Presented in course readings, evaluated in exams.
Construct, test, and evaluate programs in a cloud computing environment.
Presented, practiced, and evaluated in the practical assignments.
Communicate technical results orally and in writing.
Practiced in assignment writeups and final project talk.
Students are expected to attend all class meetings and exams,
and turn in all assignments before the due date and time.
Exceptions will only be made for the grave circumstances
outlined in the duLac student life handbook.
Students who must be absent due to participation in a
University activity must confer with the instructor at the
earliest possible date.
All assignments are due at the beginning of class on the date due.
Late assignments will receive no credit.
This includes assignments submitted after class has begun.
Paper assignments should be brought to class.
Programming assignments will be turned in electronically by copying all
required files to a "dropbox" directory.
You are free to turn in assignments
multiple times before the deadline expires.
It would be a good habit to turn in an incomplete but working
assignment on a daily basis.
Thus, there is no excuse for failing to turn in an assignment:
everyone should turn in something long before the deadline.
Exceptions will be made only in grave circumstances.
Grades will be weighted as follows:
| Assignments || 50 % (A0, A1, A2, A3, A4)
| Final Project || 30 % (Presentation, Final Paper, and Code)
| Midterm Exam || 10 %
| Final Exam || 10 %
Take-home assignments are to be completed individually.
The final project is to be completed in pairs.
In-class exams are to be completed alone.
Students are encouraged to seek out and exploit reference manuals,
books, websites, and other documentation that will help you to complete each take-home assignment,
provided that you indicate what sources you have used.
However, all software development, experimental work,
and writing of results must be by your own hands, in your own words.
Notre Dame students are expected to abide by the Academic Code of Honor Pledge: "As a member of the Notre Dame community, I will not participate in or tolerate academic dishonesty."
A Note About the Assignments
To undertake the class assignments, you will have to
spend some time learning how to use each of these systems.
The lectures will explain the general principles and direct you towards
reference materials. However, you will need to figure out some of the
details on your own by reading manuals and experimenting
on your own. Do not expect to succeed on the first try.
Plan to spend tmie looking up options, experimenting with features,
and puzzling out error messages.
You may need to consult with the instructors during class, office hours,
or via email to solve problems. Thus, start working on each assignment
right away, and do not leave it until the last minute.