Project Ideas

The following are rough project ideas for your consideration. Part of your job will be to crystallize the purpose, methods, and scope of your specific project into a project proposal. Note that not all projects involving writing a lot of code, but all involve a thorough quantitative evaluation of a system. Students may undertake projects not listed here, but should consult with the instructor before submitting a proposal. You may certainly take any of these ideas and add a "twist" to make it more interesting or challenging.

Many of these project ideas make use of software and systems that are already deployed at Notre Dame, such as the Chirp distributed filesystem, the Condor distributed batch system, and the Hadoop data processing system. I encourage you to use these systems, so that you will have a ready-made testbed to work with, and will become familiar with tools useful later in your career.

  • Distributed RAID. The Chirp filesystem gives easy access to lots of different storage devices. Chirp allows the user to read and write files on a single remote disk, but does not provide any kind of replication or error checking. Remedy this by creating a library for creating and accessing large files that are striped across multiple Chirp servers. Your library support several different RAID configurations, selected when the file is created. The library should have a simple interface like chirp_raid_open, chirp_raid_pread, chirp_raid_pwrite, chirp_raid_stat, and so forth. Get started by using the Chirp API. Explore the performance of this library on a variety of workloads, varying the RAID configuration and the number of servers in use. How does the performance compare to using a single local disk or a single remote Chirp server?

  • Filesystem or Database? Filesystems are good at storing large amounts of binary data, but not very good at searching for the needle in the haystack. Perhaps a combination of the two owuld provide robust searching with good performance. Read about the Inversion filesystem, and then build something similar using current open source technologies such as FUSE and MySQL. Populate the system with a large amount of data, and evaluate the performance on a variety of workloads, begin careful to demonstrate what the limits of the system are.

  • Borrowing Remote Memory. As large as memories get, there always seems to be a program that needs more memory than you actually have. Virtual memory that swaps to disk is almost unusable now, because disks are so many orders of magnitude slower. However, accessing the memory of a nearby machine on the network might be done with a modest penalty. To address this problem, build a system that allows an ordinary program running on one machine access one large virtual memory assembled from multiple contributors. Start with the user level page table from the undergrad OS class, and build a user-level library that works with a remote memory donor. Implement multiple page replaceement algorithms, and evaluate them on several memory intensive applications. Compare the performance to disk-based virtual memory and real memory.

  • Distributed Mach. Read ahead to learn about the Mach microkernel operating system. Create a user-level library with the same basic concepts as Mach -- messages, ports, and tasks -- that allows for easy communication between processes, whether they run on the same machine, or different machines. Use your library to build up some simple operating system services or parallel applications. Explore the performance of this system, and compare it to using multiple processes on the same machine.

  • The Effect of SSDs on Modern Filesystems Set up a modern SSD and magnetic disk side by side on a single system. (Prof. Thain can provide you with the disks.) Compare the performance of basic reads and writes on the raw disks, to ensure that they behave as you expect. Then, install various filesystems on the two disks, and evaluate the performance of multiple workfloads. Do some filesystems do a better job of exploiting SSDs than others?

  • User Filesystem Study. Many assumptions about user behavior in operating systems are based on studies that are decades old. (example one, example two.) Produce a new study of how users behave in the ND CSE network. Examine tools such as strace, tcpdump, and fstrace for the purpose of recording logs of filesystem activity. Demonstrate that you can record and analyze a few hours of activity. Then, get permission from the CSE staff and a few of your friends to trace activity on a few workstations for several weeks. Write a comprehensive report on the file access behavior of those people over the semester.

  • Cloud Performance Evaluation. Cloud computing systems such as Amazon's EC2 provide additional computing capacity with the swipe of a credit card, however there is a performance penalty to be paid by traversing the network and employing virtualization. Explore the performance of of the coud by carefully measuring the performance of multiple applications ranging from CPU-bound to IO-bound on several categories of virtual machines. What is the cloud most effective at? (Amazon has generously supplied us with some credits on EC2 for this purpose -- see Dr. Thain to get a code for up to $100 of computing time on Amazon.

  • From Sequential to Scalable. Software engineering has long been the domain of sequential thinking. As we reach the end of Moore's law, it will become necessary to think in parallel for more and more applications. Take an existing application -- perhaps something relevant to your research -- and parallelize it using the Work Queue Framework, developed at Notre Dame. Using resources from the Condor Pool, CRC, or Amazon, try to scale the application up to hundreds or thousands of nodes, and explore the performance. Measure the behavior of the system with various workloads and under various failure conditions. Compare your solution to tradiitional methods of parallelization, such as multi-threading and MPI.