Project 5: Virtual Memory

  • Frequently Asked Questions
  • General Lab Instructions
  • The goals of this project are:
  • to demonstrate mastery of virtual memory concept.
  • to learn the code mechanics of operating system fault handlers.
  • to develop skills in quantitative system evaluation.
  • Project Overview

    In this project, you will build a simple but fully functional demand paged virtual memory. Although virtual memory is normally implemented in the operating system kernel, it can also be implemented at the user level. This is exactly the technique used by modern virtual machines, so you will be learning an advanced technique without having the headache of writing kernel-level code. The following figure gives an overview of the components:
    We will provide you with code that implements a "virtual" page table and a "virtual" disk. The page table will create a corresponding small virtual and physical memory, along with methods for updating the page table entries and protection bits. When an application uses the virtual memory, it will result in page faults that call a custom handler. Most of your job is to implement a page fault handler that will trap page faults and identify the correct course of action, which generally means updating the page table, and moving data back and forth between the disk and physical memory.

    Once your system is working correctly, you will evaluate the performance of several page replacement algorithms on a selection of simple programs across a range of memory sizes. You will write a short lab report that explains the experiments, describes your results, and draws conclusions about the behavior of each algorithm.

    This project is split into two parts. Part 1 is to write the virtual memory code, and is due on Friday, April 17th. Part 2 is to conduct a performance evaluation of your code and write a lab report, due on Wednesday, April 29th.

    Getting Started

    Clone the starter code for this assignment and build it:
    git clone https://github.com/dthain/cse30341-spring-2020
    cd cse30341-spring-2020
    cd project5
    make
    
    (Note that the page table code does some very Linux specific magic, and so you should be sure to run it on the CSE student machines.)

    Look through main.c and notice that the program simply creates a virtual disk and page table, and then attempts to run one of our three "programs" using the virtual memory. Because no mapping has been made between virtual and physical memory, a page fault happens immediately:

    % ./virtmem 100 10 rand alpha
    page fault on page #0
    
    The program exits because the page fault handler isn't written yet. That is your job!

    Try this as a getting started exercise. If you run the program with an equal number of pages and frames, then the disk is completely unneeded: physical memory is large enough to accommodate every page. In that case, just make page N map directly to frame N, and do nothing else. So, modify the page fault handler to do that:

    page_table_set_entry(pt,page,page,PROT_READ|PROT_WRITE);
    
    With that trivial page fault handler, all of the example programs will run, cause a number of page faults, but then run to completion. (And will also tell you the "correct" result computed for that program.)

    Congratulations, you have written your first page fault handler!

    Of course, when there are fewer frames than pages, then this simple scheme will not do. Instead, we must keep recently used pages in the physical memory, other pages on disk, and update the page table appropriately as they move back and forth.

    Example Operation

    The virtual page table is very similar to what we have discussed in class, except that it does not have a reference or dirty bit for each page. The system supports a read bit (PROT_READ), a write bit (PROT_WRITE), and an execute bit (PROT_EXEC), which is enough to make it work.

    Let's work through an example, starting with the figure at the right. Suppose that we begin with nothing in physical memory. If the application begins by trying to read page 2, this will result in a page fault. The page fault handler choose a free frame, say frame 3. It then adjusts the page table to map page 2 to frame 3, with read permissions. Then, it loads page 2 from disk into page 3. When the page fault handler completes, the read operation is re-attempted, and succeeds.
    The application continues to run, reading various pages. Suppose that it reads pages 3, 5, 6, and 7, each of which result in a page fault, and must be loaded into memory as before. Now physical memory is fully in use.
    Now suppose that the application attempts to write to page 5. Because this page only has the R bit set, a page fault will result. The page fault handler looks at the current page bits, and upon seeing that it already has the PROT_READ bit set, adds the PROT_WRITE bit. The page fault handler returns, and the application can continue. Page 5, frame 1 is modified.
    Now suppose that the application reads page 1. Page 1 is not currently paged into physical memory. The page fault handler must decide which frame to remove. Suppose that it picks page 5, frame 0 at random. Because page 5 has the PROT_WRITE bit set, we know that it is dirty. So, the page fault handler writes page 5 back to the disk, and reads page 1 in its place. Two entries in the page table are updated to reflect the new situation.

    Part 1: Write the Code

    Please review the general instructions for class assignments. You must complete the program so that it can be invoked in this way:
    ./virtmem npages nframes rand|fifo|custom alpha|beta|gamma|delta
    
    npages is the number of pages and nframes is the number of frames to create in the system. The third argument is the page replacement algorithm. You must implement rand (random replacement), fifo (first-in-first-out), and custom, an algorithm of your own invention. The final argument specifies which built-in program to run: alpha, beta, gamma, or delta. Each manipulates the virtual memory with a different pattern of access.

    You may only modify the file main.c. Your job is to implement three page replacement algorithms. rand and fifo should be implemented as discussed in class. You should invent a third algorithm, custom that performs better than rand or fifo for at least some of the four programs. "Better" in this case means that it results in fewer disk reads and writes.

    A complete and correct program will run each of the sample programs to completion with only the following output:

  • The single line of output from alpha, beta, gamma or delta, just to confirm that the program ran correctly.
  • A summary of the number of page faults, disk reads, and disk writes over the course of the program.
  • You may certainly add some printfs while testing and debugging your program, but the final version should not have any extraneous output.

    Part 1 is due at 11:59PM EST on Friday, April 17th. If you have unusual circumstances that would prevent you from meeting that deadline, please contact Prof. Thain.

    Your dropbox directory is /escnfs/courses/sp20-cse-30341.01/dropbox/YOURNAME. Make a directory called project5 there and copy your source files and Makefile into that directory. (Please don't turn in any executables, virtual disks, or other large files.)

    Part 2: Evaluate Performance

    Once the code is complete, the second part of the assignment is more like a physics lab. You will conduct a quantitative evaluation of each of the algorithms, and explain the nature of your results.

    Write up a lab report document that has the following elements:

    1. In your own words, briefly explain the purpose of the experiments and the experimental setup. Be sure to clearly state on which machine you ran the experiments, and exactly what your command line arguments were, so that we can reproduce your work in case of any confusion.
    2. Very carefully describe the custom page replacement algorithm that you have invented. Make sure to give enough detail that someone else could reproduce your algorithm, even without your code. A flow chart or some pseudo-code would be very helpful in understanding exactly what you did.
    3. Measure and graph the number of page faults, disk reads, and disk writes for each program and each page replacement algorithm using 100 pages and a varying number of frames between 3 and 100. Spend some time to make sure that your graphs are nicely laid out, correctly labelled, and easy to read.
    4. Explain the nature of the results. It's likely that no single algorithm performs the best in all situations. If one algorithm performs better than another under certain conditions, then point that out, explain the conditions, and explain why it performs better. You might find it helpful to look at the source code of the functions alpha, beta, gamma, and delta to get a sense of what they are doing.
    Part 2 is due on Wednesday, April 29th at 11:59PM. If you have unusual circumstances that would prevent you from meeting that deadline, please contact Prof. Thain.

    Please submit a file called report.pdf into the same dropbox directory for the Project 5 code.

    Grading

    Your grade on this assignment will be based on the following:
  • Correct implementation of demand paging with any arbitrary access pattern and amount of virtual and physical memory. (50%)
  • A lab report which is clearly written using correct English, contains an appropriate description of your experiments, contains correct results that are clearly presented, and draws appropriate conclusions. (30%)
  • Thorough attention to and handling of all possible error conditions, including user error. (10%)
  • Good coding style, including clear formatting, sensible variable names, and useful comments. (10%)