Project 01: Idlebin

Overview

The first project is to build your own implementation of Busybox, which according to its website is the following:

BusyBox combines tiny versions of many common UNIX utilities into a single small executable. It provides replacements for most of the utilities you usually find in GNU fileutils, shellutils, etc.

You may think of Busybox as a complete Unix userland in a single binary executable. Because of its compact size, it is found in many embedded devices such as printers, network routers, and even some phones. In fact, Alpine Linux, the official Linux distribution for Docker, uses Busybox as its default userland rather than the traditional GNU utilities found on most Linux machines. This means commands such as ls and cat are embedded inside of a single busybox executable rather than as separate programs.

Given the Busybox executable, busybox, you can invoke any of its internal commands (i.e. applets) by specifying the desired applet as the first argument to the program:

$ busybox ls -l     # Execute ls applet with the argument -l

Alteratively, you can symlink the busybox executable to the name of any applet and then execute the applet directly:

$ ln -s busybox ls  # Create symbolic link from ls to busybox

$ ./ls -l           # Execute ls applet with the argument -l

Working individually, you are to utilize low-level system calls to create your own version of Busybox called idlebin that supports multiple internal applets by noon on Saturday, September 14, 2019.

More details about this project and your deliverables are described below.

Idlebin

For this project, you are to implement various Unix utilities as applets in a single binary executable called idlebin. Specifically, you must implement a clone of cp, and then optionally implement one or two additional applets for extra credit.

Provided Applets

You are provided with a working implementation of idlebin that contains the following applets:

You can utilize the provided applets by calling idlebin as follows:

$ bin/idlebin ls --help     # Display ls applet's usage message
Usage: ls [-Ra] [PATHS]...
    -R    List subdirectories recursively
    -a    Do not ignore entries starting with .

Likewise, the provided Makefile will automatically create symlinks for each individual applet, so you can execute the provided ls applet as follows:

$ bin/ls --help             # Display ls applet's usage message
Usage: ls [-Ra] [PATHS]...
    -R    List subdirectories recursively
    -a    Do not ignore entries starting with .

As with Busybox, the idlebin applets only implement a subset of the functionality (i.e. command-line options) of their traditional GNU counterparts. For instance, the provided ls only supports the -R and -a flags, which allow it to list subdirectories recursively or to display all files, even those that begin with a ..

Required Applet

For this project, you will need to implement a clone of the cp command that supports the following usage:

$ bin/idlebin cp --help     # Display cp applet's usage message
Usage: cp [-Rv] SOURCE... TARGET
    -R    Copy directories recursively
    -v    Explain what is being done

To implement this applet, you will need to review and then utilize some of the following system calls and low-level functions: stat, open, read, write, close, readlink, symlink, opendir, readdir, closedir, and mkdir.

The behavior (in terms of operation and output) of the cp applet must match that of both the GNU version and the reference implementation provided below.

Note: In terms of the output of cp in verbose mode, you should build your version to match the output of the reference implementation when it deviates from the GNU behavior. The provided unit test will check that you match the reference implementation.

Notes

1. The work of each individual applet must go in a separate git branch that can be later merged into the master branch (ie. make a applet-cp branch, DO NOT COMMIT OR MERGE TO MASTER).

2. Think carefully about all the different use cases (ie. copy one file to a directory, copy multiple files to a directory, etc.) and how you would detect these situations.

3. When possible create small helper functions that perform one specific action.

4. Develop your applet incrementally by supporting one feature or use case at a time.

Optional Applets

Once you have implemented cp and have passed all the provided unit tests, you may program one or two more applets for extra credit.

Here is a table of possible applets to optionally implement:

Notes

1. The work of each individual applet must go in a separate git branch that can be later merged into the master branch (ie. make a applet-touch branch, DO NOT COMMIT OR MERGE TO MASTER).

2. Consult the appropriate manual pages and the GNU versions of the commands to determine the correct behavior and output your applet requires. In terms of the output, you should build to match the expectation of the unit test when it deviates from the GNU behavior.

3. To verify the correctness of your individual applet, you will need to write a [unit test] script in Python that provides a reasonable amount of test coverage for the features you implemented.

Reference Implementation

You can find a reference implementation of idlebin in the following location on the student machines:

/escnfs/home/pbui/pub/bin/idlebin

This will be updated periodically as more applets are written and added to the reference implementation.

Deliverables

As noted above, you are to work individually to implement your idlebin. You must use C99 (not C++) as the implementation language for idlebin itself, while the test scripts are to be written in Python.

Timeline

Here is a timeline of events related to this project:

Repository

To start this project, you must fork the Project 01 repository on GitLab:

https://gitlab.com/nd-cse-30341-fa19/cse-30341-fa19-project01

Once this repository has been forked, follow the instructions from Reading 00 to:

1. Make the repository private.

2. Configure access to the repository.

   Make sure you add all the members of the team in addition to the instructional staff.

Documentation

The Project 01 repository includes a README.md file with the following sections:

1. Student: This should be your name and email address.

2. Brainstorming: This should contain your responses to the provided brainstorming questions.

   To help you get started with the project, the README.md file provides you with a series of brainstorming questions that you are designed to guide you towards thinking about the different design and implementation aspects of your project. That is, the best way to get started with this project is by reading through the code base and then answering these questions.

   Ideally, these questions would be completed before you begin coding, but you are only required to submit your responses at the end of the project.

3. Applets: This contains the list of applets you are to implement. You should check off any that are completed (ie. [x]) and include any notes such as if there are any known issues or bugs.

4. Reflection: This should contain your responses to the provided reflection questions.

   To prepare you for future exams, the README.md file also contains a series of reflection questions that you are to complete after you have finished programming and testing your project. These questions are designed to help you connect the work you did in this project with the concepts discussed in class. It is quite possible that questions related to these reflections will appear on a future exam.

Source Code

In addition to the README.md file, the Project 01 repository contains the following folder hierarchy:

project01
    \_  Makefile        # This is the project Makefile
    \_  bin             # This contains the executable and symlinks
    \_  include         # This contains the header files
    \_  src             # This contains the C99 source code for idlebin executable
        \_ applets      # This conatins the C99 source code for idlbein applets
    \_  tests           # This contains the Python unit tests

You must maintain this folder structure for your project and place files in their appropriate place.

Compiling

To help you get started, we have provided a Makefile that will build the idlebin application for you:

$ make                  # Build idlebin executable and create applet symlinks
Compiling src/applets/basename.o
Compiling src/applets/false.o
Compiling src/applets/pwd.o
Compiling src/applets/ls.o
Compiling src/applets/cat.o
Compiling src/applets/yes.o
Compiling src/applets/true.o
Linking bin/idlebin
Symlinking bin/basename
Symlinking bin/false
Symlinking bin/pwd
Symlinking bin/ls
Symlinking bin/cat
Symlinking bin/yes
Symlinking bin/true

Running

Once you have the idlebin executable, you can invoke the various applets in the same way that you do with Busybox:

$ bin/idlebin           # List all applets
Usage: idlebin applet [ARGUMENTS]...
Applets:
    basename
    false
    pwd
    ls
    cat
    yes
    true

$ bin/idlebin pwd       # Execute pwd applet
/home/pbui/src/teaching/cse.30341.fa19/project01

$ bin/pwd               # Execute pwd applet
/home/pbui/src/teaching/cse.30341.fa19/project01

$ bin/pwd --help        # Display pwd applet usage
Usage: pwd

Extending

You can create additional applets to be embedded into idlebin by placing C99 source files in the src/applets folder and defining at least the following two items:

1. {APPLET}_USAGE: This must be a const char array that contains the usage message for the new APPLET you are creating; it may consist of multiple lines.

2. {APPLET}_applet: This is a function that receives the command-line arguments (int argc, char *argv[]), performs the desired operations, and then returns an int corresponding to the applet's exit status.

Here is a simple template to follow:

/* {APPLET}.c: {APPLET} applet */

#include "idlebin.h"

/* Usage */

const char {APPLET}_USAGE[] = "Usage: {APPLET}";

/* Applet */

int {APPLET}_applet(int argc, char *argv[]) {
    return EXIT_SUCCESS;
}

The provided Makefile will automatically detect any additional applets placed in the src/applets folder and will embed them into the idlebin executable for you.

Testing

To verify the correctness of the applets in idlebin, we have provided a series of unit tests implemented in Python. These are located in the tests directory and can be executed all at once with the following command:

$ make test                 # Run all test scripts
Testing basename...
test_00_help (test_basename.BasenameTestCase) ... ok
test_01_usage (test_basename.BasenameTestCase) ... ok
test_02_basename (test_basename.BasenameTestCase) ... ok
...
----------------------------------------------------------------------
Ran 81 tests in 17.488s

FAILED (failures=25, errors=10)
make: *** [Makefile:54: test] Error 1

Note, the first time you run make test you will have failures since the ls and cp applets have not been implemented yet.

You may execute all the unit tests of a specific command by running the test Python script directly:

$ ./tests/test_pwd.py -v    # Run pwd test script
Testing pwd...
test_00_help (__main__.PwdTestCase) ... ok
test_01_curdir (__main__.PwdTestCase) ... ok
test_01_curdir_valgrind (__main__.PwdTestCase) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.570s

OK

To add your own tests, simply create the corresponding test_{APPLET}.py Python Script in the tests folder. It is recommended that you model your unit tests after the existing ones, which means utilizing Python's unittest module for creating unit tests.

Rubric

Your project will be scored on the following metrics: