Checklist 01

Here is a general outline of the key concepts and commands (arranged by topic) that you should know for Exam 01.

The exam will have the following format:

  1. Commands: Identify which command performs a certain task.

  2. Short Answers: Briefly answers about the Shell, Files, Processes, Networking, Pipelines, and the Unix Philosophy.

  3. Debugging: Fix some simple shell scripts.

  4. Filters: Use Unix filters to answer some questions.

  5. Scripting: Write a shell script that utilizes command line arguments and filters.

Parts 1 through 3 is to be done first on paper. However, parts 4 and 5 can be done with the aid of your laptop and the Internet (but not other people).

Representative, but not Exhaustive

This check list is meant to be representative, rather than exhaustive (ie. there may be questions that show up on the exam that are not shown below).

Introduction

Concepts

Commands

  1. git

    Version control system for managing our files.

Files

Concepts

Commands

  1. ls

    List the contents of a directory (long, human-readable, sorted)

  2. chmod

    Set permissions using octal and symbolic arguments.

  3. stat

    View the inode information fora file.

  4. du

    View the total disk usage for a file or directory.

  5. ln

    Create hard and soft links.

  6. find

    Search directory for files by name and type.

Process

Concepts

Commands

  1. ps

    List the processes for the current user and for the whole system.

  2. top

    Interactively view and manage processes.

  3. kill

    Signal a process by PID.

  4. time

    Measure how long it takes a process to execute.

  5. pkill

    Signal a process by name.

I/O Redirection

Concepts

Commands

  1. tee

    Redirect stdin to both stdout and a file.

  2. write

    Write to another users terminal.

Networking

Concepts

Commands

  1. scp / sftp / rsync

    Transfer a file from one machine to another.

  2. ip / ifconfig

    List the IP addresses of the current machine.

  3. tmux / screen

    Maintain a persistent shell session.

  4. nc / telnet

    General purpose tools to communicate with TCP-based network services.

  5. wget / curl

    Communicate using HTTP (ie. download files from the web).

  6. ping

    Measure the latency from host to remote machine.

Shell Scripting

Concepts

  1. What is a shell script? What is a she-bang and why do we need it (or what does it do)?

  2. You should know the basics of the shell programming language:

    1. variables

    2. capturing STDOUT

    3. if statement

    4. case statement

    5. for loop

    6. while loop

    7. function

    8. trap

  3. How do you debug a shell script? What different modes do we have to help us figure out what a script is doing?

Commands

  1. sh

    Bourne shell interpreter.

  2. test

    Perform checks on file types and values.

  3. tar

    Extract or compress tarballs.

  4. unzip

    Extract zip archives.

Sample Questions

Debugging

The following scripts contain errors. Identify and correct all the bugs.

  1. This script checks each file passed via command line arguments and reports if the file is readable.

    for $file in '$@'; do
    [-r $file] | echo file is readable!
done

  2. This script checks each file passed via command line arguments and reports if the file is readable.

    is_readable() {
    test -r $@
}

while ( -n "$@" ); do
    is_readable($@) | echo file is readable!
done

Scripting

Write the following shell script:

$ ./column.sh -h
Usage: column.sh [-n N -d D -S -R] column

    -n N    Number of items to display (default is 5).
    -d D    Delimiter to use (default is ",").
    -S      Sort output.
    -R      Randomize output.

This script selects a column (default is 1) from the CSV input and then
displays the first N items.

Examples:

$ ./column.sh -d : < tmnt.txt
Leonardo
Donatello
Raphael
Michelangelo

$ ./column.sh -d : -n 1 < tmnt.txt
Leonardo

$ ./column.sh -d : -n 1 3 < tmnt.txt
katana

$ ./column.sh -d : -S 3 < tmnt.txt
bo
katana
nunchucks
sai

$ ./column.sh -d : -R 3 < tmnt.txt
nunchucks
katana
sai
bo

Note: The tmnt.txt file is described below.

Filters

Concepts

  1. What are the three tenets of the Unix Philosophy?

  2. What are regular expressions?

    You will need to know the basic metacharacters such as

    • .
    • *
    • []
    • [^]
    • ^
    • $
    • ()
    • |
    • \n
    • {m, n}

    You will also need to be familiar with the basic character classes and their character set equivalents such as:

    • [:alpha:] = [a-zA-Z]
    • [:lower:] = [a-z]
    • [:upper:] = [A-Z]
    • [:space:] = [ \t\r\n\v\f]
    • [:digit:] = [0-9]

  3. What are filters? What do they do and how are they related to the Unix Philosophy?

Commands

  1. cut

    Extract fields or characters.

  2. sort

    Order text.

  3. uniq

    Remove duplicates.

  4. shuf

    Randomize input.

  5. tr

    Translate from one set of characters to another.

  6. grep

    Search based on pattern.

  7. sed

    Search and replace or extract text based on regular expressions.

  8. head

    Display first few lines of input.

  9. tail

    Display last few lines of input.

  10. awk

    Perform complex manipulation of text.

Sample Questions

Given the file tmnt.txt, which contains:

Leonardo:blue:katana
Donatello:purple:bo
Raphael:red:sai
Michelangelo:orange:nunchucks

Pipelines

What is the result of the following shell commands?

  1. cut:

    1. echo "Leonardo,Donatello,Raphael,Michelangelo" | cut -d , -f 1

    2. echo "Leonardo,Donatello,Raphael,Michelangelo" | cut -d , -f 1,3

    3. echo "Leonardo,Donatello,Raphael,Michelangelo" | cut -d , -f 2-

    4. echo "Leonardo,Donatello,Raphael,Michelangelo" | cut -b 1,15,39

  2. tr:

    1. echo "Pizza! Pizza!" | tr '!' '?'

    2. echo "Pizza! Pizza!" | tr 'iaz' '145'

    3. echo "Pizza! Pizza!" | tr -d '[n-z]'

    4. echo "Pizza! Pizza!" | tr -d '[:punct:]'

  3. grep:

    1. cat tmnt.txt | grep '^[LM]'

    2. cat tmnt.txt | grep 'e.*o'

    3. cat tmnt.txt | grep -Eo '^[^: ]+'

    4. cat tmnt.txt | grep -Eo '[a-zA-Z]+[al]{2}[^:]*'

  4. sed:

    1. cat tmnt.txt | sed -nr 's/Leonardo/Splinter/p'

    2. cat tmnt.txt | sed -r '/lo/d'

    3. cat tmnt.txt | sed -nr 's|.*:o([^: ]+).*|O\1|p'

    4. cat tmnt.txt | sed -nr 's/.*([LD][^:]+)[: ]+([^:]+).*/\2/p'

Filters

Given the tmnt.txt file above, write Unix pipelines that perform the following tasks: