Everyone:

Next week, we will continue to explore shell scripting. This time, however, we will unlock the true power of the Unix philosophy by studying sophisticated pipelines for filtering text. Additionally, we will also learn to wield regular expressions to create nifty pattern matching tools.

TL;DR

The focus of this reading is to introduce regular expressions and revisit filters and pipelines.

Readings

The readings for this week are:

  1. The Linux Command Line:

    • Chapter 19 - Regular Expressions
    • Chapter 20 - Text Processing
  2. RegexOne

    Work through the Lessons in the Interactive Tutorial. You can use the Regexpr to visualize your regular expressions.

Optional Resources

Quiz

This week, your reading quiz is split into two sections: the first part is your normal dredd quiz, while the second part involves a series of pipelines.

Questions

Record the answers to the following Reading 03 Quiz questions in your reading03 branch:

Filters

For the second part, you are to complete the following pipelines:

  1. Convert all the input text to upper case:

    $ echo "All your base are belong to us" | ...
    ALL YOUR BASE ARE BELONG TO US
    
  2. Find and replace all instances of monkeys to gorillaz:

    $ echo "monkeys love bananas" | ...
    gorillaz love bananas
    
  3. Remove any leading whitespace from a string of text:

    $ echo "     monkeys love bananas" | ...
    monkeys love bananas
    
  4. Parse the /etc/passwd file for the shell of the root user:

    $ curl -sLk https://yld.me/raw/yWh | ...
    /bin/bash
    

    Hint: You may need to read up on the format of /etc/passwd

  5. Find and replace all instances of /bin/bash, /bin/csh, and /bin/tcsh to /usr/bin/python in /etc/passwd:

    $ curl -sLk https://yld.me/raw/yWh | ... | grep python
    root:x:0:0:root:/root:/usr/bin/python
    mysql:x:27:27:MySQL Server:/var/lib/mysql:/usr/bin/python
    xguest:x:500:501:Guest:/home/xguest:/usr/bin/python
    condor:x:108172:40:Condor Batch System:/afs/nd.edu/user37/condor:/usr/bin/python
    lukew:x:522:40:Luke Westby temp access:/var/tmp/lukew:/usr/bin/python
    
  6. Find all the records in /etc/passwd that have a number that begins with a 4 and ends with a 7:

    $ curl -sLk https://yld.me/raw/yWh | ...
    rtkit:x:499:497:RealtimeKit:/proc:/sbin/nologin
    qpidd:x:497:495:Owner of Qpidd Daemons:/var/lib/qpidd:/sbin/nologin
    uuidd:x:495:487:UUID generator helper daemon:/var/lib/libuuid:/sbin/nologin
    mailnull:x:47:47::/var/spool/mqueue:/sbin/nologin
    

Template

To record your pipelines, you will need to write your answers to the filters.sh script:

# Download filters template
$ curl -LOk https://gitlab.com/nd-cse-20289-sp19/cse-20289-sp19-assignments/raw/master/reading03/filters.sh

Each question has a corresponding function; for instance, the solution for Question 1 should be recorded in q1_answer:

#!/bin/bash

q1_answer() {
    # TODO: Complete pipeline
    echo "All your base are belong to us" | ...
}

To construct your pipelines, you should try them interactively in your shell. Once you found a reasonable solution, you can record them in the filters.sh script under the appropriate function.

Makefile

To test your pipelines, you need to download the Reading 03 Makefile, which in turn will download the test_filters.sh test script and execute it:

# Download Makefile
$ curl -LOk https://gitlab.com/nd-cse-20289-sp19/cse-20289-sp19-assignments/raw/master/reading03/Makefile

# Download test scripts and run test
$ make
...

# Test your filters
$ ./test_filters.sh
Testing filters.sh ...
     Q1 Success
     Q2 Success
     Q3 Success
     Q4 Success
     Q5 Success
     Q6 Success
  Score 3.00

Submission

To submit you work, follow the same process outlined in Reading 01:

$ git checkout master                 # Make sure we are in master branch
$ git pull --rebase                   # Make sure we are up-to-date with GitLab

$ git checkout -b reading03           # Create reading03 branch and check it out

$ cd reading03                        # Go into reading03 folder
$ $EDITOR answers.json                # Edit your answers.json file

$ ../.scripts/submit.py               # Check reading03 quiz
Submitting reading03 assignment ...
Submitting reading03 quiz ...
  Score 1.00

$ git add answers.json                # Add answers.json to staging area
$ git commit -m "Reading 03: Quiz"    # Commit work

$ $EDITOR filters.sh                  # Edit your filters.sh file

$ ./test_filters.sh                   # Test your filters
Testing filters.sh ...
     Q1 Success
     Q2 Success
     Q3 Success
     Q4 Success
     Q5 Success
     Q6 Success
  Score 3.00

$ git add Makefile                    # Add Makefile to staging area
$ git add filters.sh                  # Add filters.sh to staging area
$ git commit -m "Reading 03: Filters" # Commit work

$ git push -u origin reading03        # Push branch to GitLab

Remember to create a merge request and assign the appropriate TA from the Reading 03 TA List.