Reading 05: Visualizing Data

Everyone:

Next week, we will further explore data structures in Python. In particular, we will focus on list sequence containers and dict associative (ie. mapping) containers and utilize them to aggregate and process data. Additionally, we will utilize Jupyter notebooks and matplotlib to visualize the data we processed using our data structures.

TL;DR

The focus of this reading is to practice scripting in Python and utilize the list and dict data structures.

Readings

The readings for Monday, February 19 are:

A Byte of Python - Data Structures

Optional Resources

Here are some additional resources:

Automate The Boring Stuff
- Chapter 4 - Lists
- Chapter 5 - Dictionaries and Structuring Data

Jupyter Notebooks

As mentioned above, we will utilize Jupyter notebooks next week as an environment for coding in Python and generating matplotlib figures. If you wish to get a taste of this awesomeness, run the following command on a student machine:

$ jupyter notebook --ip studentXX.cse.nd.edu --port 9000 --no-browser

Of course, replace studentXX.cse.nd.edu with whatever student machine you are using. Once this is executed, it will give you a URL that looks something like this:

Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
    http://student00.cse.nd.edu:9000/?token=0e97be5f0cdf8aeb6512d005e47e329605bc1b2fc9365689

Simply copy and paste that URL into your local web browser. You know can create a notebook and interactively write and execute Python code via your web browser.

Quiz

This week, there is no dredd quiz. Instead, you will need to write three Python scripts: sort.py, cut.py, and uniq.py.

To test these scripts, you will need to download the Makefile and test scripts:

$ git checkout master                 # Make sure we are in master branch
$ git pull --rebase                   # Make sure we are up-to-date with GitLab

$ git checkout -b reading05           # Create reading05 branch and check it out

$ cd reading05                        # Go into reading05 folder

# Download Reading 05 Makefile
$ curl -LO https://gitlab.com/nd-cse-20289-sp18/cse-20289-sp18-assignments/raw/master/reading05/Makefile

# Execute tests (and download them)
$ make

`sort.py`

Use this cat.py as the basis for a Python script called sort.py, which is a simple implementation of the sort command that supports the flags shown below:

# Print Usage
$ ./sort.py
Usage: sort.py files...

    -r  reverse the result of comparisons

# Read from STDIN and reverse results
$ seq 1 10 | ./sort.py -r
9
8
7
6
5
4
3
2
10
1

Hints

Consider using the sorted function with the reverse keyword argument.

`cut.py`

Use this cat.py as the basis for a Python script called cut.py, which is a simple implementation of the cut command that supports the flags shown below:

# Print Usage
$ ./cut.py -h
Usage: cut.py files...

    -d  DELIM   use DELIM instead of TAB for field delimiter
    -f  FIELDS  select only these fields

# Read from file and show first field delimited by :
$ ./cut.py -d : -f 1 /etc/passwd
root
bin
...
unbound
setroubleshoot

Note: cut.py only needs to support comma-separated values for the FIELDS parameter (ie. 1 or 1,2, etc.).

Hints

Consider using sorted and set to ensure the FIELDS are unique and in order.
Consider using the str.split method to separate the different FIELDS into a list.
Consider using the str.join method to combine a list of strings into a single string.

`uniq.py`

Use this cat.py as the basis for a Python script called uniq.py, which is a simple implementation of the uniq command that supports the flags shown below:

# Print Usage
$ ./uniq.py -h
Usage: uniq.py files...

    -c  prefix lines by the number of occurrences

# Read from STDIN and display counts of each unique user
$ who | awk '{print $1}' | sort | ./uniq.py -c
      2 akeenan2
      1 bblum1
      ...
      2 yoh1

Hints

Consider using a dict to count the distinct items.
Consider using the dict.get method to increment the count for a particular item.
Consider using the dict.items method to iterate over the counts.
Consider using string formatting to get the display correct.

Submission

To submit you work, follow the same process outlined in Reading 01:

$ git checkout master                 # Make sure we are in master branch
$ git pull --rebase                   # Make sure we are up-to-date with GitLab

$ git checkout -b reading05           # Create reading05 branch and check it out

$ cd reading05                        # Go into reading05 folder

$ $EDITOR sort.py                     # Edit your sort.py file
$ ./test_sort.sh                      # Test your sort.py
Testing sort.py ...
 Usage                                    ... Success
 sort    on /etc/group                    ... Success
 sort -r on /etc/group                    ... Success
 sort    on /etc/passwd                   ... Success
 sort -r on /etc/passwd                   ... Success
 sort    on stdin (implicit)              ... Success
 sort -r on stdin (implicit)              ... Success
 sort    on stdin (explicit)              ... Success
 sort -r on stdin (explicit)              ... Success
   Score 1.00

$ $EDITOR cut.py                      # Edit your cut.py file
$ ./test_cut.sh                       # Test your cut.py
Testing cut.py ...
 Usage                                    ... Success
 cut -d : -f 1   on /etc/passwd           ... Success
 cut -d : -f 1,1 on /etc/passwd           ... Success
 cut -d : -f 1,1 on /etc/passwd           ... Success
 cut -d : -f 1,1 on /etc/passwd           ... Success
 cut -d : -f 2   on /etc/passwd           ... Success
 cut -d : -f 1,2 on /etc/passwd           ... Success
 cut -d : -f 2,1 on /etc/passwd           ... Success
 cut -d : -f 2,2 on /etc/passwd           ... Success
 cut -d : -f 3   on /etc/passwd           ... Success
 cut -d : -f 1,3 on /etc/passwd           ... Success
 cut -d : -f 3,1 on /etc/passwd           ... Success
 cut -d : -f 3,3 on /etc/passwd           ... Success
 cut -d : -f 4   on /etc/passwd           ... Success
 cut -d : -f 1,4 on /etc/passwd           ... Success
 cut -d : -f 4,1 on /etc/passwd           ... Success
 cut -d : -f 4,4 on /etc/passwd           ... Success
 cut -d : -f 5   on /etc/passwd           ... Success
 cut -d : -f 1,5 on /etc/passwd           ... Success
 cut -d : -f 5,1 on /etc/passwd           ... Success
 cut -d : -f 5,5 on /etc/passwd           ... Success
 cut -d : -f 6   on /etc/passwd           ... Success
 cut -d : -f 1,6 on /etc/passwd           ... Success
 cut -d : -f 6,1 on /etc/passwd           ... Success
 cut -d : -f 6,6 on /etc/passwd           ... Success
 cut -d : -f 1   on stdin (implicit)      ... Success
 cut -d : -f 1   on stdin (explicit)      ... Success
   Score 1.50

$ $EDITOR uniq.py                      # Edit your uniq.py file
$ ./test_uniq.sh                       # Test your uniq.py
Testing uniq.py ...
 Usage                                    ... Success
 uniq    on /etc/group                    ... Success
 uniq -c on /etc/group                    ... Success
 uniq    on /etc/passwd                   ... Success
 uniq -c on /etc/passwd                   ... Success
 uniq    on /etc/fstab                    ... Success
 uniq -c on /etc/fstab                    ... Success
 uniq    on stdin (explicit)              ... Success
 uniq -c on stdin (explicit)              ... Success
   Score 1.50

$ git add Makefile                    # Add Makefile to staging area
$ git add sort.py                     # Add sort.py to staging area
$ git add cut.py                      # Add cut.py to staging area
$ git add uniq.py                     # Add uniq.py to staging area
$ git commit -m "Reading 05: Scripts" # Commit work

$ git push -u origin reading05        # Push branch to GitLab

Remember to create a merge request and assign the appropriate TA from the Reading 05 TA List.