Reading 06: Processing Data

Everyone:

Next week, we will explore a new programming paradigm called functional programming, which, if you look closely, is similar to the Unix Philosophy: By decomposing data processing problems into small functions that work together to process data without side-effects, we can expose opportunities for concurrency and parallelism to build powerful abstractions such as MapReduce.

TL;DR

The focus of this reading is to introduce you to functional programming in Python and MapReduce.

Readings

The readings for Monday, February 26 are:

Optional Resources

Here are some additional resources:

Quiz

This week, the reading is split into two sections: the first part is a short dredd quiz, while the second part involves three short Python scripts: evens_fp.py, evens_lc.py, and evens_gr.py.

To test these scripts, you will need to download the Makefile and test scripts:

$ git checkout master                 # Make sure we are in master branch
$ git pull --rebase                   # Make sure we are up-to-date with GitLab

$ git checkout -b reading06           # Create reading06 branch and check it out

$ cd reading06                        # Go into reading06 folder

# Download Reading 06 Makefile
$ curl -LO https://gitlab.com/nd-cse-20289-sp18/cse-20289-sp18-assignments/raw/master/reading06/Makefile

# Execute tests (and download them)
$ make

Questions

Record the answers to the following Reading 06 Quiz questions in your reading06 branch:

Scripts

Given the following Python script, evens.py:

#!/usr/bin/env python3

import sys

results = []
for number in sys.stdin:
    number = number.strip()
    if int(number) % 2 == 0:
        results.append(number)

print(' '.join(results))

Create a second version of this script called evens_fp.py that re-implements the original evens.py script in one line using map, filter, and lambda:
```
#!/usr/bin/env python3

import sys

print(' '.join(
    # TODO: One-line expression with map, filter, lambda
))
```
Create a third version of this script called evens_lc.py that re-implements the original evens.py script in one line using list comprehensions:
```
#!/usr/bin/env python3

import sys

print(' '.join(
    # TODO: One-line expression with list comprehension
))
```

Create a fourth version of this script called evens_gr.py that re-implements the original evens.py script by creating a generator function using the yield keyword.

#!/usr/bin/env python3

import sys

def evens(stream):
    # TODO: Implementation that uses yield statement

print(' '.join(evens(sys.stdin)))

To test your scripts manually, you should be able to reproduce the following output:

$ seq 1 10 | ./evens_fp.py
2 4 6 8 10

$ seq 1 10 | ./evens_lc.py
2 4 6 8 10

$ seq 1 10 | ./evens_gr.py
2 4 6 8 10

Submission

To submit you work, follow the same process outlined in Reading 01:

$ git checkout master                 # Make sure we are in master branch
$ git pull --rebase                   # Make sure we are up-to-date with GitLab

$ git checkout -b reading06           # Create reading06 branch and check it out

$ cd reading06                        # Go into reading06 folder

$ $EDITOR answers.json                # Edit your answers.json file

$ ../.scripts/submit.py               # Check reading06 quiz
Submitting reading06 assignment ...
Submitting reading06 quiz ...
      Q1 0.80
      Q2 0.20
   Score 1.00

$ git add answers.json                # Add answers.json to staging area
$ git commit -m "Reading 06: Quiz"    # Commit work

$ $EDITOR evens_fp.py                 # Edit your evens_fp.py file
$ $EDITOR evens_lc.py                 # Edit your evens_lc.py file
$ $EDITOR evens_gr.py                 # Edit your evens_gr.py file

$ make                                # Test all scripts
Testing evens_fp.py ...
 evens_fp.py on seq 1 10                  ... Success
 evens_fp.py on seq 10 20                 ... Success
 evens_fp.py on seq 1 1000000 (count)     ... Success
   Score 1.00

Testing evens_lc.py ...
 evens_lc.py on seq 1 10                  ... Success
 evens_lc.py on seq 10 20                 ... Success
 evens_lc.py on seq 1 1000000 (count)     ... Success
   Score 1.00

Testing evens_gr.py ...
 evens_gr.py on seq 1 10                  ... Success
 evens_gr.py on seq 10 20                 ... Success
 evens_gr.py on seq 1 1000000 (count)     ... Success
   Score 1.00

$ git add Makefile                    # Add Makefile to staging area
$ git add evens_fp.py                 # Add evens_fp.py to staging area
$ git add evens_lc.py                 # Add evens_lc.py to staging area
$ git add evens_gr.py                 # Add evens_gr.py to staging area
$ git commit -m "Reading 06: Scripts" # Commit work

$ git push -u origin reading06        # Push branch to GitLab

Remember to create a merge request and assign the appropriate TA from the Reading 06 TA List.