The goal of the second homework assignment is to allow you to practice with IO redirection, managing processes, and utilizing network utilities. The activities in this assignment will require you apply the tools discussed in class to solve problems in creative ways.
For this assignment, record your responses to the following activities in the
README.md file in the homework02 folder of your assignments
Bitbucket repository and push your work by 11:59 PM Friday, January 29,
2016.
The first activity requires you to compare different methods of transferring
files between two student machines using scp, sftp, and rsync. To
complete this activity follow the steps below and then record your answers
the questions in your README.md:
In order to compare these three utilities, we must first setup a workspace and populate it with data files to transfer.
To do this, you should first create the folder /tmp/${USER}-workspace on
two different student machines: the first machine will be called the
source since it holds the original files and will send them to the
second machine which will be called the target.
Next, you should use the dd command to generate a 10MB file full of random data in the workspace on the source machine.
Hint: You can use the file /dev/urandom as the input source for
random data as explained on StackOverflow.
Finally, in the workspace with the 10MB file, you should create 10
hard links to the 10MB file.
Your source workspace should look something like this:
$ ls -l /tmp/${USER}-workspace
total 110M
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 10MB
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data0
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data1
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data2
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data3
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data4
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data5
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data6
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data7
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data8
-rw-r--r-- 11 pbui dip 10M Jan 24 13:14 data9
Now that we have our workspaces setup with data, we can now actually perform
the data transfers. For each of the three commands: scp, sftp, and
rsync, you are to transfer all of the data* files from the workspace on
the source machine to the workspace on the target machine using
one command (per utility).
scp: This should be straightforward if you use shell globbing to specify the file names to transfer.
sftp: To perform the file transfers in one command, you will need to create a text file containing the appropriate sftp operations and then use IO redirection to stream these instructions into the sftp program.
rsync: This should also be straightforward and similar to the scp command.
Use each of the three commands multiple times to practice transferring files between the source and target machine.
Once you have completed the steps above, answer the following questions in
your README.md:
What commands did you run to create your /tmp/${USER}-workspace
workspace and the data files? Provide a comment # Comment above each set
of commands to describe what you did. For example:
# Create workspace on source machine
commands
# Generate 10MB file full of random data
commands
# Create 10 hard links to 10MB file
commands
# Create workspace on target machine
commands
Run the du command in the /tmp/${USER}-workspace folder of the
source machine. What is the total disk usage? Is this number
surprising? Why or why not?
After you have performed the file transfers, run the du command in the
/tmp/${USER}-workspace folder of the target machine. What is the total
disk usage? Is this number different from the one above? Why or why not?
Describe what commands you used for each of the three utilities above to perform the file transfers. For example:
# Transfer data files using scp
scp ...
# Transfer data files using sftp
sftp ...
# Transfer data files using rsync
rsync
For sftp also include the contents of the text file you created to automate its operations.
When using scp and sftp multiple times, how often is each data file transferred? When using rsync multiple times, how often is each data file transferred? How is this feature of rsync useful?
Of the three commands, which one do you prefer and why?
When you are done with your experiments, please remove the workspace directories from both the source and the target machines. Otherwise, the machines will run out of space.
rm -fr /tmp/${USER}-workspace
The second activity requires you to go down the rabbit hole:
... in order to talk to the Oracle. Of course, access to the Oracle is heavily restricted, so you will need to use your hacking skills to locate her and gain an audience with her. Once you connect with her, she will hopefully give you a message about how to use your awesome Unix skills.
To begin your journey, you will have to find the entrance to her sanctuary.
Thanks to a glitch in the Matrix we have been able to narrow her location
to the machine xavier.h4x0r.space. Scan that machine for a HTTP port in
the 9000 - 10000 range and then proceed down the path of discovery and
enlightenment.
As you travel to the Oracle, record what you saw and what commands you
executed in your README.md. Be thorough as possible, as we will need this
log to reconstruct your path and hopefully use it for future meetings. Here
is an example of what your README.md for this activity should look like:
1. Scan `xavier.h4x0r.space` for HTTP port:
$ command ...
Output...
As you can see here, there is are X ports in the 9000 - 10000 range.
To check if the port was a HTTP server, I next used the X command...
2. Access HTTP server:
$ command ...
Output...
...
This activity requires that you work on the student machines or have access to AFS on your Unix machine.
The hints may spoil the surprises in the journey, so only view them when you are stuck: Toggle Hints.
To scan xavier.h4x0r.space for open ports, you can use the
nmap utility.
To decode your passcode, you will need to use base64.
To contact the DOORMAN, you can use either curl or wget once again.
To signal the SLEEPER, you can use either user pkill or kill.
To finally talk to the ORACLE, you can either use telnet or nc.
For extra credit, you are to setup SSH Keys and configure your Bitbucket account to utilize them. The following are resources on how to set this up:
Basically, with SSH Keys properly configured, you can do things such as push and pull to Bitbucket without having to enter your password everytime and login to remote Unix machines (such as your VM!) without a password. This is handy for automating tasks where no human interaction is desirable.
To get credit, you must show either a TA or the instructor a demonstration of either pushing to your repository or pulling from it using SSH.
If you have any questions, comments, or concerns regarding the course, please
provide your feedback at the end of your README.md.
To submit your assignment, please commit your work to the homework02 folder
in your assignments Bitbucket repository by 11:59 PM Friday, January
29, 2016.