Finished Performance_Tracking for question 5. Had to make some changes to the code, because the code in zeppelin notebooks did not work. It was only looking at the master table and selecting yearID. yearID was not part of the master table , so i merged this with the fielding table.

Did performance measurement for question 4. Created ipython notebook for question 4 as well. Rob can edit the markdown for this.

Finished up Performance_Tracking for question 3b. Moved py files from the notebooks folder to the src folder. Renamed results to match the naming conventions for other visualizations. Created ipython notebooks for question 3b

1) Ran Code for questions 1-3 on pyspark in cluster mode, with multiple nodes. Measured and captured the difference in performance between running it on a single EC-2 instance, and running it on a cluster.

2) Added some screenshots for the final report, to show the cluster configuration.

3) Added ipython notebooks for performance metrics in local mode.

4) Added json files for zeppelin notebooks

5) Created new source files for Code pyspark code run in zeppelin notebooks, in cluster mode

6) Added test results for question 3 when using hive to calculate the median data.

7) Added R code from Rob for question 3 local exploration

8) Renamed some of the local exploration files

Finished spark implementation for question 1. Need to do some visualizations for this, and write the final report for it. Added iPython notebooks that detail the code for question 1. Formalized local pandas source, and remote pyspark source, and did a performance comparison.

Modified local programs to output csv data. Measured the amount of time each query took on a local pc. Started a document for Challenges_Encountered