- Apr 29, 2018
-
-
Nischol Antao authored
-
- Apr 28, 2018
-
-
Nischol Antao authored
-
- Apr 25, 2018
-
-
Nischol Antao authored
-
- Apr 24, 2018
-
-
Nischol Antao authored
1) Ran Code for questions 1-3 on pyspark in cluster mode, with multiple nodes. Measured and captured the difference in performance between running it on a single EC-2 instance, and running it on a cluster. 2) Added some screenshots for the final report, to show the cluster configuration. 3) Added ipython notebooks for performance metrics in local mode. 4) Added json files for zeppelin notebooks 5) Created new source files for Code pyspark code run in zeppelin notebooks, in cluster mode 6) Added test results for question 3 when using hive to calculate the median data. 7) Added R code from Rob for question 3 local exploration 8) Renamed some of the local exploration files
-
Nischol Antao authored
-
- Apr 22, 2018
-
-
Nischol Antao authored
-
- Apr 21, 2018
-
-
Nischol Antao authored
Added Guidelines for the final report, and updated the report to include the new sections we should include.
-
Nischol Antao authored
Finished spark implementation for question 1. Need to do some visualizations for this, and write the final report for it. Added iPython notebooks that detail the code for question 1. Formalized local pandas source, and remote pyspark source, and did a performance comparison.
-
- Apr 19, 2018
-
-
Nischol Antao authored
Modified local programs to output csv data. Measured the amount of time each query took on a local pc. Started a document for Challenges_Encountered
-
- Apr 17, 2018
-
-
Nischol Antao authored
-