-
- Downloads
[SPARK-15031][EXAMPLE] Use SparkSession in Scala/Python/Java example.
## What changes were proposed in this pull request? This PR aims to update Scala/Python/Java examples by replacing `SQLContext` with newly added `SparkSession`. - Use **SparkSession Builder Pattern** in 154(Scala 55, Java 52, Python 47) files. - Add `getConf` in Python SparkContext class: `python/pyspark/context.py` - Replace **SQLContext Singleton Pattern** with **SparkSession Singleton Pattern**: - `SqlNetworkWordCount.scala` - `JavaSqlNetworkWordCount.java` - `sql_network_wordcount.py` Now, `SQLContexts` are used only in R examples and the following two Python examples. The python examples are untouched in this PR since it already fails some unknown issue. - `simple_params_example.py` - `aft_survival_regression.py` ## How was this patch tested? Manual. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #12809 from dongjoon-hyun/SPARK-15031.
Showing
- examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java 4 additions, 8 deletions...e/spark/examples/ml/JavaAFTSurvivalRegressionExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaALSExample.java 6 additions, 9 deletions...ain/java/org/apache/spark/examples/ml/JavaALSExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaBinarizerExample.java 7 additions, 8 deletions...va/org/apache/spark/examples/ml/JavaBinarizerExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaBisectingKMeansExample.java 7 additions, 11 deletions.../apache/spark/examples/ml/JavaBisectingKMeansExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaBucketizerExample.java 7 additions, 11 deletions...a/org/apache/spark/examples/ml/JavaBucketizerExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaChiSqSelectorExample.java 7 additions, 8 deletions...rg/apache/spark/examples/ml/JavaChiSqSelectorExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaCountVectorizerExample.java 7 additions, 12 deletions.../apache/spark/examples/ml/JavaCountVectorizerExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaDCTExample.java 7 additions, 8 deletions...ain/java/org/apache/spark/examples/ml/JavaDCTExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaDecisionTreeClassificationExample.java 5 additions, 8 deletions...rk/examples/ml/JavaDecisionTreeClassificationExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaDecisionTreeRegressionExample.java 5 additions, 8 deletions.../spark/examples/ml/JavaDecisionTreeRegressionExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java 5 additions, 10 deletions...org/apache/spark/examples/ml/JavaDeveloperApiExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaElementwiseProductExample.java 7 additions, 8 deletions...ache/spark/examples/ml/JavaElementwiseProductExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaEstimatorTransformerParamExample.java 6 additions, 10 deletions...ark/examples/ml/JavaEstimatorTransformerParamExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaGradientBoostedTreeClassifierExample.java 5 additions, 6 deletions...examples/ml/JavaGradientBoostedTreeClassifierExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaGradientBoostedTreeRegressorExample.java 5 additions, 9 deletions.../examples/ml/JavaGradientBoostedTreeRegressorExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaIndexToStringExample.java 7 additions, 11 deletions...rg/apache/spark/examples/ml/JavaIndexToStringExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaKMeansExample.java 5 additions, 9 deletions.../java/org/apache/spark/examples/ml/JavaKMeansExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaLDAExample.java 5 additions, 9 deletions...ain/java/org/apache/spark/examples/ml/JavaLDAExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaLinearRegressionWithElasticNetExample.java 5 additions, 8 deletions...xamples/ml/JavaLinearRegressionWithElasticNetExample.java
- examples/src/main/java/org/apache/spark/examples/ml/JavaLogisticRegressionSummaryExample.java 5 additions, 8 deletions...ark/examples/ml/JavaLogisticRegressionSummaryExample.java
Loading
Please register or sign in to comment