Skip to content
Snippets Groups Projects
Commit 5fb97dca authored by Shivaram Venkataraman's avatar Shivaram Venkataraman Committed by Davies Liu
Browse files

[SPARK-7954] [SPARKR] Create SparkContext in sparkRSQL init

cc davies

Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>

Closes #6507 from shivaram/sparkr-init and squashes the following commits:

6fdd169 [Shivaram Venkataraman] Create SparkContext in sparkRSQL init
parent 82a396c2
No related branches found
No related tags found
No related merge requests found
......@@ -225,14 +225,21 @@ sparkR.init <- function(
#' sqlContext <- sparkRSQL.init(sc)
#'}
sparkRSQL.init <- function(jsc) {
sparkRSQL.init <- function(jsc = NULL) {
if (exists(".sparkRSQLsc", envir = .sparkREnv)) {
return(get(".sparkRSQLsc", envir = .sparkREnv))
}
# If jsc is NULL, create a Spark Context
sc <- if (is.null(jsc)) {
sparkR.init()
} else {
jsc
}
sqlContext <- callJStatic("org.apache.spark.sql.api.r.SQLUtils",
"createSQLContext",
jsc)
"createSQLContext",
sc)
assign(".sparkRSQLsc", sqlContext, envir = .sparkREnv)
sqlContext
}
......@@ -249,12 +256,19 @@ sparkRSQL.init <- function(jsc) {
#' sqlContext <- sparkRHive.init(sc)
#'}
sparkRHive.init <- function(jsc) {
sparkRHive.init <- function(jsc = NULL) {
if (exists(".sparkRHivesc", envir = .sparkREnv)) {
return(get(".sparkRHivesc", envir = .sparkREnv))
}
ssc <- callJMethod(jsc, "sc")
# If jsc is NULL, create a Spark Context
sc <- if (is.null(jsc)) {
sparkR.init()
} else {
jsc
}
ssc <- callJMethod(sc, "sc")
hiveCtx <- tryCatch({
newJObject("org.apache.spark.sql.hive.HiveContext", ssc)
}, error = function(err) {
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment