Skip to content
Snippets Groups Projects
Commit 06bae8af authored by wm624@hotmail.com's avatar wm624@hotmail.com Committed by Shivaram Venkataraman
Browse files

[SPARK-15439][SPARKR] Failed to run unit test in SparkR

## What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)
There are some failures when running SparkR unit tests.
In this PR, I fixed two of these failures in test_context.R and test_sparkSQL.R
The first one is due to different masked name. I added missed names in the expected arrays.
The second one is because one PR removed the logic of a previous fix of missing subset method.

The file privilege issue is still there. I am debugging it. SparkR shell can run the test case successfully.
test_that("pipeRDD() on RDDs", {
  actual <- collect(pipeRDD(rdd, "more"))
When using run-test script, it complains no such directories as below:
cannot open file '/tmp/Rtmp4FQbah/filee2273f9d47f7': No such file or directory

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Manually test it

Author: wm624@hotmail.com <wm624@hotmail.com>

Closes #13284 from wangmiao1981/R.
parent 06ed1fa3
No related branches found
No related tags found
No related merge requests found
......@@ -1445,7 +1445,11 @@ setMethod("[", signature(x = "SparkDataFrame"),
#' }
setMethod("subset", signature(x = "SparkDataFrame"),
function(x, subset, select, drop = F, ...) {
x[subset, select, drop = drop]
if (missing(subset)) {
x[, select, drop = drop, ...]
} else {
x[subset, select, drop = drop, ...]
}
})
#' Select
......
......@@ -27,6 +27,11 @@ test_that("Check masked functions", {
namesOfMasked <- c("describe", "cov", "filter", "lag", "na.omit", "predict", "sd", "var",
"colnames", "colnames<-", "intersect", "rank", "rbind", "sample", "subset",
"summary", "transform", "drop", "window", "as.data.frame")
namesOfMaskedCompletely <- c("cov", "filter", "sample")
if (as.numeric(R.version$major) == 3 && as.numeric(R.version$minor) > 2) {
namesOfMasked <- c("endsWith", "startsWith", namesOfMasked)
namesOfMaskedCompletely <- c("endsWith", "startsWith", namesOfMaskedCompletely)
}
expect_equal(length(maskedBySparkR), length(namesOfMasked))
expect_equal(sort(maskedBySparkR), sort(namesOfMasked))
# above are those reported as masked when `library(SparkR)`
......@@ -36,7 +41,6 @@ test_that("Check masked functions", {
any(grepl("=\"ANY\"", capture.output(showMethods(x)[-1])))
}))
maskedCompletely <- masked[!funcHasAny]
namesOfMaskedCompletely <- c("cov", "filter", "sample")
expect_equal(length(maskedCompletely), length(namesOfMaskedCompletely))
expect_equal(sort(maskedCompletely), sort(namesOfMaskedCompletely))
})
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment