Skip to content
Snippets Groups Projects
Commit 6902edab authored by Yanbo Liang's avatar Yanbo Liang
Browse files

[SPARK-17315][FOLLOW-UP][SPARKR][ML] Fix print of Kolmogorov-Smirnov test summary

## What changes were proposed in this pull request?
#14881 added Kolmogorov-Smirnov Test wrapper to SparkR. I found that ```print.summary.KSTest``` was implemented inappropriately and result in no effect.
Running the following code for KSTest:
```Scala
data <- data.frame(test = c(0.1, 0.15, 0.2, 0.3, 0.25, -1, -0.5))
df <- createDataFrame(data)
testResult <- spark.kstest(df, "test", "norm")
summary(testResult)
```
Before this PR:
![image](https://cloud.githubusercontent.com/assets/1962026/18615016/b9a2823a-7d4f-11e6-934b-128beade355e.png)
After this PR:
![image](https://cloud.githubusercontent.com/assets/1962026/18615014/aafe2798-7d4f-11e6-8b99-c705bb9fe8f2.png)
The new implementation is similar with [```print.summary.GeneralizedLinearRegressionModel```](https://github.com/apache/spark/blob/master/R/pkg/R/mllib.R#L284) of SparkR and [```print.summary.glm```](https://svn.r-project.org/R/trunk/src/library/stats/R/glm.R) of native R.

BTW, I removed the comparison of ```print.summary.KSTest``` in unit test, since it's only wrappers of the summary output which has been checked. Another reason is that these comparison will output summary information to the test console, it will make the test output in a mess.

## How was this patch tested?
Existing test.

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #15139 from yanboliang/spark-17315.
parent c133907c
No related branches found
No related tags found
No related merge requests found
......@@ -1398,20 +1398,22 @@ setMethod("summary", signature(object = "KSTest"),
distParams <- unlist(callJMethod(jobj, "distParams"))
degreesOfFreedom <- callJMethod(jobj, "degreesOfFreedom")
list(p.value = pValue, statistic = statistic, nullHypothesis = nullHypothesis,
nullHypothesis.name = distName, nullHypothesis.parameters = distParams,
degreesOfFreedom = degreesOfFreedom)
ans <- list(p.value = pValue, statistic = statistic, nullHypothesis = nullHypothesis,
nullHypothesis.name = distName, nullHypothesis.parameters = distParams,
degreesOfFreedom = degreesOfFreedom, jobj = jobj)
class(ans) <- "summary.KSTest"
ans
})
# Prints the summary of KSTest
#' @rdname spark.kstest
#' @param x test result object of KSTest by \code{spark.kstest}.
#' @param x summary object of KSTest returned by \code{summary}.
#' @export
#' @note print.summary.KSTest since 2.1.0
print.summary.KSTest <- function(x, ...) {
jobj <- x@jobj
jobj <- x$jobj
summaryStr <- callJMethod(jobj, "summary")
cat(summaryStr)
invisible(summaryStr)
cat(summaryStr, "\n")
invisible(x)
}
......@@ -760,13 +760,7 @@ test_that("spark.kstest", {
expect_equal(stats$p.value, rStats$p.value, tolerance = 1e-4)
expect_equal(stats$statistic, unname(rStats$statistic), tolerance = 1e-4)
printStr <- print.summary.KSTest(testResult)
expect_match(printStr, paste0("Kolmogorov-Smirnov test summary:\\n",
"degrees of freedom = 0 \\n",
"statistic = 0.38208[0-9]* \\n",
"pValue = 0.19849[0-9]* \\n",
".*"), perl = TRUE)
expect_match(capture.output(stats)[1], "Kolmogorov-Smirnov test summary:")
testResult <- spark.kstest(df, "test", "norm", -0.5)
stats <- summary(testResult)
......@@ -775,13 +769,7 @@ test_that("spark.kstest", {
expect_equal(stats$p.value, rStats$p.value, tolerance = 1e-4)
expect_equal(stats$statistic, unname(rStats$statistic), tolerance = 1e-4)
printStr <- print.summary.KSTest(testResult)
expect_match(printStr, paste0("Kolmogorov-Smirnov test summary:\\n",
"degrees of freedom = 0 \\n",
"statistic = 0.44003[0-9]* \\n",
"pValue = 0.09470[0-9]* \\n",
".*"), perl = TRUE)
expect_match(capture.output(stats)[1], "Kolmogorov-Smirnov test summary:")
})
sparkR.session.stop()
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment