Skip to content
Snippets Groups Projects
Commit 9740954f authored by Yanbo Liang's avatar Yanbo Liang Committed by Joseph K. Bradley
Browse files

[ML] testEstimatorAndModelReadWrite should call checkModelData

## What changes were proposed in this pull request?
Although we defined ```checkModelData``` in [```read/write``` test](https://github.com/apache/spark/blob/master/mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala#L994) of ML estimators/models and pass it to ```testEstimatorAndModelReadWrite```, ```testEstimatorAndModelReadWrite``` omits to call ```checkModelData``` to check the equality of model data. So actually we did not run the check of model data equality for all test cases currently, we should fix it.
BTW, fix the bug of LDA read/write test which did not set ```docConcentration```. This bug should have failed test, but it does not complain because we did not run ```checkModelData``` actually.
cc jkbradley mengxr
## How was this patch tested?
No new unit test, should pass the exist ones.

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #11513 from yanboliang/ml-check-model-data.
parent 46881b4e
No related branches found
No related tags found
No related merge requests found
...@@ -52,7 +52,8 @@ object LDASuite { ...@@ -52,7 +52,8 @@ object LDASuite {
"checkpointInterval" -> 30, "checkpointInterval" -> 30,
"learningOffset" -> 1023.0, "learningOffset" -> 1023.0,
"learningDecay" -> 0.52, "learningDecay" -> 0.52,
"subsamplingRate" -> 0.051 "subsamplingRate" -> 0.051,
"docConcentration" -> Array(2.0)
) )
} }
......
...@@ -82,6 +82,7 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite => ...@@ -82,6 +82,7 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite =>
* - Explicitly set Params, and train model * - Explicitly set Params, and train model
* - Test save/load using [[testDefaultReadWrite()]] on Estimator and Model * - Test save/load using [[testDefaultReadWrite()]] on Estimator and Model
* - Check Params on Estimator and Model * - Check Params on Estimator and Model
* - Compare model data
* *
* This requires that the [[Estimator]] and [[Model]] share the same set of [[Param]]s. * This requires that the [[Estimator]] and [[Model]] share the same set of [[Param]]s.
* @param estimator Estimator to test * @param estimator Estimator to test
...@@ -117,6 +118,8 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite => ...@@ -117,6 +118,8 @@ trait DefaultReadWriteTest extends TempDirectory { self: Suite =>
val param = model.getParam(p) val param = model.getParam(p)
assert(model.get(param).get === model2.get(param).get) assert(model.get(param).get === model2.get(param).get)
} }
checkModelData(model, model2)
} }
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment