-
- Downloads
[SPARK-16750][ML] Fix GaussianMixture training failed due to feature column type mistake
## What changes were proposed in this pull request? ML ```GaussianMixture``` training failed due to feature column type mistake. The feature column type should be ```ml.linalg.VectorUDT``` but got ```mllib.linalg.VectorUDT``` by mistake. See [SPARK-16750](https://issues.apache.org/jira/browse/SPARK-16750) for how to reproduce this bug. Why the unit tests did not complain this errors? Because some estimators/transformers missed calling ```transformSchema(dataset.schema)``` firstly during ```fit``` or ```transform```. I will also add this function to all estimators/transformers who missed in this PR. ## How was this patch tested? No new tests, should pass existing ones. Author: Yanbo Liang <ybliang8@gmail.com> Closes #14378 from yanboliang/spark-16750.
Showing
- mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala 2 additions, 0 deletions...cala/org/apache/spark/ml/clustering/BisectingKMeans.scala
- mllib/src/main/scala/org/apache/spark/ml/clustering/GaussianMixture.scala 5 additions, 3 deletions...cala/org/apache/spark/ml/clustering/GaussianMixture.scala
- mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala 2 additions, 0 deletions...rc/main/scala/org/apache/spark/ml/clustering/KMeans.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/Interaction.scala 1 addition, 0 deletions.../main/scala/org/apache/spark/ml/feature/Interaction.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/MinMaxScaler.scala 1 addition, 0 deletions...main/scala/org/apache/spark/ml/feature/MinMaxScaler.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala 2 additions, 1 deletion...ala/org/apache/spark/ml/feature/QuantileDiscretizer.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala 1 addition, 0 deletions...src/main/scala/org/apache/spark/ml/feature/RFormula.scala
- mllib/src/main/scala/org/apache/spark/ml/feature/SQLTransformer.scala 1 addition, 0 deletions...in/scala/org/apache/spark/ml/feature/SQLTransformer.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.scala 2 additions, 2 deletions...rg/apache/spark/ml/regression/AFTSurvivalRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala 2 additions, 1 deletion...a/org/apache/spark/ml/regression/IsotonicRegression.scala
Loading
Please register or sign in to comment