-
- Downloads
[SPARK-21050][ML] Word2vec persistence overflow bug fix
## What changes were proposed in this pull request? The method calculateNumberOfPartitions() uses Int, not Long (unlike the MLlib version), so it is very easily to have an overflow in calculating the number of partitions for ML persistence. This modifies the calculations to use Long. ## How was this patch tested? New unit test. I verified that the test fails before this patch. Author: Joseph K. Bradley <joseph@databricks.com> Closes #18265 from jkbradley/word2vec-save-fix.
Showing
- mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala 28 additions, 10 deletions...src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala
- mllib/src/test/scala/org/apache/spark/ml/feature/Word2VecSuite.scala 10 additions, 0 deletions...est/scala/org/apache/spark/ml/feature/Word2VecSuite.scala
Loading
Please register or sign in to comment