-
- Downloads
[SPARK-14843][ML] Fix encoding error in LibSVMRelation
## What changes were proposed in this pull request? We use `RowEncoder` in libsvm data source to serialize the label and features read from libsvm files. However, the schema passed in this encoder is not correct. As the result, we can't correctly select `features` column from the DataFrame. We should use full data schema instead of `requiredSchema` to serialize the data read in. Then do projection to select required columns later. ## How was this patch tested? `LibSVMRelationSuite`. Author: Liang-Chi Hsieh <simonh@tw.ibm.com> Closes #12611 from viirya/fix-libsvm.
Showing
- mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala 6 additions, 3 deletions...la/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala
- mllib/src/test/scala/org/apache/spark/ml/source/libsvm/LibSVMRelationSuite.scala 7 additions, 2 deletions...g/apache/spark/ml/source/libsvm/LibSVMRelationSuite.scala
Loading
Please register or sign in to comment