-
- Downloads
SPARK-2272 [MLlib] Feature scaling which standardizes the range of independent...
SPARK-2272 [MLlib] Feature scaling which standardizes the range of independent variables or features of data Feature scaling is a method used to standardize the range of independent variables or features of data. In data processing, it is generally performed during the data preprocessing step. In this work, a trait called `VectorTransformer` is defined for generic transformation on a vector. It contains one method to be implemented, `transform` which applies transformation on a vector. There are two implementations of `VectorTransformer` now, and they all can be easily extended with PMML transformation support. 1) `StandardScaler` - Standardizes features by removing the mean and scaling to unit variance using column summary statistics on the samples in the training set. 2) `Normalizer` - Normalizes samples individually to unit L^n norm Author: DB Tsai <dbtsai@alpinenow.com> Closes #1207 from dbtsai/dbtsai-feature-scaling and squashes the following commits: 78c15d3 [DB Tsai] Alpine Data Labs
Showing
- mllib/src/main/scala/org/apache/spark/mllib/feature/Normalizer.scala 76 additions, 0 deletions...ain/scala/org/apache/spark/mllib/feature/Normalizer.scala
- mllib/src/main/scala/org/apache/spark/mllib/feature/StandardScaler.scala 119 additions, 0 deletions...scala/org/apache/spark/mllib/feature/StandardScaler.scala
- mllib/src/main/scala/org/apache/spark/mllib/feature/VectorTransformer.scala 51 additions, 0 deletions...la/org/apache/spark/mllib/feature/VectorTransformer.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala 1 addition, 1 deletion...org/apache/spark/mllib/linalg/distributed/RowMatrix.scala
- mllib/src/test/scala/org/apache/spark/mllib/feature/NormalizerSuite.scala 120 additions, 0 deletions...cala/org/apache/spark/mllib/feature/NormalizerSuite.scala
- mllib/src/test/scala/org/apache/spark/mllib/feature/StandardScalerSuite.scala 200 additions, 0 deletions.../org/apache/spark/mllib/feature/StandardScalerSuite.scala
Loading
Please register or sign in to comment