-
- Downloads
[SPARK-7248] implemented random number generators for DataFrames
Adds the functions `rand` (Uniform Dist) and `randn` (Normal Dist.) as expressions to DataFrames. cc mengxr rxin Author: Burak Yavuz <brkyvz@gmail.com> Closes #5819 from brkyvz/df-rng and squashes the following commits: 50d69d4 [Burak Yavuz] add seed for test that failed 4234c3a [Burak Yavuz] fix Rand expression 13cad5c [Burak Yavuz] couple fixes 7d53953 [Burak Yavuz] waiting for hive tests b453716 [Burak Yavuz] move radn with seed down 03637f0 [Burak Yavuz] fix broken hive func c5909eb [Burak Yavuz] deleted old implementation of Rand 6d43895 [Burak Yavuz] implemented random generators
Showing
- python/pyspark/sql/functions.py 24 additions, 1 deletionpython/pyspark/sql/functions.py
- python/pyspark/sql/tests.py 10 additions, 0 deletionspython/pyspark/sql/tests.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/random.scala 56 additions, 0 deletions...la/org/apache/spark/sql/catalyst/expressions/random.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ConstantFoldingSuite.scala 2 additions, 2 deletions...e/spark/sql/catalyst/optimizer/ConstantFoldingSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/functions.scala 29 additions, 1 deletionsql/core/src/main/scala/org/apache/spark/sql/functions.scala
- sql/core/src/main/scala/org/apache/spark/sql/mathfunctions.scala 0 additions, 2 deletions...e/src/main/scala/org/apache/spark/sql/mathfunctions.scala
- sql/core/src/test/java/test/org/apache/spark/sql/JavaDataFrameSuite.java 3 additions, 0 deletions...st/java/test/org/apache/spark/sql/JavaDataFrameSuite.java
- sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala 22 additions, 0 deletions...st/scala/org/apache/spark/sql/ColumnExpressionSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala 3 additions, 4 deletions...ive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
Loading
Please register or sign in to comment