-
- Downloads
[SPARK-19439][PYSPARK][SQL] PySpark's registerJavaFunction Should Support UDAFs
## What changes were proposed in this pull request? Support register Java UDAFs in PySpark so that user can use Java UDAF in PySpark. Besides that I also add api in `UDFRegistration` ## How was this patch tested? Unit test is added Author: Jeff Zhang <zjffdu@apache.org> Closes #17222 from zjffdu/SPARK-19439.
Showing
- python/pyspark/sql/context.py 23 additions, 0 deletionspython/pyspark/sql/context.py
- python/pyspark/sql/tests.py 10 additions, 0 deletionspython/pyspark/sql/tests.py
- sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala 27 additions, 6 deletions...src/main/scala/org/apache/spark/sql/UDFRegistration.scala
- sql/core/src/test/java/test/org/apache/spark/sql/JavaUDAFSuite.java 55 additions, 0 deletions...rc/test/java/test/org/apache/spark/sql/JavaUDAFSuite.java
- sql/core/src/test/java/test/org/apache/spark/sql/MyDoubleAvg.java 1 addition, 1 deletion.../src/test/java/test/org/apache/spark/sql/MyDoubleAvg.java
- sql/core/src/test/java/test/org/apache/spark/sql/MyDoubleSum.java 4 additions, 4 deletions.../src/test/java/test/org/apache/spark/sql/MyDoubleSum.java
- sql/hive/pom.xml 7 additions, 0 deletionssql/hive/pom.xml
- sql/hive/src/test/java/org/apache/spark/sql/hive/JavaDataFrameSuite.java 1 addition, 1 deletion...st/java/org/apache/spark/sql/hive/JavaDataFrameSuite.java
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala 4 additions, 1 deletion...ache/spark/sql/hive/execution/AggregationQuerySuite.scala
Loading
Please register or sign in to comment