-
- Downloads
[SPARK-20335][SQL][BACKPORT-2.1] Children expressions of Hive UDF impacts the...
[SPARK-20335][SQL][BACKPORT-2.1] Children expressions of Hive UDF impacts the determinism of Hive UDF ### What changes were proposed in this pull request? This PR is to backport https://github.com/apache/spark/pull/17635 to Spark 2.1 --- ```JAVA /** * Certain optimizations should not be applied if UDF is not deterministic. * Deterministic UDF returns same result each time it is invoked with a * particular input. This determinism just needs to hold within the context of * a query. * * return true if the UDF is deterministic */ boolean deterministic() default true; ``` Based on the definition of [UDFType](https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFType.java#L42-L50), when Hive UDF's children are non-deterministic, Hive UDF is also non-deterministic. ### How was this patch tested? Added test cases. Author: Xiao Li <gatorsmile@gmail.com> Closes #17652 from gatorsmile/backport-17635.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala 2 additions, 2 deletions...e/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala 13 additions, 0 deletions...ache/spark/sql/hive/execution/AggregationQuerySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala 30 additions, 0 deletions...la/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
Loading
Please register or sign in to comment