-
- Downloads
[SPARK-3911] [SQL] HiveSimpleUdf can not be optimized in constant folding
``` explain extended select cos(null) from src limit 1; ``` outputs: ``` Project [HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFCos(null) AS c_0#5] MetastoreRelation default, src, None == Optimized Logical Plan == Limit 1 Project [HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFCos(null) AS c_0#5] MetastoreRelation default, src, None == Physical Plan == Limit 1 Project [HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFCos(null) AS c_0#5] HiveTableScan [], (MetastoreRelation default, src, None), None ``` After patching this PR it outputs ``` == Parsed Logical Plan == Limit 1 Project ['cos(null) AS c_0#0] UnresolvedRelation None, src, None == Analyzed Logical Plan == Limit 1 Project [HiveSimpleUdf#org.apache.hadoop.hive.ql.udf.UDFCos(null) AS c_0#0] MetastoreRelation default, src, None == Optimized Logical Plan == Limit 1 Project [null AS c_0#0] MetastoreRelation default, src, None == Physical Plan == Limit 1 Project [null AS c_0#0] HiveTableScan [], (MetastoreRelation default, src, None), None ``` Author: Cheng Hao <hao.cheng@intel.com> Closes #2771 from chenghao-intel/hive_udf_constant_folding and squashes the following commits: 1379c73 [Cheng Hao] duplicate the PlanTest with catalyst/plans/PlanTest 1e52dda [Cheng Hao] add unit test for hive simple udf constant folding 01609ff [Cheng Hao] support constant folding for HiveSimpleUdf
Showing
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala 2 additions, 1 deletion.../scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala 10 additions, 0 deletions...e/src/main/scala/org/apache/spark/sql/hive/hiveUdfs.scala
- sql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala 3 additions, 1 deletionsql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/hive/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala 57 additions, 0 deletions.../scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HivePlanTest.scala 32 additions, 0 deletions...la/org/apache/spark/sql/hive/execution/HivePlanTest.scala
Please register or sign in to comment