-
- Downloads
[SPARK-19548][SQL] Support Hive UDFs which return typed Lists/Maps
## What changes were proposed in this pull request? This PR adds support for Hive UDFs that return fully typed java Lists or Maps, for example `List<String>` or `Map<String, Integer>`. It is also allowed to nest these structures, for example `Map<String, List<Integer>>`. Raw collections or collections using wildcards are still not supported, and cannot be supported due to the lack of type information. ## How was this patch tested? Modified existing tests in `HiveUDFSuite`, and I have added test cases for raw collection and collection using wildcards. Author: Herman van Hovell <hvanhovell@databricks.com> Closes #16886 from hvanhovell/SPARK-19548.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala 28 additions, 10 deletions...main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala 1 addition, 1 deletion...e/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFRawList.java 31 additions, 0 deletions.../java/org/apache/spark/sql/hive/execution/UDFRawList.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFRawMap.java 31 additions, 0 deletions...t/java/org/apache/spark/sql/hive/execution/UDFRawMap.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFToIntIntMap.java 9 additions, 9 deletions...a/org/apache/spark/sql/hive/execution/UDFToIntIntMap.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFToListInt.java 4 additions, 4 deletions...ava/org/apache/spark/sql/hive/execution/UDFToListInt.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFToListMapStringListInt.java 34 additions, 0 deletions...e/spark/sql/hive/execution/UDFToListMapStringListInt.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFToListString.java 3 additions, 3 deletions.../org/apache/spark/sql/hive/execution/UDFToListString.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFToStringIntMap.java 9 additions, 10 deletions...rg/apache/spark/sql/hive/execution/UDFToStringIntMap.java
- sql/hive/src/test/java/org/apache/spark/sql/hive/execution/UDFWildcardList.java 31 additions, 0 deletions.../org/apache/spark/sql/hive/execution/UDFWildcardList.java
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala 69 additions, 20 deletions...la/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
Loading
Please register or sign in to comment