-
- Downloads
[SPARK-15956][SQL] When unwrapping ORC avoid pattern matching at runtime
## What changes were proposed in this pull request? Extend the returning of unwrapper functions from primitive types to all types. ## How was this patch tested? The patch should pass all unit tests. Reading ORC files with non-primitive types with this change reduced the read time by ~15%. === The github diff is very noisy. Attaching the screenshots below for improved readability:   Author: Brian Cho <bcho@fb.com> Closes #13676 from dafrista/improve-orc-master.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala 287 additions, 141 deletions...main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala 2 additions, 1 deletion...rc/main/scala/org/apache/spark/sql/hive/TableReader.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/ScriptTransformation.scala 4 additions, 2 deletions...pache/spark/sql/hive/execution/ScriptTransformation.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala 15 additions, 6 deletions...e/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala 6 additions, 0 deletions.../scala/org/apache/spark/sql/hive/HiveInspectorSuite.scala
Loading
Please register or sign in to comment