-
- Downloads
[SPARK-17509][SQL] When wrapping catalyst datatype to Hive data type avoid…
## What changes were proposed in this pull request? When wrapping catalyst datatypes to Hive data type, wrap function was doing an expensive pattern matching which was consuming around 11% of cpu time. Avoid the pattern matching by returning the wrapper only once and reuse it. ## How was this patch tested? Tested by running the job on cluster and saw around 8% cpu improvements. Author: Sital Kedia <skedia@fb.com> Closes #15064 from sitalkedia/skedia/hive_wrapper.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala 133 additions, 174 deletions...main/scala/org/apache/spark/sql/hive/HiveInspectors.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala 12 additions, 3 deletions...e/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala
Loading
Please register or sign in to comment