-
- Downloads
[SPARK-21739][SQL] Cast expression should initialize timezoneId when it is...
[SPARK-21739][SQL] Cast expression should initialize timezoneId when it is called statically to convert something into TimestampType ## What changes were proposed in this pull request? https://issues.apache.org/jira/projects/SPARK/issues/SPARK-21739 This issue is caused by introducing TimeZoneAwareExpression. When the **Cast** expression converts something into TimestampType, it should be resolved with setting `timezoneId`. In general, it is resolved in LogicalPlan phase. However, there are still some places that use Cast expression statically to convert datatypes without setting `timezoneId`. In such cases, `NoSuchElementException: None.get` will be thrown for TimestampType. This PR is proposed to fix the issue. We have checked the whole project and found two such usages(i.e., in`TableReader` and `HiveTableScanExec`). ## How was this patch tested? unit test Author: donnyzone <wellfengzhu@gmail.com> Closes #18960 from DonnyZone/spark-21739.
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala 6 additions, 2 deletions...rc/main/scala/org/apache/spark/sql/hive/TableReader.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScanExec.scala 6 additions, 2 deletions...g/apache/spark/sql/hive/execution/HiveTableScanExec.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala 17 additions, 0 deletions...scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala
Loading
Please register or sign in to comment