-
- Downloads
[SPARK-4041][SQL] Attributes names in table scan should converted to lowercase...
[SPARK-4041][SQL] Attributes names in table scan should converted to lowercase when compare with relation attributes In ```MetastoreRelation``` the attributes name is lowercase because of hive using lowercase for fields name, so we should convert attributes name in table scan lowercase in ```indexWhere(_.name == a.name)```. ```neededColumnIDs``` may be not correct if not convert to lowercase. Author: wangfei <wangfei1@huawei.com> Author: scwf <wangfei1@huawei.com> Closes #2884 from scwf/fixColumnIds and squashes the following commits: 6174046 [scwf] use AttributeMap for this issue dc74a24 [wangfei] use lowerName and add a test case for this issue 3ff3a80 [wangfei] more safer change 294fcb7 [scwf] attributes names in table scan should convert lowercase in neededColumnsIDs
Showing
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 6 additions, 0 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala 5 additions, 4 deletions...a/org/apache/spark/sql/hive/execution/HiveTableScan.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala 9 additions, 0 deletions.../apache/spark/sql/hive/execution/HiveTableScanSuite.scala
Loading
Please register or sign in to comment