-
- Downloads
[SPARK-7749] [SQL] Fixes partition discovery for non-partitioned tables
When no partition columns can be found, we should have an empty `PartitionSpec`, rather than a `PartitionSpec` with empty partition columns. This PR together with #6285 should fix SPARK-7749. Author: Cheng Lian <lian@databricks.com> Author: Yin Huai <yhuai@databricks.com> Closes #6287 from liancheng/spark-7749 and squashes the following commits: a799ff3 [Cheng Lian] Adds test cases for SPARK-7749 c4949be [Cheng Lian] Minor refactoring, and tolerant _TEMPORARY directory name 5aa87ea [Yin Huai] Make parsePartitions more robust. fc56656 [Cheng Lian] Returns empty PartitionSpec if no partition columns can be inferred 19ae41e [Cheng Lian] Don't list base directory as leaf directory
Showing
- sql/core/src/main/scala/org/apache/spark/sql/sources/PartitioningUtils.scala 57 additions, 27 deletions...cala/org/apache/spark/sql/sources/PartitioningUtils.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala 1 addition, 6 deletions.../main/scala/org/apache/spark/sql/sources/interfaces.scala
- sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetPartitionDiscoverySuite.scala 42 additions, 7 deletions...he/spark/sql/parquet/ParquetPartitionDiscoverySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/parquetSuites.scala 50 additions, 1 deletion.../test/scala/org/apache/spark/sql/hive/parquetSuites.scala
Loading
Please register or sign in to comment