-
- Downloads
[SPARK-19817][SQL] Make it clear that `timeZone` option is a general option in...
[SPARK-19817][SQL] Make it clear that `timeZone` option is a general option in DataFrameReader/Writer. ## What changes were proposed in this pull request? As timezone setting can also affect partition values, it works for all formats, we should make it clear. ## How was this patch tested? Existing tests. Author: Takuya UESHIN <ueshin@databricks.com> Closes #17281 from ueshin/issues/SPARK-19817.
Showing
- python/pyspark/sql/readwriter.py 28 additions, 18 deletionspython/pyspark/sql/readwriter.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala 2 additions, 1 deletion...ala/org/apache/spark/sql/catalyst/catalog/interface.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala 3 additions, 2 deletions...cala/org/apache/spark/sql/catalyst/json/JSONOptions.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala 2 additions, 0 deletions...la/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala 6 additions, 3 deletions...spark/sql/catalyst/expressions/JsonExpressionsSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala 18 additions, 4 deletions...src/main/scala/org/apache/spark/sql/DataFrameReader.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala 18 additions, 4 deletions...src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala 1 addition, 1 deletion...pache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala 1 addition, 1 deletion...he/spark/sql/execution/datasources/FileFormatWriter.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningAwareFileIndex.scala 1 addition, 1 deletion...ql/execution/datasources/PartitioningAwareFileIndex.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala 3 additions, 2 deletions...ache/spark/sql/execution/datasources/csv/CSVOptions.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala 3 additions, 2 deletions...apache/spark/sql/execution/datasources/csv/CSVSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala 2 additions, 2 deletions...ache/spark/sql/execution/datasources/json/JsonSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetPartitionDiscoverySuite.scala 7 additions, 4 deletions.../datasources/parquet/ParquetPartitionDiscoverySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/PartitionedWriteSuite.scala 3 additions, 1 deletion.../org/apache/spark/sql/sources/PartitionedWriteSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/ResolvedDataSourceSuite.scala 2 additions, 1 deletion...rg/apache/spark/sql/sources/ResolvedDataSourceSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala 1 addition, 1 deletion...scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala
Loading
Please register or sign in to comment