-
- Downloads
[SPARK-9100] [SQL] Adds DataFrame reader/writer shortcut methods for ORC
This PR adds DataFrame reader/writer shortcut methods for ORC in both Scala and Python. Author: Cheng Lian <lian@databricks.com> Closes #7444 from liancheng/spark-9100 and squashes the following commits: 284d043 [Cheng Lian] Fixes PySpark test cases and addresses PR comments e0b09fb [Cheng Lian] Adds DataFrame reader/writer shortcut methods for ORC
Showing
- python/pyspark/sql/readwriter.py 41 additions, 3 deletionspython/pyspark/sql/readwriter.py
- python/test_support/sql/orc_partitioned/._SUCCESS.crc 0 additions, 0 deletionspython/test_support/sql/orc_partitioned/._SUCCESS.crc
- python/test_support/sql/orc_partitioned/_SUCCESS 0 additions, 0 deletionspython/test_support/sql/orc_partitioned/_SUCCESS
- python/test_support/sql/orc_partitioned/b=0/c=0/.part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc 0 additions, 0 deletions...part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc
- python/test_support/sql/orc_partitioned/b=0/c=0/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc 0 additions, 0 deletions...c=0/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc
- python/test_support/sql/orc_partitioned/b=1/c=1/.part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc 0 additions, 0 deletions...part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc
- python/test_support/sql/orc_partitioned/b=1/c=1/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc 0 additions, 0 deletions...c=1/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala 9 additions, 0 deletions...src/main/scala/org/apache/spark/sql/DataFrameReader.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala 12 additions, 0 deletions...src/main/scala/org/apache/spark/sql/DataFrameWriter.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcHadoopFsRelationSuite.scala 1 addition, 2 deletions.../apache/spark/sql/hive/orc/OrcHadoopFsRelationSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcPartitionDiscoverySuite.scala 6 additions, 8 deletions...pache/spark/sql/hive/orc/OrcPartitionDiscoverySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala 6 additions, 6 deletions...t/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcTest.scala 4 additions, 4 deletions...rc/test/scala/org/apache/spark/sql/hive/orc/OrcTest.scala
Loading
Please register or sign in to comment