-
- Downloads
[SPARK-5658][SQL] Finalize DDL and write support APIs
https://issues.apache.org/jira/browse/SPARK-5658 Author: Yin Huai <yhuai@databricks.com> This patch had conflicts when merged, resolved by Committer: Michael Armbrust <michael@databricks.com> Closes #4446 from yhuai/writeSupportFollowup and squashes the following commits: f3a96f7 [Yin Huai] davies's comments. 225ff71 [Yin Huai] Use Scala TestHiveContext to initialize the Python HiveContext in Python tests. 2306f93 [Yin Huai] Style. 2091fcd [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup 537e28f [Yin Huai] Correctly clean up temp data. ae4649e [Yin Huai] Fix Python test. 609129c [Yin Huai] Doc format. 92b6659 [Yin Huai] Python doc and other minor updates. cbc717f [Yin Huai] Rename dataSourceName to source. d1c12d3 [Yin Huai] No need to delete the duplicate rule since it has been removed in master. 22cfa70 [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup d91ecb8 [Yin Huai] Fix test. 4c76d78 [Yin Huai] Simplify APIs. 3abc215 [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup 0832ce4 [Yin Huai] Fix test. 98e7cdb [Yin Huai] Python style. 2bf44ef [Yin Huai] Python APIs. c204967 [Yin Huai] Format a10223d [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup 9ff97d8 [Yin Huai] Add SaveMode to saveAsTable. 9b6e570 [Yin Huai] Update doc. c2be775 [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup 99950a2 [Yin Huai] Use Java enum for SaveMode. 4679665 [Yin Huai] Remove duplicate rule. 77d89dc [Yin Huai] Update doc. e04d908 [Yin Huai] Move import and add (Scala-specific) to scala APIs. cf5703d [Yin Huai] Add checkAnswer to Java tests. 7db95ff [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup 6dfd386 [Yin Huai] Add java test. f2f33ef [Yin Huai] Fix test. e702386 [Yin Huai] Apache header. b1e9b1b [Yin Huai] Format. ed4e1b4 [Yin Huai] Merge remote-tracking branch 'upstream/master' into writeSupportFollowup af9e9b3 [Yin Huai] DDL and write support API followup. 2a6213a [Yin Huai] Update API names. e6a0b77 [Yin Huai] Update test. 43bae01 [Yin Huai] Remove createTable from HiveContext. 5ffc372 [Yin Huai] Add more load APIs to SQLContext. 5390743 [Yin Huai] Add more save APIs to DataFrame.
Showing
- python/pyspark/sql/context.py 68 additions, 0 deletionspython/pyspark/sql/context.py
- python/pyspark/sql/dataframe.py 69 additions, 3 deletionspython/pyspark/sql/dataframe.py
- python/pyspark/sql/tests.py 104 additions, 3 deletionspython/pyspark/sql/tests.py
- sql/core/src/main/java/org/apache/spark/sql/sources/SaveMode.java 45 additions, 0 deletions.../src/main/java/org/apache/spark/sql/sources/SaveMode.java
- sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala 139 additions, 21 deletionssql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameImpl.scala 14 additions, 47 deletions...e/src/main/scala/org/apache/spark/sql/DataFrameImpl.scala
- sql/core/src/main/scala/org/apache/spark/sql/IncomputableColumn.scala 7 additions, 20 deletions.../main/scala/org/apache/spark/sql/IncomputableColumn.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala 1 addition, 1 deletionsql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 155 additions, 9 deletions...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 5 additions, 9 deletions...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/json/JSONRelation.scala 24 additions, 6 deletions...c/main/scala/org/apache/spark/sql/json/JSONRelation.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/newParquet.scala 36 additions, 9 deletions.../main/scala/org/apache/spark/sql/parquet/newParquet.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/ddl.scala 32 additions, 8 deletions...ore/src/main/scala/org/apache/spark/sql/sources/ddl.scala
- sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala 19 additions, 0 deletions.../main/scala/org/apache/spark/sql/sources/interfaces.scala
- sql/core/src/test/java/org/apache/spark/sql/sources/JavaSaveLoadSuite.java 97 additions, 0 deletions.../java/org/apache/spark/sql/sources/JavaSaveLoadSuite.java
- sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala 61 additions, 31 deletionssql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/CreateTableAsSelectSuite.scala 21 additions, 8 deletions...g/apache/spark/sql/sources/CreateTableAsSelectSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala 44 additions, 15 deletions...st/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala 0 additions, 76 deletions...rc/main/scala/org/apache/spark/sql/hive/HiveContext.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 7 additions, 6 deletions...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
Loading
Please register or sign in to comment