-
- Downloads
[SPARK-4131] Support "Writing data into the filesystem from queries"
## What changes were proposed in this pull request? This PR implements the sql feature: INSERT OVERWRITE [LOCAL] DIRECTORY directory1 [ROW FORMAT row_format] [STORED AS file_format] SELECT ... FROM ... ## How was this patch tested? Added new unittests and also pulled the code to fb-spark so that we could test writing to hdfs directory. Author: Jane Wang <janewang@fb.com> Closes #18975 from janewangfb/port_local_directory.
Showing
- sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 6 additions, 2 deletions...in/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala 3 additions, 0 deletions...k/sql/catalyst/analysis/UnsupportedOperationChecker.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala 72 additions, 7 deletions...ala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala 25 additions, 1 deletion...rk/sql/catalyst/plans/logical/basicLogicalOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala 78 additions, 1 deletion...scala/org/apache/spark/sql/execution/SparkSqlParser.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/command/InsertIntoDataSourceDirCommand.scala 82 additions, 0 deletions...ql/execution/command/InsertIntoDataSourceDirCommand.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala 16 additions, 1 deletion...in/scala/org/apache/spark/sql/execution/command/ddl.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala 13 additions, 8 deletions.../spark/sql/execution/datasources/DataSourceStrategy.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLParserSuite.scala 51 additions, 1 deletion...g/apache/spark/sql/execution/command/DDLParserSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/InsertSuite.scala 60 additions, 0 deletions...test/scala/org/apache/spark/sql/sources/InsertSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 10 additions, 1 deletion...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala 203 additions, 0 deletions...ala/org/apache/spark/sql/hive/execution/HiveTmpPath.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala 131 additions, 0 deletions...e/spark/sql/hive/execution/InsertIntoHiveDirCommand.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala 11 additions, 202 deletions...apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala 73 additions, 0 deletions.../org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertSuite.scala 181 additions, 1 deletion...rc/test/scala/org/apache/spark/sql/hive/InsertSuite.scala
Loading
Please register or sign in to comment