-
- Downloads
[SPARK-12573][SPARK-12574][SQL] Move SQL Parser from Hive to Catalyst
This PR moves a major part of the new SQL parser to Catalyst. This is a prelude to start using this parser for all of our SQL parsing. The following key changes have been made: The ANTLR Parser & Supporting classes have been moved to the Catalyst project. They are now part of the ```org.apache.spark.sql.catalyst.parser``` package. These classes contained quite a bit of code that was originally from the Hive project, I have added aknowledgements whenever this applied. All Hive dependencies have been factored out. I have also taken this chance to clean-up the ```ASTNode``` class, and to improve the error handling. The HiveQl object that provides the functionality to convert an AST into a LogicalPlan has been refactored into three different classes, one for every SQL sub-project: - ```CatalystQl```: This implements Query and Expression parsing functionality. - ```SparkQl```: This is a subclass of CatalystQL and provides SQL/Core only functionality such as Explain and Describe. - ```HiveQl```: This is a subclass of ```SparkQl``` and this adds Hive-only functionality to the parser such as Analyze, Drop, Views, CTAS & Transforms. This class still depends on Hive. cc rxin Author: Herman van Hovell <hvanhovell@questtec.nl> Closes #10583 from hvanhovell/SPARK-12575.
Showing
- dev/deps/spark-deps-hadoop-2.2 1 addition, 3 deletionsdev/deps/spark-deps-hadoop-2.2
- dev/deps/spark-deps-hadoop-2.3 1 addition, 3 deletionsdev/deps/spark-deps-hadoop-2.3
- dev/deps/spark-deps-hadoop-2.4 1 addition, 3 deletionsdev/deps/spark-deps-hadoop-2.4
- dev/deps/spark-deps-hadoop-2.6 1 addition, 3 deletionsdev/deps/spark-deps-hadoop-2.6
- pom.xml 6 additions, 0 deletionspom.xml
- project/SparkBuild.scala 56 additions, 48 deletionsproject/SparkBuild.scala
- sql/catalyst/pom.xml 22 additions, 0 deletionssql/catalyst/pom.xml
- sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/FromClauseParser.g 3 additions, 1 deletion...3/org/apache/spark/sql/catalyst/parser/FromClauseParser.g
- sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/IdentifiersParser.g 3 additions, 1 deletion.../org/apache/spark/sql/catalyst/parser/IdentifiersParser.g
- sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SelectClauseParser.g 3 additions, 1 deletion...org/apache/spark/sql/catalyst/parser/SelectClauseParser.g
- sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlLexer.g 19 additions, 8 deletions...tlr3/org/apache/spark/sql/catalyst/parser/SparkSqlLexer.g
- sql/catalyst/src/main/antlr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g 18 additions, 13 deletions...lr3/org/apache/spark/sql/catalyst/parser/SparkSqlParser.g
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/parser/ParseUtils.java 162 additions, 0 deletions...java/org/apache/spark/sql/catalyst/parser/ParseUtils.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystQl.scala 961 additions, 0 deletions...main/scala/org/apache/spark/sql/catalyst/CatalystQl.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ASTNode.scala 93 additions, 0 deletions.../scala/org/apache/spark/sql/catalyst/parser/ASTNode.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseDriver.scala 156 additions, 0 deletions...la/org/apache/spark/sql/catalyst/parser/ParseDriver.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserConf.scala 26 additions, 0 deletions...ala/org/apache/spark/sql/catalyst/parser/ParserConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala 19 additions, 1 deletionsql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkQl.scala 84 additions, 0 deletions...c/main/scala/org/apache/spark/sql/execution/SparkQl.scala
- sql/hive/pom.xml 0 additions, 20 deletionssql/hive/pom.xml
Loading
Please register or sign in to comment