-
- Downloads
[SPARK-12139][SQL] REGEX Column Specification
## What changes were proposed in this pull request?
Hive interprets regular expression, e.g., `(a)?+.+` in query specification. This PR enables spark to support this feature when hive.support.quoted.identifiers is set to true.
## How was this patch tested?
- Add unittests in SQLQuerySuite.scala
- Run spark-shell tested the original failed query:
scala> hc.sql("SELECT `(a|b)?+.+` from test1").collect.foreach(println)
Author: Jane Wang <janewang@fb.com>
Closes #18023 from janewangfb/support_select_regex.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala 28 additions, 1 deletion...a/org/apache/spark/sql/catalyst/analysis/unresolved.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala 36 additions, 7 deletions...ala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParserUtils.scala 6 additions, 0 deletions...la/org/apache/spark/sql/catalyst/parser/ParserUtils.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 8 additions, 0 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala 24 additions, 3 deletionssql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
- sql/core/src/test/resources/sql-tests/inputs/query_regex_column.sql 52 additions, 0 deletions...rc/test/resources/sql-tests/inputs/query_regex_column.sql
- sql/core/src/test/resources/sql-tests/results/query_regex_column.sql.out 313 additions, 0 deletions...st/resources/sql-tests/results/query_regex_column.sql.out
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala 6 additions, 4 deletions.../scala/org/apache/spark/sql/DataFrameAggregateSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala 115 additions, 110 deletions...cala/org/apache/spark/sql/DataFrameNaFunctionsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameStatSuite.scala 46 additions, 41 deletions.../test/scala/org/apache/spark/sql/DataFrameStatSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala 30 additions, 25 deletions.../src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala 80 additions, 1 deletion...re/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 24 additions, 17 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala 21 additions, 17 deletions...ache/spark/sql/execution/datasources/json/JsonSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala 2 additions, 1 deletion...ql/execution/datasources/parquet/ParquetFilterSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/DataSourceTest.scala 5 additions, 2 deletions...t/scala/org/apache/spark/sql/sources/DataSourceTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/sources/TableScanSuite.scala 29 additions, 26 deletions...t/scala/org/apache/spark/sql/sources/TableScanSuite.scala
Loading
Please register or sign in to comment