-
- Downloads
[SPARK-18141][SQL] Fix to quote column names in the predicate clause of the...
[SPARK-18141][SQL] Fix to quote column names in the predicate clause of the JDBC RDD generated sql statement ## What changes were proposed in this pull request? SQL query generated for the JDBC data source is not quoting columns in the predicate clause. When the source table has quoted column names, spark jdbc read fails with column not found error incorrectly. Error: org.h2.jdbc.JdbcSQLException: Column "ID" not found; Source SQL statement: SELECT "Name","Id" FROM TEST."mixedCaseCols" WHERE (Id < 1) This PR fixes by quoting column names in the generated SQL for predicate clause when filters are pushed down to the data source. Source SQL statement after the fix: SELECT "Name","Id" FROM TEST."mixedCaseCols" WHERE ("Id" < 1) ## How was this patch tested? Added new test case to the JdbcSuite Author: sureshthalamati <suresh.thalamati@gmail.com> Closes #15662 from sureshthalamati/filter_quoted_cols-SPARK-18141. (cherry picked from commit 70c5549e) Signed-off-by:gatorsmile <gatorsmile@gmail.com>
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala 25 additions, 20 deletions...apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala 2 additions, 1 deletion...e/spark/sql/execution/datasources/jdbc/JDBCRelation.scala
- sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala 55 additions, 18 deletions.../src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
Loading
Please register or sign in to comment