-
- Downloads
[SPARK-12236][SQL] JDBC filter tests all pass if filters are not really pushed down
https://issues.apache.org/jira/browse/SPARK-12236 Currently JDBC filters are not tested properly. All the tests pass even if the filters are not pushed down due to Spark-side filtering. In this PR, Firstly, I corrected the tests to properly check the pushed down filters by removing Spark-side filtering. Also, `!=` was being tested which is actually not pushed down. So I removed them. Lastly, I moved the `stripSparkFilter()` function to `SQLTestUtils` as this functions would be shared for all tests for pushed down filters. This function would be also shared with ORC datasource as the filters for that are also not being tested properly. Author: hyukjinkwon <gurwls223@gmail.com> Closes #10221 from HyukjinKwon/SPARK-12236.
Showing
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala 0 additions, 15 deletions...ql/execution/datasources/parquet/ParquetFilterSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala 4 additions, 6 deletions.../src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala 15 additions, 0 deletions...c/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala
Loading
Please register or sign in to comment