Skip to content
Snippets Groups Projects
Commit ba843292 authored by Yuming Wang's avatar Yuming Wang Committed by gatorsmile
Browse files

[SPARK-21790][TESTS][FOLLOW-UP] Add filter pushdown verification back.

## What changes were proposed in this pull request?

The previous PR(https://github.com/apache/spark/pull/19000) removed filter pushdown verification, This PR add them back.

## How was this patch tested?
manual tests

Author: Yuming Wang <wgyumg@gmail.com>

Closes #19002 from wangyum/SPARK-21790-follow-up.
parent 988b84d7
No related branches found
No related tags found
No related merge requests found
......@@ -22,6 +22,7 @@ import java.util.Properties
import java.math.BigDecimal
import org.apache.spark.sql.Row
import org.apache.spark.sql.execution.{WholeStageCodegenExec, RowDataSourceScanExec}
import org.apache.spark.sql.test.SharedSQLContext
import org.apache.spark.sql.types._
import org.apache.spark.tags.DockerTest
......@@ -255,6 +256,18 @@ class OracleIntegrationSuite extends DockerJDBCIntegrationSuite with SharedSQLCo
val df = dfRead.filter(dfRead.col("date_type").lt(dt))
.filter(dfRead.col("timestamp_type").lt(ts))
val parentPlan = df.queryExecution.executedPlan
assert(parentPlan.isInstanceOf[WholeStageCodegenExec])
val node = parentPlan.asInstanceOf[WholeStageCodegenExec]
val metadata = node.child.asInstanceOf[RowDataSourceScanExec].metadata
// The "PushedFilters" part should exist in Dataframe's
// physical plan and the existence of right literals in
// "PushedFilters" is used to prove that the predicates
// pushing down have been effective.
assert(metadata.get("PushedFilters").isDefined)
assert(metadata("PushedFilters").contains(dt.toString))
assert(metadata("PushedFilters").contains(ts.toString))
val row = df.collect()(0)
assert(row.getDate(0).equals(dateVal))
assert(row.getTimestamp(1).equals(timestampVal))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment