-
- Downloads
[SPARK-18998][SQL] Add a cbo conf to switch between default statistics and estimated statistics
## What changes were proposed in this pull request? We add a cbo configuration to switch between default stats and estimated stats. We also define a new statistics method `planStats` in LogicalPlan with conf as its parameter, in order to pass the cbo switch and other estimation related configurations in the future. `planStats` is used on the caller sides (i.e. in Optimizer and Strategies) to make transformation decisions based on stats. ## How was this patch tested? Add a test case using a dummy LogicalPlan. Author: Zhenhua Wang <wzh_zju@163.com> Closes #16401 from wzhfy/cboSwitch.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystConf.scala 6 additions, 0 deletions...in/scala/org/apache/spark/sql/catalyst/CatalystConf.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 3 additions, 3 deletions...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala 24 additions, 0 deletions...apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/LimitPushdownSuite.scala 2 additions, 1 deletion...che/spark/sql/catalyst/optimizer/LimitPushdownSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 6 additions, 6 deletions...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 9 additions, 0 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/test/scala/org/apache/spark/sql/statsEstimation/StatsEstimationSuite.scala 67 additions, 0 deletions...ache/spark/sql/statsEstimation/StatsEstimationSuite.scala
Loading
Please register or sign in to comment