-
- Downloads
[SPARK-7469] [SQL] DAG visualization: show SQL query operators
The DAG visualization currently displays only low-level Spark primitives (e.g. `map`, `reduceByKey`, `filter` etc.). For SQL, these aren't particularly useful. Instead, we should display higher level physical operators (e.g. `Filter`, `Exchange`, `ShuffleHashJoin`). cc marmbrus ----------------- **Before** <img src="https://issues.apache.org/jira/secure/attachment/12731586/before.png" width="600px"/> ----------------- **After** (Pay attention to the words) <img src="https://issues.apache.org/jira/secure/attachment/12731587/after.png" width="600px"/> ----------------- Author: Andrew Or <andrew@databricks.com> Closes #5999 from andrewor14/dag-viz-sql and squashes the following commits: 0db23a4 [Andrew Or] Merge branch 'master' of github.com:apache/spark into dag-viz-sql 1e211db [Andrew Or] Update comment 0d49fd6 [Andrew Or] Merge branch 'master' of github.com:apache/spark into dag-viz-sql ffd237a [Andrew Or] Fix style 202dac1 [Andrew Or] Make ignoreParent false by default e61b1ab [Andrew Or] Visualize SQL operators, not low-level Spark primitives 569034a [Andrew Or] Add a flag to ignore parent settings and scopes
Showing
- core/src/main/scala/org/apache/spark/rdd/RDDOperationScope.scala 14 additions, 6 deletions...c/main/scala/org/apache/spark/rdd/RDDOperationScope.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/InMemoryColumnarTableScan.scala 1 addition, 1 deletion...apache/spark/sql/columnar/InMemoryColumnarTableScan.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Aggregate.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/execution/Aggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Exchange.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/sql/execution/Exchange.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/ExistingRDD.scala 1 addition, 1 deletion...in/scala/org/apache/spark/sql/execution/ExistingRDD.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Expand.scala 1 addition, 1 deletion...rc/main/scala/org/apache/spark/sql/execution/Expand.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Generate.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/sql/execution/Generate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/GeneratedAggregate.scala 1 addition, 1 deletion...a/org/apache/spark/sql/execution/GeneratedAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/LocalTableScan.scala 1 addition, 1 deletion...scala/org/apache/spark/sql/execution/LocalTableScan.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala 15 additions, 4 deletions...main/scala/org/apache/spark/sql/execution/SparkPlan.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Window.scala 1 addition, 1 deletion...rc/main/scala/org/apache/spark/sql/execution/Window.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala 13 additions, 13 deletions...scala/org/apache/spark/sql/execution/basicOperators.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/commands.scala 1 addition, 1 deletion.../main/scala/org/apache/spark/sql/execution/commands.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala 2 additions, 2 deletions.../scala/org/apache/spark/sql/execution/debug/package.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala 1 addition, 1 deletion.../apache/spark/sql/execution/joins/BroadcastHashJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala 1 addition, 1 deletion...spark/sql/execution/joins/BroadcastLeftSemiJoinHash.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastNestedLoopJoin.scala 1 addition, 1 deletion...e/spark/sql/execution/joins/BroadcastNestedLoopJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/CartesianProduct.scala 1 addition, 1 deletion...g/apache/spark/sql/execution/joins/CartesianProduct.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashOuterJoin.scala 1 addition, 1 deletion.../org/apache/spark/sql/execution/joins/HashOuterJoin.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/LeftSemiJoinBNL.scala 1 addition, 1 deletion...rg/apache/spark/sql/execution/joins/LeftSemiJoinBNL.scala
Loading
Please register or sign in to comment