Skip to content
Snippets Groups Projects
Commit 9f3273bd authored by Yadong Qi's avatar Yadong Qi Committed by Cheng Lian
Browse files

[SPARK-6397][SQL] Check the missingInput simply

https://github.com/apache/spark/pull/5082

/cc liancheng

Author: Yadong Qi <qiyadong2010@gmail.com>

Closes #5132 from watermen/sql-missingInput-new and squashes the following commits:

1e5bdc5 [Yadong Qi] Check the missingInput simply
parent bf044def
No related branches found
No related tags found
No related merge requests found
...@@ -85,9 +85,8 @@ class CheckAnalysis { ...@@ -85,9 +85,8 @@ class CheckAnalysis {
cleaned.foreach(checkValidAggregateExpression) cleaned.foreach(checkValidAggregateExpression)
case o if o.children.nonEmpty && case o if o.children.nonEmpty && o.missingInput.nonEmpty =>
!o.references.filter(_.name != "grouping__id").subsetOf(o.inputSet) => val missingAttributes = o.missingInput.map(_.prettyString).mkString(",")
val missingAttributes = (o.references -- o.inputSet).map(_.prettyString).mkString(",")
val input = o.inputSet.map(_.prettyString).mkString(",") val input = o.inputSet.map(_.prettyString).mkString(",")
failAnalysis(s"resolved attributes $missingAttributes missing from $input") failAnalysis(s"resolved attributes $missingAttributes missing from $input")
......
...@@ -17,7 +17,7 @@ ...@@ -17,7 +17,7 @@
package org.apache.spark.sql.catalyst.plans package org.apache.spark.sql.catalyst.plans
import org.apache.spark.sql.catalyst.expressions.{Attribute, AttributeSet, Expression} import org.apache.spark.sql.catalyst.expressions.{VirtualColumn, Attribute, AttributeSet, Expression}
import org.apache.spark.sql.catalyst.trees.TreeNode import org.apache.spark.sql.catalyst.trees.TreeNode
import org.apache.spark.sql.types.{ArrayType, DataType, StructField, StructType} import org.apache.spark.sql.types.{ArrayType, DataType, StructField, StructType}
...@@ -48,7 +48,8 @@ abstract class QueryPlan[PlanType <: TreeNode[PlanType]] extends TreeNode[PlanTy ...@@ -48,7 +48,8 @@ abstract class QueryPlan[PlanType <: TreeNode[PlanType]] extends TreeNode[PlanTy
* Subclasses should override this method if they produce attributes internally as it is used by * Subclasses should override this method if they produce attributes internally as it is used by
* assertions designed to prevent the construction of invalid plans. * assertions designed to prevent the construction of invalid plans.
*/ */
def missingInput: AttributeSet = references -- inputSet def missingInput: AttributeSet = (references -- inputSet)
.filter(_.name != VirtualColumn.groupingIdName)
/** /**
* Runs [[transform]] with `rule` on all expressions present in this query operator. * Runs [[transform]] with `rule` on all expressions present in this query operator.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment