Skip to content
Snippets Groups Projects
Commit 511a7314 authored by Michael Armbrust's avatar Michael Armbrust
Browse files

[SPARK-2561][SQL] Fix apply schema

We need to use the analyzed attributes otherwise we end up with a tree that will never resolve.

Author: Michael Armbrust <michael@databricks.com>

Closes #1470 from marmbrus/fixApplySchema and squashes the following commits:

f968195 [Michael Armbrust] Use analyzed attributes when applying the schema.
4969015 [Michael Armbrust] Add test case.
parent a4d60208
No related branches found
No related tags found
No related merge requests found
......@@ -430,7 +430,7 @@ class SchemaRDD(
* @group schema
*/
private def applySchema(rdd: RDD[Row]): SchemaRDD = {
new SchemaRDD(sqlContext, SparkLogicalPlan(ExistingRdd(logicalPlan.output, rdd)))
new SchemaRDD(sqlContext, SparkLogicalPlan(ExistingRdd(queryExecution.analyzed.output, rdd)))
}
// =======================================================================
......
......@@ -33,6 +33,12 @@ class DslQuerySuite extends QueryTest {
testData.collect().toSeq)
}
test("repartition") {
checkAnswer(
testData.select('key).repartition(10).select('key),
testData.select('key).collect().toSeq)
}
test("agg") {
checkAnswer(
testData2.groupBy('a)('a, Sum('b)),
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment