Skip to content
Snippets Groups Projects
Commit f4e8c289 authored by scwf's avatar scwf Committed by Michael Armbrust
Browse files

[SPARK-4042][SQL] Append columns ids and names before broadcast

Append columns ids and names before broadcast ```hiveExtraConf```  in ```HadoopTableReader```.

Author: scwf <wangfei1@huawei.com>

Closes #2885 from scwf/HadoopTableReader and squashes the following commits:

a8c498c [scwf] append columns ids and names before broadcast
parent 3a9d66cf
No related branches found
No related tags found
No related merge requests found
......@@ -68,6 +68,9 @@ case class HiveTableScan(
@transient
private[this] val hiveExtraConf = new HiveConf(context.hiveconf)
// append columns ids and names before broadcast
addColumnMetadataToConf(hiveExtraConf)
@transient
private[this] val hadoopReader =
new HadoopTableReader(attributes, relation, context, hiveExtraConf)
......@@ -105,8 +108,6 @@ case class HiveTableScan(
hiveConf.set(serdeConstants.LIST_COLUMNS, relation.attributes.map(_.name).mkString(","))
}
addColumnMetadataToConf(hiveExtraConf)
/**
* Prunes partitions not involve the query plan.
*
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment