Skip to content
Snippets Groups Projects
Commit fc3f77b4 authored by navis.ryu's avatar navis.ryu Committed by Yin Huai
Browse files

[SPARK-11614][SQL] serde parameters should be set only when all params are ready

see HIVE-7975 and HIVE-12373

With changed semantic of setters in thrift objects in hive, setter should be called only after all parameters are set. It's not problem of current state but will be a problem in some day.

Author: navis.ryu <navis@apache.org>

Closes #9580 from navis/SPARK-11614.
parent 67c75828
No related branches found
No related tags found
No related merge requests found
......@@ -804,12 +804,13 @@ private[hive] case class MetastoreRelation
val serdeInfo = new org.apache.hadoop.hive.metastore.api.SerDeInfo
sd.setSerdeInfo(serdeInfo)
// maps and lists should be set only after all elements are ready (see HIVE-7975)
serdeInfo.setSerializationLib(p.storage.serde)
val serdeParameters = new java.util.HashMap[String, String]()
serdeInfo.setParameters(serdeParameters)
table.serdeProperties.foreach { case (k, v) => serdeParameters.put(k, v) }
p.storage.serdeProperties.foreach { case (k, v) => serdeParameters.put(k, v) }
serdeInfo.setParameters(serdeParameters)
new Partition(hiveQlTable, tPartition)
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment