Skip to content
Snippets Groups Projects
Commit bfeedc69 authored by Cheng Lian's avatar Cheng Lian
Browse files

[SPARK-7684] [SQL] Invoking HiveContext.newTemporaryConfiguration() shouldn't...

[SPARK-7684] [SQL] Invoking HiveContext.newTemporaryConfiguration() shouldn't create new metastore directory

The "Database does not exist" error reported in SPARK-7684 was caused by `HiveContext.newTemporaryConfiguration()`, which always creates a new temporary metastore directory and returns a metastore configuration pointing that directory. This makes `TestHive.reset()` always replaces old temporary metastore with an empty new one.

Author: Cheng Lian <lian@databricks.com>

Closes #6359 from liancheng/spark-7684 and squashes the following commits:

95d2eb8 [Cheng Lian] Addresses @marmbrust's comment
042769d [Cheng Lian] Don't create new temp directory in HiveContext.newTemporaryConfiguration()
parent fd31fd49
No related branches found
No related tags found
No related merge requests found
......@@ -158,7 +158,7 @@ class HiveContext(sc: SparkContext) extends SQLContext(sc) {
*/
@transient
protected[hive] lazy val executionHive: ClientWrapper = {
logInfo(s"Initilizing execution hive, version $hiveExecutionVersion")
logInfo(s"Initializing execution hive, version $hiveExecutionVersion")
new ClientWrapper(
version = IsolatedClientLoader.hiveVersion(hiveExecutionVersion),
config = newTemporaryConfiguration())
......
......@@ -82,9 +82,11 @@ class TestHiveContext(sc: SparkContext) extends HiveContext(sc) {
lazy val warehousePath = Utils.createTempDir()
private lazy val temporaryConfig = newTemporaryConfiguration()
/** Sets up the system initially or after a RESET command */
protected override def configure(): Map[String, String] =
newTemporaryConfiguration() ++ Map("hive.metastore.warehouse.dir" -> warehousePath.toString)
temporaryConfig ++ Map("hive.metastore.warehouse.dir" -> warehousePath.toString)
val testTempDir = Utils.createTempDir()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment