Skip to content
Snippets Groups Projects
Commit 6cbf83c0 authored by Cheng Hao's avatar Cheng Hao Committed by Michael Armbrust
Browse files

[SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test


Logically, we should remove the Hive Table/Database first and then reset the Hive configuration, repoint to the new data warehouse directory etc.
Otherwise it raised exceptions like "Database doesn't not exists: default" in the local testing.

Author: Cheng Hao <hao.cheng@intel.com>

Closes #2352 from chenghao-intel/test_hive and squashes the following commits:

74fd76b [Cheng Hao] eliminate the error log

(cherry picked from commit 8194fc66)
Signed-off-by: default avatarMichael Armbrust <michael@databricks.com>
parent f17b7957
No related branches found
No related tags found
No related merge requests found
......@@ -309,15 +309,6 @@ class TestHiveContext(sc: SparkContext) extends HiveContext(sc) {
log.asInstanceOf[org.apache.log4j.Logger].setLevel(org.apache.log4j.Level.WARN)
}
// It is important that we RESET first as broken hooks that might have been set could break
// other sql exec here.
runSqlHive("RESET")
// For some reason, RESET does not reset the following variables...
runSqlHive("set datanucleus.cache.collections=true")
runSqlHive("set datanucleus.cache.collections.lazy=true")
// Lots of tests fail if we do not change the partition whitelist from the default.
runSqlHive("set hive.metastore.partition.name.whitelist.pattern=.*")
loadedTables.clear()
catalog.client.getAllTables("default").foreach { t =>
logDebug(s"Deleting table $t")
......@@ -343,6 +334,14 @@ class TestHiveContext(sc: SparkContext) extends HiveContext(sc) {
FunctionRegistry.unregisterTemporaryUDF(udfName)
}
// It is important that we RESET first as broken hooks that might have been set could break
// other sql exec here.
runSqlHive("RESET")
// For some reason, RESET does not reset the following variables...
runSqlHive("set datanucleus.cache.collections=true")
runSqlHive("set datanucleus.cache.collections.lazy=true")
// Lots of tests fail if we do not change the partition whitelist from the default.
runSqlHive("set hive.metastore.partition.name.whitelist.pattern=.*")
configure()
runSqlHive("USE default")
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment