Skip to content
Snippets Groups Projects
Commit c07838b5 authored by Dennis Huo's avatar Dennis Huo Committed by Michael Armbrust
Browse files

[SPARK-9206] [SQL] Fix HiveContext classloading for GCS connector.

IsolatedClientLoader.isSharedClass includes all of com.google.\*, presumably
for Guava, protobuf, and/or other shared Google libraries, but needs to
count com.google.cloud.\* as "hive classes" when determining which ClassLoader
to use. Otherwise, things like HiveContext.parquetFile will throw a
ClassCastException when fs.defaultFS is set to a Google Cloud Storage (gs://)
path. On StackOverflow: http://stackoverflow.com/questions/31478955

EDIT: Adding yhuai who worked on the relevant classloading isolation pieces.

Author: Dennis Huo <dhuo@google.com>

Closes #7549 from dennishuo/dhuo-fix-hivecontext-gcs and squashes the following commits:

1f8db07 [Dennis Huo] Fix HiveContext classloading for GCS connector.
parent 60c0ce13
No related branches found
No related tags found
No related merge requests found
......@@ -125,7 +125,7 @@ private[hive] class IsolatedClientLoader(
name.contains("log4j") ||
name.startsWith("org.apache.spark.") ||
name.startsWith("scala.") ||
name.startsWith("com.google") ||
(name.startsWith("com.google") && !name.startsWith("com.google.cloud")) ||
name.startsWith("java.lang.") ||
name.startsWith("java.net") ||
sharedPrefixes.exists(name.startsWith)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment