Skip to content
Snippets Groups Projects
Commit 314e48a3 authored by Michael Armbrust's avatar Michael Armbrust Committed by Wenchen Fan
Browse files

[SPARK-18055][SQL] Use correct mirror in ExpresionEncoder

Previously, we were using the mirror of passed in `TypeTag` when reflecting to build an encoder.  This fails when the outer class is built in (i.e. `Seq`'s default mirror is based on root classloader) but inner classes (i.e. `A` in `Seq[A]`) are defined in the REPL or a library.

This patch changes us to always reflect based on a mirror created using the context classloader.

Author: Michael Armbrust <michael@databricks.com>

Closes #17201 from marmbrus/replSeqEncoder.
parent 56e1bd33
No related branches found
No related tags found
No related merge requests found
......@@ -473,4 +473,15 @@ class ReplSuite extends SparkFunSuite {
assertDoesNotContain("AssertionError", output)
assertDoesNotContain("Exception", output)
}
test("newProductSeqEncoder with REPL defined class") {
val output = runInterpreterInPasteMode("local-cluster[1,4,4096]",
"""
|case class Click(id: Int)
|spark.implicits.newProductSeqEncoder[Click]
""".stripMargin)
assertDoesNotContain("error:", output)
assertDoesNotContain("Exception", output)
}
}
......@@ -45,8 +45,8 @@ import org.apache.spark.util.Utils
object ExpressionEncoder {
def apply[T : TypeTag](): ExpressionEncoder[T] = {
// We convert the not-serializable TypeTag into StructType and ClassTag.
val mirror = typeTag[T].mirror
val tpe = typeTag[T].tpe
val mirror = ScalaReflection.mirror
val tpe = typeTag[T].in(mirror).tpe
if (ScalaReflection.optionOfProductType(tpe)) {
throw new UnsupportedOperationException(
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment