Skip to content
Snippets Groups Projects
Commit 48b459dd authored by Josh Rosen's avatar Josh Rosen Committed by Reynold Xin
Browse files

[SPARK-17301][SQL] Remove unused classTag field from AtomicType base class

There's an unused `classTag` val in the AtomicType base class which is causing unnecessary slowness in deserialization because it needs to grab ScalaReflectionLock and create a new runtime reflection mirror. Removing this unused code gives a small but measurable performance boost in SQL task deserialization.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #14869 from JoshRosen/remove-unused-classtag.
parent 736a7911
No related branches found
No related tags found
No related merge requests found
......@@ -17,13 +17,10 @@
package org.apache.spark.sql.types
import scala.reflect.ClassTag
import scala.reflect.runtime.universe.{runtimeMirror, TypeTag}
import scala.reflect.runtime.universe.TypeTag
import org.apache.spark.annotation.DeveloperApi
import org.apache.spark.sql.catalyst.ScalaReflectionLock
import org.apache.spark.sql.catalyst.expressions.Expression
import org.apache.spark.util.Utils
/**
* A non-concrete data type, reserved for internal uses.
......@@ -130,11 +127,6 @@ protected[sql] abstract class AtomicType extends DataType {
private[sql] type InternalType
private[sql] val tag: TypeTag[InternalType]
private[sql] val ordering: Ordering[InternalType]
@transient private[sql] val classTag = ScalaReflectionLock.synchronized {
val mirror = runtimeMirror(Utils.getSparkClassLoader)
ClassTag[InternalType](mirror.runtimeClass(tag.tpe))
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment