Skip to content
Snippets Groups Projects
Commit 1fbe2785 authored by Davies Liu's avatar Davies Liu Committed by Reynold Xin
Browse files

[SPARK-15255][SQL] limit the length of name for cached DataFrame

## What changes were proposed in this pull request?

We use the tree string of an SparkPlan as the name of cached DataFrame, that could be very long, cause the browser to be not responsive. This PR will limit the length of the name to 1000 characters.

## How was this patch tested?

Here is how the UI looks right now:

![ui](https://cloud.githubusercontent.com/assets/40902/15163355/d5640f9c-16bc-11e6-8655-809af8a4fed1.png)

Author: Davies Liu <davies@databricks.com>

Closes #13033 from davies/cache_name.
parent 66554596
No related branches found
No related tags found
No related merge requests found
......@@ -19,6 +19,8 @@ package org.apache.spark.sql.execution.columnar
import scala.collection.mutable.ArrayBuffer
import org.apache.commons.lang.StringUtils
import org.apache.spark.{Accumulable, Accumulator}
import org.apache.spark.network.util.JavaUtils
import org.apache.spark.rdd.RDD
......@@ -177,7 +179,9 @@ private[sql] case class InMemoryRelation(
}
}.persist(storageLevel)
cached.setName(tableName.map(n => s"In-memory table $n").getOrElse(child.toString))
cached.setName(
tableName.map(n => s"In-memory table $n")
.getOrElse(StringUtils.abbreviate(child.toString, 1024)))
_cachedColumnBuffers = cached
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment