Skip to content
Snippets Groups Projects
Commit 2e0c0845 authored by Yin Huai's avatar Yin Huai Committed by Michael Armbrust
Browse files

[SPARK-5789][SQL]Throw a better error message if JsonRDD.parseJson encounters...

[SPARK-5789][SQL]Throw a better error message if JsonRDD.parseJson encounters unrecoverable parsing errors.

Author: Yin Huai <yhuai@databricks.com>

Closes #4582 from yhuai/jsonErrorMessage and squashes the following commits:

152dbd4 [Yin Huai] Update error message.
1466256 [Yin Huai] Throw a better error message when a JSON object in the input dataset span multiple records (lines for files or strings for an RDD of strings).
parent 2cbb3e43
No related branches found
No related tags found
No related merge requests found
......@@ -306,6 +306,10 @@ private[sql] object JsonRDD extends Logging {
val parsed = mapper.readValue(record, classOf[Object]) match {
case map: java.util.Map[_, _] => scalafy(map).asInstanceOf[Map[String, Any]] :: Nil
case list: java.util.List[_] => scalafy(list).asInstanceOf[Seq[Map[String, Any]]]
case _ =>
sys.error(
s"Failed to parse record $record. Please make sure that each line of the file " +
"(or each string in the RDD) is a valid JSON object or an array of JSON objects.")
}
parsed
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment