Skip to content
Snippets Groups Projects
Commit b3862d3c authored by Liang-Chi Hsieh's avatar Liang-Chi Hsieh Committed by Yin Huai
Browse files

[SPARK-10705] [SQL] Avoid using external rows in DataFrame.toJSON 

JIRA: https://issues.apache.org/jira/browse/SPARK-10705

As described in the JIRA ticket, `DataFrame.toJSON` uses `DataFrame.mapPartitions`, which converts internal rows to external rows. We should use `queryExecution.toRdd.mapPartitions` that directly uses internal rows for better performance.

Author: Liang-Chi Hsieh <viirya@appier.com>

Closes #8865 from viirya/df-tojson-internalrow.
parent 341b13f8
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment