Skip to content
Snippets Groups Projects
Commit e77fa81a authored by Davies Liu's avatar Davies Liu Committed by Josh Rosen
Browse files

[SPARK-3554] [PySpark] use broadcast automatically for large closure

Py4j can not handle large string efficiently, so we should use broadcast for large closure automatically. (Broadcast use local filesystem to pass through data).

Author: Davies Liu <davies.liu@gmail.com>

Closes #2417 from davies/command and squashes the following commits:

fbf4e97 [Davies Liu] bugfix
aefd508 [Davies Liu] use broadcast automatically for large closure
parent 9306297d
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment