python/pyspark/sql/tests.py · 35438fb0ad3bcda5c5a3a0ccde1a620699d012db · cs525-sp18-g07 / spark

9 years ago

[SPARK-16175] [PYSPARK] handle None for UDT · 35438fb0

Davies Liu authored 9 years ago

## What changes were proposed in this pull request?

Scala UDT will bypass all the null and will not pass them into serialize() and deserialize() of UDT, this PR update the Python UDT to do this as well.

## How was this patch tested?

Added tests.

Author: Davies Liu <davies@databricks.com>

Closes #13878 from davies/udt_null.

35438fb0

History

[SPARK-16175] [PYSPARK] handle None for UDT

Davies Liu authored 9 years ago

## What changes were proposed in this pull request?

Scala UDT will bypass all the null and will not pass them into serialize() and deserialize() of UDT, this PR update the Python UDT to do this as well.

## How was this patch tested?

Added tests.

Author: Davies Liu <davies@databricks.com>

Closes #13878 from davies/udt_null.

tests.py 77.14 KiB