Skip to content
Snippets Groups Projects
  • Josh Rosen's avatar
    1f1819b2
    [SPARK-3114] [PySpark] Fix Python UDFs in Spark SQL. · 1f1819b2
    Josh Rosen authored
    This fixes SPARK-3114, an issue where we inadvertently broke Python UDFs in Spark SQL.
    
    This PR modifiers the test runner script to always run the PySpark SQL tests, irrespective of whether SparkSQL itself has been modified.  It also includes Davies' fix for the bug.
    
    Closes #2026.
    
    Author: Josh Rosen <joshrosen@apache.org>
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #2027 from JoshRosen/pyspark-sql-fix and squashes the following commits:
    
    9af2708 [Davies Liu] bugfix: disable compression of command
    0d8d3a4 [Josh Rosen] Always run Python Spark SQL tests.
    1f1819b2
    History
    [SPARK-3114] [PySpark] Fix Python UDFs in Spark SQL.
    Josh Rosen authored
    This fixes SPARK-3114, an issue where we inadvertently broke Python UDFs in Spark SQL.
    
    This PR modifiers the test runner script to always run the PySpark SQL tests, irrespective of whether SparkSQL itself has been modified.  It also includes Davies' fix for the bug.
    
    Closes #2026.
    
    Author: Josh Rosen <joshrosen@apache.org>
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #2027 from JoshRosen/pyspark-sql-fix and squashes the following commits:
    
    9af2708 [Davies Liu] bugfix: disable compression of command
    0d8d3a4 [Josh Rosen] Always run Python Spark SQL tests.