Skip to content
Snippets Groups Projects
  • Wenchen Fan's avatar
    1150a19b
    [SPARK-8670] [SQL] Nested columns can't be referenced in pyspark · 1150a19b
    Wenchen Fan authored
    This bug is caused by a wrong column-exist-check in `__getitem__` of pyspark dataframe. `DataFrame.apply` accepts not only top level column names, but also nested column name like `a.b`, so we should remove that check from `__getitem__`.
    
    Author: Wenchen Fan <cloud0fan@outlook.com>
    
    Closes #8202 from cloud-fan/nested.
    1150a19b
    History
    [SPARK-8670] [SQL] Nested columns can't be referenced in pyspark
    Wenchen Fan authored
    This bug is caused by a wrong column-exist-check in `__getitem__` of pyspark dataframe. `DataFrame.apply` accepts not only top level column names, but also nested column name like `a.b`, so we should remove that check from `__getitem__`.
    
    Author: Wenchen Fan <cloud0fan@outlook.com>
    
    Closes #8202 from cloud-fan/nested.