Skip to content
Snippets Groups Projects
  • Sheamus K. Parkes's avatar
    e7791c4f
    [SPARK-13842] [PYSPARK] pyspark.sql.types.StructType accessor enhancements · e7791c4f
    Sheamus K. Parkes authored
    ## What changes were proposed in this pull request?
    
    Expand the possible ways to interact with the contents of a `pyspark.sql.types.StructType` instance.
      - Iterating a `StructType` will iterate its fields
        - `[field.name for field in my_structtype]`
      - Indexing with a string will return a field by name
        - `my_structtype['my_field_name']`
      - Indexing with an integer will return a field by position
        - `my_structtype[0]`
      - Indexing with a slice will return a new `StructType` with just the chosen fields:
        - `my_structtype[1:3]`
      - The length is the number of fields (should also provide "truthiness" for free)
        - `len(my_structtype) == 2`
    
    ## How was this patch tested?
    
    Extended the unit test coverage in the accompanying `tests.py`.
    
    Author: Sheamus K. Parkes <shea.parkes@milliman.com>
    
    Closes #12251 from skparkes/pyspark-structtype-enhance.
    e7791c4f
    History
    [SPARK-13842] [PYSPARK] pyspark.sql.types.StructType accessor enhancements
    Sheamus K. Parkes authored
    ## What changes were proposed in this pull request?
    
    Expand the possible ways to interact with the contents of a `pyspark.sql.types.StructType` instance.
      - Iterating a `StructType` will iterate its fields
        - `[field.name for field in my_structtype]`
      - Indexing with a string will return a field by name
        - `my_structtype['my_field_name']`
      - Indexing with an integer will return a field by position
        - `my_structtype[0]`
      - Indexing with a slice will return a new `StructType` with just the chosen fields:
        - `my_structtype[1:3]`
      - The length is the number of fields (should also provide "truthiness" for free)
        - `len(my_structtype) == 2`
    
    ## How was this patch tested?
    
    Extended the unit test coverage in the accompanying `tests.py`.
    
    Author: Sheamus K. Parkes <shea.parkes@milliman.com>
    
    Closes #12251 from skparkes/pyspark-structtype-enhance.