Skip to content
Snippets Groups Projects
  1. Sep 03, 2016
    • Herman van Hovell's avatar
      [SPARK-17335][SQL] Fix ArrayType and MapType CatalogString. · c2a1576c
      Herman van Hovell authored
      ## What changes were proposed in this pull request?
      the `catalogString` for `ArrayType` and `MapType` currently calls the `simpleString` method on its children. This is a problem when the child is a struct, the `struct.simpleString` implementation truncates the number of fields it shows (25 at max). This breaks the generation of a proper `catalogString`, and has shown to cause errors while writing to Hive.
      
      This PR fixes this by providing proper `catalogString` implementations for `ArrayData` or `MapData`.
      
      ## How was this patch tested?
      Added testing for `catalogString` to `DataTypeSuite`.
      
      Author: Herman van Hovell <hvanhovell@databricks.com>
      
      Closes #14938 from hvanhovell/SPARK-17335.
      c2a1576c
  2. Jun 11, 2016
    • Eric Liang's avatar
      [SPARK-15881] Update microbenchmark results for WideSchemaBenchmark · 5bb4564c
      Eric Liang authored
      ## What changes were proposed in this pull request?
      
      These were not updated after performance improvements. To make updating them easier, I also moved the results from inline comments out into a file, which is auto-generated when the benchmark is re-run.
      
      Author: Eric Liang <ekl@databricks.com>
      
      Closes #13607 from ericl/sc-3538.
      5bb4564c
Loading