Skip to content
  • Burak Yavuz's avatar
    26062d22
    [SPARK-11198][STREAMING][KINESIS] Support de-aggregation of records during recovery · 26062d22
    Burak Yavuz authored
    While the KCL handles de-aggregation during the regular operation, during recovery we use the lower level api, and therefore need to de-aggregate the records.
    
    tdas Testing is an issue, we need protobuf magic to do the aggregated records. Maybe we could depend on KPL for tests?
    
    Author: Burak Yavuz <brkyvz@gmail.com>
    
    Closes #9403 from brkyvz/kinesis-deaggregation.
    26062d22
    [SPARK-11198][STREAMING][KINESIS] Support de-aggregation of records during recovery
    Burak Yavuz authored
    While the KCL handles de-aggregation during the regular operation, during recovery we use the lower level api, and therefore need to de-aggregate the records.
    
    tdas Testing is an issue, we need protobuf magic to do the aggregated records. Maybe we could depend on KPL for tests?
    
    Author: Burak Yavuz <brkyvz@gmail.com>
    
    Closes #9403 from brkyvz/kinesis-deaggregation.
Loading