Skip to content
  • Hari Shreedharan's avatar
    8c5a2226
    [SPARK-3054][STREAMING] Add unit tests for Spark Sink. · 8c5a2226
    Hari Shreedharan authored
    This patch adds unit tests for Spark Sink.
    
    It also removes the private[flume] for Spark Sink,
    since the sink is instantiated from Flume configuration (looks like this is ignored by reflection which is used by
    Flume, but we should still remove it anyway).
    
    Author: Hari Shreedharan <hshreedharan@apache.org>
    Author: Hari Shreedharan <hshreedharan@cloudera.com>
    
    Closes #1958 from harishreedharan/spark-sink-test and squashes the following commits:
    
    e3110b9 [Hari Shreedharan] Add a sleep to allow sink to commit the transactions
    120b81e [Hari Shreedharan] Fix complexity in threading model in test
    4df5be6 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test
    c9190d1 [Hari Shreedharan] Indentation and spaces changes
    7fedc5a [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test
    abc20cb [Hari Shreedharan] Minor test changes
    7b9b649 [Hari Shreedharan] Merge branch 'master' into spark-sink-test
    f2c56c9 [Hari Shreedharan] Update SparkSinkSuite.scala
    a24aac8 [Hari Shreedharan] Remove unused var
    c86d615 [Hari Shreedharan] [SPARK-3054][STREAMING] Add unit tests for Spark Sink.
    8c5a2226
    [SPARK-3054][STREAMING] Add unit tests for Spark Sink.
    Hari Shreedharan authored
    This patch adds unit tests for Spark Sink.
    
    It also removes the private[flume] for Spark Sink,
    since the sink is instantiated from Flume configuration (looks like this is ignored by reflection which is used by
    Flume, but we should still remove it anyway).
    
    Author: Hari Shreedharan <hshreedharan@apache.org>
    Author: Hari Shreedharan <hshreedharan@cloudera.com>
    
    Closes #1958 from harishreedharan/spark-sink-test and squashes the following commits:
    
    e3110b9 [Hari Shreedharan] Add a sleep to allow sink to commit the transactions
    120b81e [Hari Shreedharan] Fix complexity in threading model in test
    4df5be6 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test
    c9190d1 [Hari Shreedharan] Indentation and spaces changes
    7fedc5a [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into spark-sink-test
    abc20cb [Hari Shreedharan] Minor test changes
    7b9b649 [Hari Shreedharan] Merge branch 'master' into spark-sink-test
    f2c56c9 [Hari Shreedharan] Update SparkSinkSuite.scala
    a24aac8 [Hari Shreedharan] Remove unused var
    c86d615 [Hari Shreedharan] [SPARK-3054][STREAMING] Add unit tests for Spark Sink.
Loading