Skip to content
Snippets Groups Projects
  • Burak Yavuz's avatar
    78c8aaf8
    [SPARK-14555] Second cut of Python API for Structured Streaming · 78c8aaf8
    Burak Yavuz authored
    ## What changes were proposed in this pull request?
    
    This PR adds Python APIs for:
     - `ContinuousQueryManager`
     - `ContinuousQueryException`
    
    The `ContinuousQueryException` is a very basic wrapper, it doesn't provide the functionality that the Scala side provides, but it follows the same pattern for `AnalysisException`.
    
    For `ContinuousQueryManager`, all APIs are provided except for registering listeners.
    
    This PR also attempts to fix test flakiness by stopping all active streams just before tests.
    
    ## How was this patch tested?
    
    Python Doc tests and unit tests
    
    Author: Burak Yavuz <brkyvz@gmail.com>
    
    Closes #12673 from brkyvz/pyspark-cqm.
    78c8aaf8
    History
    [SPARK-14555] Second cut of Python API for Structured Streaming
    Burak Yavuz authored
    ## What changes were proposed in this pull request?
    
    This PR adds Python APIs for:
     - `ContinuousQueryManager`
     - `ContinuousQueryException`
    
    The `ContinuousQueryException` is a very basic wrapper, it doesn't provide the functionality that the Scala side provides, but it follows the same pattern for `AnalysisException`.
    
    For `ContinuousQueryManager`, all APIs are provided except for registering listeners.
    
    This PR also attempts to fix test flakiness by stopping all active streams just before tests.
    
    ## How was this patch tested?
    
    Python Doc tests and unit tests
    
    Author: Burak Yavuz <brkyvz@gmail.com>
    
    Closes #12673 from brkyvz/pyspark-cqm.