Skip to content
Snippets Groups Projects
  • Marcelo Vanzin's avatar
    0cba4951
    [SPARK-20641][CORE] Add key-value store abstraction and LevelDB implementation. · 0cba4951
    Marcelo Vanzin authored
    This change adds an abstraction and LevelDB implementation for a key-value
    store that will be used to store UI and SHS data.
    
    The interface is described in KVStore.java (see javadoc). Specifics
    of the LevelDB implementation are discussed in the javadocs of both
    LevelDB.java and LevelDBTypeInfo.java.
    
    Included also are a few small benchmarks just to get some idea of
    latency. Because they're too slow for regular unit test runs, they're
    disabled by default.
    
    Tested with the included unit tests, and also as part of the overall feature
    implementation (including running SHS with hundreds of apps).
    
    Author: Marcelo Vanzin <vanzin@cloudera.com>
    
    Closes #17902 from vanzin/shs-ng/M1.
    0cba4951
    History
    [SPARK-20641][CORE] Add key-value store abstraction and LevelDB implementation.
    Marcelo Vanzin authored
    This change adds an abstraction and LevelDB implementation for a key-value
    store that will be used to store UI and SHS data.
    
    The interface is described in KVStore.java (see javadoc). Specifics
    of the LevelDB implementation are discussed in the javadocs of both
    LevelDB.java and LevelDBTypeInfo.java.
    
    Included also are a few small benchmarks just to get some idea of
    latency. Because they're too slow for regular unit test runs, they're
    disabled by default.
    
    Tested with the included unit tests, and also as part of the overall feature
    implementation (including running SHS with hundreds of apps).
    
    Author: Marcelo Vanzin <vanzin@cloudera.com>
    
    Closes #17902 from vanzin/shs-ng/M1.
pom.xml 95.35 KiB