Skip to content
Snippets Groups Projects
  • Lianhui Wang's avatar
    5ad68ba5
    [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose... · 5ad68ba5
    Lianhui Wang authored
    [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators.
    
    ## What changes were proposed in this pull request?
    when query only use metadata (example: partition key), it can return results based on metadata without scanning files. Hive did it in HIVE-1003.
    
    ## How was this patch tested?
    add unit tests
    
    Author: Lianhui Wang <lianhuiwang09@gmail.com>
    Author: Wenchen Fan <wenchen@databricks.com>
    Author: Lianhui Wang <lianhuiwang@users.noreply.github.com>
    
    Closes #13494 from lianhuiwang/metadata-only.
    5ad68ba5
    History
    [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose...
    Lianhui Wang authored
    [SPARK-15752][SQL] Optimize metadata only query that has an aggregate whose children are deterministic project or filter operators.
    
    ## What changes were proposed in this pull request?
    when query only use metadata (example: partition key), it can return results based on metadata without scanning files. Hive did it in HIVE-1003.
    
    ## How was this patch tested?
    add unit tests
    
    Author: Lianhui Wang <lianhuiwang09@gmail.com>
    Author: Wenchen Fan <wenchen@databricks.com>
    Author: Lianhui Wang <lianhuiwang@users.noreply.github.com>
    
    Closes #13494 from lianhuiwang/metadata-only.