Skip to content
Snippets Groups Projects
  • hyukjinkwon's avatar
    60472dbf
    [SPARK-21485][SQL][DOCS] Spark SQL documentation generation for built-in functions · 60472dbf
    hyukjinkwon authored
    ## What changes were proposed in this pull request?
    
    This generates a documentation for Spark SQL built-in functions.
    
    One drawback is, this requires a proper build to generate built-in function list.
    Once it is built, it only takes few seconds by `sql/create-docs.sh`.
    
    Please see https://spark-test.github.io/sparksqldoc/ that I hosted to show the output documentation.
    
    There are few more works to be done in order to make the documentation pretty, for example, separating `Arguments:` and `Examples:` but I guess this should be done within `ExpressionDescription` and `ExpressionInfo` rather than manually parsing it. I will fix these in a follow up.
    
    This requires `pip install mkdocs` to generate HTMLs from markdown files.
    
    ## How was this patch tested?
    
    Manually tested:
    
    ```
    cd docs
    jekyll build
    ```
    ,
    
    ```
    cd docs
    jekyll serve
    ```
    
    and
    
    ```
    cd sql
    create-docs.sh
    ```
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #18702 from HyukjinKwon/SPARK-21485.
    60472dbf
    History
    [SPARK-21485][SQL][DOCS] Spark SQL documentation generation for built-in functions
    hyukjinkwon authored
    ## What changes were proposed in this pull request?
    
    This generates a documentation for Spark SQL built-in functions.
    
    One drawback is, this requires a proper build to generate built-in function list.
    Once it is built, it only takes few seconds by `sql/create-docs.sh`.
    
    Please see https://spark-test.github.io/sparksqldoc/ that I hosted to show the output documentation.
    
    There are few more works to be done in order to make the documentation pretty, for example, separating `Arguments:` and `Examples:` but I guess this should be done within `ExpressionDescription` and `ExpressionInfo` rather than manually parsing it. I will fix these in a follow up.
    
    This requires `pip install mkdocs` to generate HTMLs from markdown files.
    
    ## How was this patch tested?
    
    Manually tested:
    
    ```
    cd docs
    jekyll build
    ```
    ,
    
    ```
    cd docs
    jekyll serve
    ```
    
    and
    
    ```
    cd sql
    create-docs.sh
    ```
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #18702 from HyukjinKwon/SPARK-21485.