Skip to content
Snippets Groups Projects
Commit 331293c3 authored by Amit Dev's avatar Amit Dev Committed by Sean Owen
Browse files

[SPARK-13300][DOCUMENTATION] Added pygments.rb dependancy

Looks like pygments.rb gem is also required for jekyll build to work. At least on Ubuntu/RHEL I could not do build without this dependency. So added this to steps.

Author: Amit Dev <amitdev@gmail.com>

Closes #11180 from amitdev/master.
parent 354d4c24
No related branches found
No related tags found
No related merge requests found
...@@ -10,15 +10,18 @@ whichever version of Spark you currently have checked out of revision control. ...@@ -10,15 +10,18 @@ whichever version of Spark you currently have checked out of revision control.
## Prerequisites ## Prerequisites
The Spark documentation build uses a number of tools to build HTML docs and API docs in Scala, The Spark documentation build uses a number of tools to build HTML docs and API docs in Scala,
Python and R. To get started you can run the following commands Python and R.
$ sudo gem install jekyll You need to have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and
$ sudo gem install jekyll-redirect-from [Python](https://docs.python.org/2/using/unix.html#getting-and-installing-the-latest-version-of-python)
installed. Also install the following libraries:
```sh
$ sudo gem install jekyll jekyll-redirect-from pygments.rb
$ sudo pip install Pygments $ sudo pip install Pygments
# Following is needed only for generating API docs
$ sudo pip install sphinx $ sudo pip install sphinx
$ Rscript -e 'install.packages(c("knitr", "devtools"), repos="http://cran.stat.ucla.edu/")' $ Rscript -e 'install.packages(c("knitr", "devtools"), repos="http://cran.stat.ucla.edu/")'
```
## Generating the Documentation HTML ## Generating the Documentation HTML
We include the Spark documentation as part of the source (as opposed to using a hosted wiki, such as We include the Spark documentation as part of the source (as opposed to using a hosted wiki, such as
...@@ -38,14 +41,16 @@ compiled files. ...@@ -38,14 +41,16 @@ compiled files.
$ jekyll build $ jekyll build
You can modify the default Jekyll build as follows: You can modify the default Jekyll build as follows:
```sh
# Skip generating API docs (which takes a while) # Skip generating API docs (which takes a while)
$ SKIP_API=1 jekyll build $ SKIP_API=1 jekyll build
# Serve content locally on port 4000 # Serve content locally on port 4000
$ jekyll serve --watch $ jekyll serve --watch
# Build the site with extra features used on the live page # Build the site with extra features used on the live page
$ PRODUCTION=1 jekyll build $ PRODUCTION=1 jekyll build
```
## API Docs (Scaladoc, Sphinx, roxygen2) ## API Docs (Scaladoc, Sphinx, roxygen2)
...@@ -59,7 +64,7 @@ When you run `jekyll` in the `docs` directory, it will also copy over the scalad ...@@ -59,7 +64,7 @@ When you run `jekyll` in the `docs` directory, it will also copy over the scalad
Spark subprojects into the `docs` directory (and then also into the `_site` directory). We use a Spark subprojects into the `docs` directory (and then also into the `_site` directory). We use a
jekyll plugin to run `build/sbt unidoc` before building the site so if you haven't run it (recently) it jekyll plugin to run `build/sbt unidoc` before building the site so if you haven't run it (recently) it
may take some time as it generates all of the scaladoc. The jekyll plugin also generates the may take some time as it generates all of the scaladoc. The jekyll plugin also generates the
PySpark docs [Sphinx](http://sphinx-doc.org/). PySpark docs using [Sphinx](http://sphinx-doc.org/).
NOTE: To skip the step of building and copying over the Scala, Python, R API docs, run `SKIP_API=1 NOTE: To skip the step of building and copying over the Scala, Python, R API docs, run `SKIP_API=1
jekyll`. jekyll`.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment