Step-by-step tutorial on how one can use Sphinx with Vertex AI pipelines.
The aim of this text is to share the process for utilizing Sphinx to auto-generate the documentation of your machine studying mission.
I‘m going to make use of superior options of Sphinx such because the addition of logos, notes, photographs, and markdown paperwork. Additionally, I’m going to indicate the python bundle you want in order that Sphinx can extract the docstrings offered in your Vertex pipelines.
Let’s get began! As you could know, having up-to-date documentation in your machine studying tasks is important for each the manufacturing and proof of idea phases. Why it’s important? As a result of it helps to make clear and simplify your modules, collaborate together with your group, combine rapidly a brand new group member, make sooner evolutions, and share with the enterprise house owners.
Personally, I’ve skilled so many circumstances wherein resulting from time-to-market constraints the documentation was ignored, however this turned out to be deadly as soon as the mission was launched in manufacturing. Due to this fact I counsel you to sidestep any handbook process, to generate your documentation, as such procedures all the time find yourself getting desynchronized and time-consuming.
So, earlier than publishing your mission, take a while to verify the readability of your mission. In my case, I have a tendency to make use of the next recordsdata :
- README— a file simple to learn that gives an introduction and common data on the mission reminiscent of the aim, technical data, the software program parts
- LICENCE — a file that mentions the license step-by-step process to comply with for contributors
- USAGE — a file to clarify how one can use the mission
- CHANGELOG — a file that tracks the adjustments and the launched variations of the mission
Please notice that a very powerful file is the README. The contribution and utilization data may be added on to the Readme file. The changelog file may be added in a while earlier than releasing the mission in manufacturing. To edit the recordsdata you should use markdown, easy textual content, or reStructuredText.
See beneath the overview of the method we’re going to describe.
What’s Sphinx ?
Sphinx is a robust and easy-to-use open supply auto-generator device extremely utilized by the Python group. It is ready to generate wonderful structured documentation. There exists just a few options reminiscent of MkDocs, Doxygen, pdoc, and others, however Sphinx stays a whole and easy-to-use robust competitor.
The primary options:
- help for a number of output codecs: HTML, PDF, plain textual content, EPUB, TeX, and so forth.
- automated technology of the documentation
- automated hyperlink technology
- multi-language help
- numerous extensions accessible
I. Arrange the setting
II. Set up a digital setting
III. Set up Sphinx
IV. Set-up Sphinx
V. Construct the documentation
I. Arrange the setting
- Python 3
- Native digital machine or Vertex AI Workbench (Jupyter pocket book operating in a digital setting with Python 3)
- Python mission that accommodates Vertex AI code
- Virtualenv
- Kfx — extension for kubeflow pipeline sdk
- MyST parser — taste of Markdown
- Vertex mission containing sdk pipelines
Let’s use an end-to-end open supply instance of a Vertex AI pipeline underneath the Apache-2.0 license. The mission is an effective instance because the mission makes use of Vertex pipelines and does not use a documentation generator.
First, clone the supply code and go to the vertex-pipelines-end-to-end-samples listing:
git clone https://github.com/GoogleCloudPlatform/vertex-pipelines-end-to-end-samples.git
cd vertex-pipelines-end-to-end-samples
II. Create a digital setting & activate it
III. Set up Sphinx
Create a file requirements-sphinx.txt and add :
myst-parser==0.15
requests==2.28.1
sphinx==4.5.0
sphinx-click==4.3.0
sphinx-me==0.3
sphinx-rtd-theme==1.0.0
rst2pdf==0.99
kfx
Set up directly Sphinx and its extensions listed within the requirements-sphinx.txt:
pip set up -r requirements-sphinx.txt
Create a docs listing (if doesn’t exist) to retailer the Sphinx format :
mkdir docs
cd docs
Generate the preliminary listing construction with sphinx-quickstart command:
sphinx-quickstart
Select separate sources and construct directories, the mission title, writer title, mission launch, and the mission language. You will discover beneath my configuration:
It is best to receive the next tree construction :
As you’ll be able to see, we selected to separate the construct and the supply directories. Let’s give just a few explanations about its content material.
The construct/ listing is supposed to maintain the generated documentation. It’s empty for now as we don’t have but any generated documentation.
The make.bat (Home windows) and Makefile(Unix) recordsdata are scripts that simplify the technology of documentation.
The supply/conf.py is the configuration file of the Sphinx mission. It accommodates the default configuration keys and the configuration you specified to sphinx-quickstart.
The supply/index.rst is the basis doc of the mission that accommodates the desk of contents tree (toctree) directive the place it’s best to listing all of the modules you need to embody in your doc.
The _static listing accommodates customized stylesheets and different static recordsdata.
The _templates listing shops the Sphinx templates.
IV. Arrange Sphinx
Establish the python modules: /pipelines
The listing /pipelines accommodates the python code we need to embody within the Sphinx documentation. Observe that Sphinx sees the submodules current within the pipelines bundle provided that you add an __init__.py file within the /pipelines listing.
Generate the Sphinx sources
Use the sphinx-apidoc to construct your API documentation (be certain you might be on the root of the mission). The created Sphinx sources are saved at docs/supply/pipelines.
sphinx-apidoc -f -o docs/supply/pipelines pipelines/You may verify that the next recordsdata had been created at docs/supply/pipelines:
Copy the markdown recordsdata to the docs/supply
Copy the README.md, CONTRIBUTING.md, and USAGE.md recordsdata routinely within the Sphinx supply listing (docs/supply/). Add within the docs/Makefile the next traces to automate the synchronization of markdown recordsdata:
COPY_README = ../README.md
COPY_CONTRIBUTING = ../CONTRIBUTING.md
COPY_USAGE = ../USAGE.md#sincronyze MD recordsdata
$(shell cp -f $(COPY_README) $(SOURCEDIR))
$(shell cp -f $(COPY_CONTRIBUTING) $(SOURCEDIR))
$(shell cp -f $(COPY_USAGE) $(SOURCEDIR))
Edit the index.rst
Use the notice directive for the knowledge you need to spotlight.
.. notice:: Sphinx with Vertex AI.
Use picture directive so as to add a picture. The really useful picture dimension has a width between 400–800 pixels.
.. picture:: ../photographs/xgboost_architecture.png
:align: middle
:width: 800px
:alt: alternate textual content
Beneath the toctree directive listing all of the modules you need to be included within the last documentation (README, modules).
.. toctree::
:maxdepth: 2
:caption: Contents: README
pipelines/modules
CONTRIBUTING
USAGE
Please discover my index.rst beneath:
Edit conf.py — the primary configuration file of Sphinx
Outline the trail:
# Outline path
sys.path.insert(0, os.path.abspath("../.."))
Add your extensions:
extensions = [
"sphinx.ext.duration",
"sphinx.ext.doctest",
"sphinx.ext.viewcode",
"sphinx.ext.autosummary",
"sphinx.ext.intersphinx",
"sphinx_rtd_theme",
"sphinx_click",
"myst_parser",
"sphinx.ext.todo",
"sphinx.ext.coverage",
"myst_parser",
]
Listing the listing of recordsdata to be parsed:
source_suffix = {
".rst": "restructuredtext",
".md": "markdown",
}
Specify the HTML theme:
html_theme = "sphinx_rtd_theme"
So as to add a emblem make certain that the picture is current in supply/_static. I’ve used the vertex AI emblem . Then you’ll be able to outline the brand path :
html_logo = "_static/vertex.png"
Listing all of the exterior hyperlinks current within the markdown recordsdata:
intersphinx_mapping = { "python": ("https://python.readthedocs.org/en/newest/", None)}
See my configuration file conf.py:
V. Construct the documentation
To generate HTML documentation with Sphinx go to /docs and use the command:
make html
Use Firefox to open the HTML web page:
firefox docs/construct/html/index.html
In the event you managed to undergo all of the steps it’s best to have the ability to see an much more interesting HTML web page.
The KFX extension will allow Sphinx to learn the Kubeflow parts, operate names, parameters, and docstrings.
Automate the construct of the documentation utilizing the Makefile(current on the root of the mission). Edit the Makefile and add the next traces:
create-sphinx-sources:
cd docs; make clear; cd ..; rm -r docs/supply/pipelines; sphinx-apidoc -f -o docs/supply/pipelines pipelines/generate-doc:
@ $(MAKE) create-sphinx-sources &&
cd docs; make html
Then name the make generate-doc:
make generate-doc
We reached the tip of our journey with Sphinx. I hope that you simply discovered the content material helpful!
We’ve got seen how one can use Sphinx, a robust device to generate documentation in your machine studying mission. We’ve got custom-made the documentation with logos, photographs, and markdown content material. After all, Sphinx comes with loads of different extensions you should use to render your documentation much more interesting.
