poky/documentation

documentation

This is the directory that contains the Yocto Project documentation. The Yocto Project source repositories at https://git.yoctoproject.org/cgit.cgi have two instances of the "documentation" directory. You should understand each of these instances.

poky/documentation - The directory within the poky Git repository containing the set of Yocto Project manuals. When you clone the poky Git repository, the documentation directory contains the manuals. The state of the manuals in this directory is guaranteed to reflect the latest Yocto Project release. The manuals at the tip of this directory will also likely contain most manual development changes.

yocto-docs/documentation - The Git repository for the Yocto Project manuals. This repository is where manual development occurs. If you plan on contributing back to the Yocto Project documentation, you should set up a local Git repository based on this upstream repository as follows:

                           git clone git://git.yoctoproject.org/yocto-docs

                         Changes and patches are first pushed to the
                         yocto-docs Git repository.  Later, they make it
                         into the poky Git repository found at
                         git://git.yoctoproject.org/poky.

Manual Organization

Here the folders corresponding to individual manuals:

• brief-yoctoprojectqs - Yocto Project Quick Start
• overview-manual - Yocto Project Overview and Concepts Manual
• contributor-guide - Yocto Project and OpenEmbedded Contributor Guide
• ref-manual - Yocto Project Reference Manual
• bsp-guide - Yocto Project Board Support Package (BSP) Developer's Guide
• dev-manual - Yocto Project Development Tasks Manual
• kernel-dev - Yocto Project Linux Kernel Development Manual
• profile-manual - Yocto Project Profiling and Tracing Manual
• sdk-manual - Yocto Project Software Development Kit (SDK) Developer's Guide.
• toaster-manual - Toaster User Manual
• test-manual - Yocto Project Test Environment Manual
• migration-guides - Yocto Project Release and Migration Notes

Each folder is self-contained regarding content and figures.

If you want to find HTML versions of the Yocto Project manuals on the web, the current versions reside at https://docs.yoctoproject.org.

poky.yaml

This file defines variables used for documentation production. The variables are used to define release pathnames, URLs for the published manuals, etc.

standards.md

This file specifies some rules to follow when contributing to the documentation.

template/

Contains a template.svg, to make it easier to create consistent SVG diagrams.

Sphinx

The Yocto Project documentation was migrated from the original DocBook format to Sphinx based documentation for the Yocto Project 3.2 release. This section will provide additional information related to the Sphinx migration, and guidelines for developers willing to contribute to the Yocto Project documentation.

Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Georg Brandl and licensed under the BSD license. It was originally created for the Python documentation.

Extensive documentation is available on the Sphinx website: https://www.sphinx-doc.org/en/master/. Sphinx is designed to be extensible thanks to the ability to write our own custom extensions, as Python modules, which will be executed during the generation of the documentation.

Yocto Project documentation website

The website hosting the Yocto Project documentation, can be found at: https://docs.yoctoproject.org/.

The entire Yocto Project documentation, as well as the BitBake manual, is published on this website, including all previously released versions. A version switcher was added, as a drop-down menu on the top of the page to switch back and forth between the various versions of the current active Yocto Project releases.

Transition pages have been added (as rst files) to show links to old versions of the Yocto Project documentation with links to each manual generated with DocBook.

How to build the Yocto Project documentation

To build the documentation, you need Sphinx and a few other packages, which depend on your host GNU/Linux distribution. Such packages are listed on https://docs.yoctoproject.org/dev/ref-manual/system-requirements.html#required-packages-for-the-build-host

To build the documentation locally, run:

$ cd documentation $ make html

The resulting HTML index page will be _build/html/index.html, and you can browse your own copy of the locally generated documentation with your browser.

Alternatively, you can use Pipenv to automatically install all required dependencies in a virtual environment:

$ cd documentation $ pipenv install $ pipenv run make html

Style checking the Yocto Project documentation

The project is starting to use Vale (https://vale.sh/) to validate the text style.

To install Vale:

$ pip install vale

To run Vale:

$ make stylecheck

Style checking the whole documentation might take some time and generate a lot of warnings/errors, thus one can run Vale on a subset of files or directories:

$ make stylecheck VALEDOCS= $ make stylecheck VALEDOCS=" " $ make stylecheck VALEDOCS=

Lint checking the Yocto Project documentation

To fix errors which are not reported by Sphinx itself, the project uses sphinx-lint (https://github.com/sphinx-contrib/sphinx-lint).

To install sphinx-lint:

$ pip install sphinx-lint

To run sphinx-lint:

$ make sphinx-lint

Lint checking the whole documentation might take some time and generate a lot of warnings/errors, thus one can run sphinx-lint on a subset of files or directories:

$ make sphinx-lint SPHINXLINTDOCS= $ make sphinx-lint SPHINXLINTDOCS=" " $ make sphinx-lint SPHINXLINTDOCS=

Sphinx theme and CSS customization

The Yocto Project documentation is currently based on the "Read the Docs" Sphinx theme, with a few changes to make sure the look and feel of the project documentation is preserved.

Most of the theme changes can be done using the file 'sphinx-static/theme_overrides.css'. Most CSS changes in this file were inherited from the DocBook CSS stylesheets.

Sphinx design guidelines and principles

The initial Docbook to Sphinx migration was done with an automated tool called Pandoc (https://pandoc.org/). The tool produced some clean output markdown text files. After the initial automated conversion additional changes were done to fix up headings, images and links. In addition Sphinx has built in mechanisms (directives) which were used to replace similar functions implemented in Docbook such as glossary, variables substitutions, notes and references.

Headings

The layout of the Yocto Project manuals is organized as follows

Book
  Chapter
    Section
      Subsection
        Subsubsection
          Subsubsubsection

Here are the heading styles that we use in the manuals:

Book                       => overline ===
  Chapter                  => overline ***
    Section                => ====
      Subsection           => ----
        Subsubsection      => ~~~~
          Subsubsubsection => ^^^^

With this proposal, we preserve the same TOCs between Sphinx and Docbook.

Built-in glossary

Sphinx has a glossary directive. From https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:

This directive must contain a reST definition list with terms and
definitions. It's then possible to refer to each definition through the
[https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
'term' role].

So anywhere in any of the Yocto Project manuals, :term:VAR can be used to refer to an item from the glossary, and a link is created automatically. A general index of terms is also generated by Sphinx automatically.

Global substitutions

The Yocto Project documentation makes heavy use of global variables. In Docbook these variables are stored in the file poky.ent. This Docbook feature is not handled automatically with Pandoc. Sphinx has builtin support for substitutions (https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions), however there are important shortcomings. For example they cannot be used/nested inside code-block sections.

A Sphinx extension was implemented to support variable substitutions to mimic the DocBook based documentation behavior. Variable substitutions are done while reading/parsing the .rst files. The pattern for variables substitutions is the same as with DocBook, e.g. &VAR;.

The implementation of the extension can be found here in the file documentation/sphinx/yocto-vars.py, this extension is enabled by default when building the Yocto Project documentation. All variables are set in a file call poky.yaml, which was initially generated from poky.ent. The file was converted into YAML so that it is easier to process by the custom Sphinx extension (which is a Python module).

For example, the following .rst content will produce the 'expected' content:

.. code-block:: $ mkdir poky-&DISTRO; or $ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;

Variables can be nested, like it was the case for DocBook:

YOCTO_HOME_URL : "https://www.yoctoproject.org" YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"

Note directive

Sphinx has a builtin 'note' directive that produces clean Note section in the output file. There are various types of directives such as "attention", "caution", "danger", "error", "hint", "important", "tip", "warning", "admonition" that are supported, and additional directives can be added as Sphinx extension if needed.

Figures

The Yocto Project documentation has many figures/images. Sphinx has a 'figure' directive which is straightforward to use. To include a figure in the body of the documentation:

.. image:: figures/YP-flow-diagram.png

Links and References

The following types of links can be used: links to other locations in the same document, to locations in other documents and to external websites.

More information can be found here: https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.

For external links, we use this syntax: link text <link URL>__

instead of: link text <link URL>_

Both syntaxes work, but the latter also creates a "link text" reference target which could conflict with other references with the same name. So, only use this variant when you wish to make multiple references to this link, reusing only the target name.

See https://stackoverflow.com/questions/27420317/restructured-text-rst-http-links-underscore-vs-use

Anchor (<#link>) links are forbidden as they are not checked by Sphinx during the build and may be broken without knowing about it.

References

The following extension is enabled by default: sphinx.ext.autosectionlabel (https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).

This extension allows you to refer sections by their titles. Note that autosectionlabel_prefix_document is enabled by default, so that we can insert references from any document.

For example, to insert an HTML link to a section from documentation/manual/intro.rst, use:

Please check this :ref:manual/intro:Cross-References to Locations in the Same Document

Alternatively a custom text can be used instead of using the section text:

Please check this :ref:section <manual/intro:Cross-References to Locations in the Same Document>

TIP: The following command can be used to dump all the references that are defined in the project documentation:

   python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv

This dump contains all links and for each link it shows the default "Link Text" that Sphinx would use. If the default link text is not appropriate, a custom link text can be used in the ':ref:' directive.

Extlinks

The sphinx.ext.extlinks extension is enabled by default (https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension), and it is configured with the 'extlinks' definitions in the 'documentation/conf.py' file:

'yocto_home': ('https://yoctoproject.org%s', None), 'yocto_wiki': ('https://wiki.yoctoproject.org%s', None), 'yocto_dl': ('https://downloads.yoctoproject.org%s', None), 'yocto_lists': ('https://lists.yoctoproject.org%s', None), 'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None), 'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None), 'yocto_docs': ('https://docs.yoctoproject.org%s', None), 'yocto_git': ('https://git.yoctoproject.org%s', None), 'oe_home': ('https://www.openembedded.org%s', None), 'oe_lists': ('https://lists.openembedded.org%s', None), 'oe_git': ('https://git.openembedded.org%s', None), 'oe_wiki': ('https://www.openembedded.org/wiki%s', None), 'oe_layerindex': ('https://layers.openembedded.org%s', None), 'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),

It creates convenient shortcuts which can be used throughout the documentation rst files, as:

Please check this :yocto_wiki:wiki page </Weekly_Status>

Intersphinx links

The sphinx.ext.intersphinx extension is enabled by default (https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html), so that we can cross reference content from other Sphinx based documentation projects, such as the BitBake manual.

References to the BitBake manual can directly be done:

• With a specific description instead of the section name: :ref:Azure Storage fetcher (az://) <bitbake-user-manual/bitbake-user-manual-fetching:fetchers>
• With the section name: :ref:bitbake-user-manual/bitbake-user-manual-intro:usage and syntax option

If you want to refer to an entire document (or chapter) in the BitBake manual, you have to use the ":doc:" macro with the "bitbake:" prefix:

• :doc:BitBake User Manual <bitbake:index>
• :doc:bitbake:bitbake-user-manual/bitbake-user-manual-metadata" chapter

Note that a reference to a variable (:term:VARIABLE) automatically points to the BitBake manual if the variable is not described in the Reference Manual's Variable Glossary. However, if you need to bypass this, you can explicitely refer to a description in the BitBake manual as follows:

:term:bitbake:BB_NUMBER_PARSE_THREADS

This would be the same if we had identical document filenames in both the Yocto Project and BitBake manuals:

:ref:bitbake:directory/file:section title

Submitting documentation changes

Please see the top level README file in this repository for details of where to send patches.