While search queries already handled words with hyphens correctly, they
did not do so for words with dots.
To fix this, we
- enhance the word tokenizer to treat both dots ('.') and hyphens ('-')
as valid characters within words.
(For robustness, explicitly exclude dots/hyphens at the start or end
of a word from indexing.)
- adjust query processing to avoid splitting on dots in search input
This allows search queries to correctly match terms such as
'local.conf', 'site.conf', and similar ones now.
Fixes: [YOCTO #14534]
(From yocto-docs rev: 02cc810c17e37563a206629d703dc21fc4280cbf)
Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit 80084a4cabdf7f61c7e93eda8ddbd5bc7d54e041)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The js_splitter_code string contains backslashes that Python tries to
use as escape sequence but doesn't manage to, hence the following
SyntaxWarning message:
documentation/conf.py:188: SyntaxWarning: invalid escape sequence '\p'
.split(/[^\p{Letter}\p{Number}_\p{Emoji_Presentation}-]+/gu)
Considering that we want this to be sent verbatim to the JS, let's make
this a raw string instead.
Fixes: d4a98ee19e0c ("conf.py: tweak SearchEnglish to be hyphen-friendly")
(From yocto-docs rev: 92bc958301d0aeab0a19e75b8604f01770358195)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit c1056672ef45b197136eb8815728d426337a5901)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This modifies the default indexer split() and js splitQuery()
methods to support searching for words with hyphens.
While this might not be an ideal, rock solid, and fully future-proof
solution, it allows at least to search for strings inlcuding hyphens,
such as 'bitbake-layers', 'send-error-report', or 'oe-core'.
Below is a bit more detailed explanation of the two modifications done:
1) The default split regex in the sphinx-doc SearchLanguage base class
is:
| _word_re = re.compile(r'\w+')
which we simply extend to include hyphens '-'.
This will result in a searchindex.js that contains words with hyphens,
too.
2) The 'searchtool.js' code notes for its splitQuery() implementation:
| /**
| * Default splitQuery function. Can be overridden in ``sphinx.search`` with a
| * custom function per language.
| *
| * The regular expression works by splitting the string on consecutive characters
| * that are not Unicode letters, numbers, underscores, or emoji characters.
| * This is the same as ``\W+`` in Python, preserving the surrogate pair area.
| */
| if (typeof splitQuery === "undefined") {
| var splitQuery = (query) => query
| .split(/[^\p{Letter}\p{Number}_\p{Emoji_Presentation}]+/gu)
| .filter(term => term) // remove remaining empty strings
| }
The hook for this is documented in the sphinx-docs 'SearchLanguage'
base class.
| .. attribute:: js_splitter_code
|
| Return splitter function of JavaScript version. The function should be
| named as ``splitQuery``. And it should take a string and return list of
| strings.
|
| .. versionadded:: 3.0
We use this to define a simplified splitQuery() function with a split
argument that splits on empty spaces only.
We extend SearchEnglish (which extends SearchLanguage) here to retain
the stemmer code and stopwords for English.
[YOCTO #14534]
(From yocto-docs rev: 72e70a6af6eb3d4ca41f9e64069d8d3f61b420a3)
Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit d4a98ee19e0cbd6be96923dc72faee143a6b294b)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Import the favicon from https://www.yoctoproject.org/, convert it to
16x16 (as per the Sphinx documentation) to make a 1291B image of the
Yocto logo.
(From yocto-docs rev: 73119a723b48a4038479591f0443c5acb3d21337)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit e3ee43e6d70685a2404aae2d60557a42879b0bb1)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
To make references to the bitbake repo, add an extlink for it and use it
in the docs with ":bitbake_git:`lib/bb/utils.py </tree/lib/bb/utils.py>`".
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
(From yocto-docs rev: 579da2e7222c1bd21948205f470d97435f3b2cc3)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit 8bf3c656ec54a582c75ca7c135121a15f8e4f631)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This patch makes the "latexpdf" target compile the documentation with
xelatex instead of the default pdflatex engine.
The reason behind this is stated in [YOCTO #14357]: pdflatex does not
support compiling foreign characters, so we need to resort to another
engine, here xelatex.
It also increases the texmf config buf_size to 10000000 to avoid a
compilation error.
(From yocto-docs rev: 0a6944b9f90b4d3babbdec9dee18fb8195b0db06)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit 93848842b3ad8aa6b9c9f46d60f2c2ad396c6971)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Newer versions of Sphinx already define a :cve: role that points to
cve.org, instead of the role we defined in conf.py that points to
nvd.nist.gov.
Rename our role to :cve_nist: to avoid warnings (treated as errors).
This is also backwards compatible, meaning we can build the doc with an
older Sphinx if needed.
The file were automatically replaced with following command:
find . -name '*.rst' -exec sed -i 's/:cve:/:cve_nist:/g' {} \+
Cherry pick:
* Changes on following files removed from cherry pick (not part of kirkstone):
documentation/migration-guides/release-notes-4.1.1.rst
documentation/migration-guides/release-notes-4.1.2.rst
documentation/migration-guides/release-notes-4.1.3.rst
documentation/migration-guides/release-notes-4.1.4.rst
documentation/migration-guides/release-notes-4.1.rst
documentation/migration-guides/release-notes-4.2.1.rst
documentation/migration-guides/release-notes-4.2.2.rst
documentation/migration-guides/release-notes-4.2.3.rst
documentation/migration-guides/release-notes-4.2.4.rst
documentation/migration-guides/release-notes-4.2.rst
documentation/migration-guides/release-notes-4.3.1.rst
documentation/migration-guides/release-notes-4.3.2.rst
documentation/migration-guides/release-notes-4.3.3.rst
documentation/migration-guides/release-notes-4.3.4.rst
documentation/migration-guides/release-notes-4.3.rst
documentation/migration-guides/release-notes-5.0.2.rst
documentation/migration-guides/release-notes-5.0.3.rst
documentation/migration-guides/release-notes-5.0.rst
documentation/migration-guides/release-notes-5.1.rst
* Fix minor conflicts in following files:
documentation/migration-guides/release-notes-3.4.2.rst: missing :term:
before CVE_PRODUCT
documentation/migration-guides/release-notes-4.0.2.rst: missing :term:
before PACKAGECONFIG
documentation/migration-guides/release-notes-4.0.7.rst: missing cve
2022-32912 on webkitgtk
Suggested-By: Quentin Schulz <quentin.schulz@cherry.de>
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
(From yocto-docs rev: f432e78fef82c5e5bfdfff08bb18757dc3479465)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit 15fa3b7e85dde50d7236c1738ad607531cc654b8)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
New cve_mitre macro for CVEs that are not found (yet) on https://nvd.nist.gov/
(From yocto-docs rev: ff675ce150187561d6d1e8b8ff0eeb10b2696ec6)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This reverts commit c9922076f5c1285d9cfd6aff8ce5b6635d88222f.
Since the minimum version required to build the docs has been bumped in
the previous commit to 4.0, this commit is not required. Moreover, since
Sphinx 5.0 triggers a warning when extlinks captions are not using %s
substitution and Sphinx 6.0 will fail to build without the substitution
characters, this revert is now required to be able to build the docs.
Cc: Quentin Schulz <foss+yocto@0leil.net>
(From yocto-docs rev: 444df054f0bf3ad7614e1613b22ee5b9e8cb94ef)
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Backporting from master the ones that are applicable to kirkstone.
(From yocto-docs rev: c10d65ef3bbdf4fe3abc03e3aef3d4ca8c2ad87f)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
extlinks captions support using %s substitution but only from sphinx 4.0
onwards.
c.f. https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html#confval-extlinks
Weirdly enough, on older sphinx versions, the caption is just a prefix
to the actual text passed to the extlink. Therefore, in that specific
case, CVE- or CVE-%s are identical in meaning for sphinx >=4.0 and since
only CVE- caption works on sphinx <4.0, let's go with CVE- caption
prefix.
Fixes: b311070d866cf "manuals: add 3.4 and 3.4.1 release notes after migration information"
Cc: Quentin Schulz <foss+yocto@0leil.net>
(From yocto-docs rev: c9922076f5c1285d9cfd6aff8ce5b6635d88222f)
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow conf.py to read the versions it needs from poky.yaml and have
set_versions.py write this out. This means we don't have to change as
many files when making new releases.
(From yocto-docs rev: bfe74c67f327f0c6445cb4129ee0c32db022b95a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Only done for release 3.4 and 3.4.1 so far.
Release notes are kept under the migration-guides/ directory for
the moment, for easier reviewing.
(From yocto-docs rev: b311070d866cf4ab51c72ff3ac7bc41ea3e932cb)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yocto Project releases use a specific Bitbake release. Currently the
bitbake: mapping is pointing to the current master version of Bitbake
documentation. This is an issue if some links disappear over time and
someone is still building old documentation (which is the case on the YP
autobuilder for example). Also, the documentation at the current master
version of Bitbake might not be correct for the Yocto Project version
associated with the doc the user is currently looking at, potentially
causing confusion.
In master branch, nothing needs to be done. For release branches, the
bitbake_version variable needs to be set accordingly. See
https://wiki.yoctoproject.org/wiki/Releases for the mapping.
Cc: Quentin Schulz <foss+yocto@0leil.net>
(From yocto-docs rev: 457a591a46c57519046f9e3f8bffd632bb2275ea)
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SVG directly included in EPUB output has multiple issues,
in particular font size and alignment ones (tested on two
EPUB readers). Instead, using PNG, generated from SVG when available
as the primary format for images. GIF and JPEG are fine too.
(From yocto-docs rev: ff3876ca4910cf61bc25bfeb07e65ad6f6a93b48)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reviewed-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, the license for the manuals was only
stated in the SPDX headers at the beginning of source files
(From yocto-docs rev: cef60f23d5601a18d7ba7e894af2aa459fb2efb8)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needed now that warnings are turned into errors
(From yocto-docs rev: 3f99cff8c6c672967eb09f6950d16ed30260ca68)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building with a outdated version of Sphinx print warnings that does not
appear on up-to-date sphinx.
This patch prevent building the documentation with any version older
than 3.1 (First version to build without warnings in my tests)
See threads "documentation: Add a simple Sphinx extension to check its version"
https://lists.yoctoproject.org/g/docs/topic/patch_documentation_add_a/79919516
and "toaster-manual: Fix a warning related to the code-block directive"
https://lists.yoctoproject.org/g/docs/topic/patch_toaster_manual_fix_a/79656195
(From yocto-docs rev: 4de0f3dd4d5df0a0700f704a599bb41726d15a5f)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The yocto_git external link directive is modified to include the
`/cgit/cgit.cgi` element of the URL so that we can simplify the links in
the text.
Manual links to git.yoctoproject.org are converted to use the yocto_git
directive where possible. Note that this directive can't be used in some
places such as example code.
(From yocto-docs rev: 3a8ba5dcc783411c73fe49fb217cbc4d6528d9a7)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The `yocto_wiki` external link directive is modified to include the
`/wiki` element of the URL so that we can simplify the links in the
text.
Note that there are still a couple of places where this directive
cannot be used, such as in the table of contents in index.rst.
(From yocto-docs rev: d8aa5f93d349f27db3d03a2c4bcc205649f45a8d)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This simplifies linking to git repositories on openembedded.org.
(From yocto-docs rev: 03e13ca4d013e7712216a66eb4cdeb4a456be6a9)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default PDF output shows only chapter headings in the table of
contents and in the outline (aka PDF bookmarks). We should override
these defaults to set something more suitable.
With a depth of 2 for the TOC we see both section and subsection
headings which is enough to get the list of classes in the reference
manual and the list of topics under "Common Tasks" in the development
manual. Going to a deeper level would make the TOC unwieldy but we do
want to make sure we can navigate more precisely using the outline
(commonly shown in a left tab in PDF viewers) so we set the depth to 5
for the outline.
(From yocto-docs rev: 5db16d3b01da0a138b6413347fcf2321a1bfae08)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the ADT manual has long been superseded by the SDK manual,
remove the entire adt-manual directory, and the references to it in
the two top-level files "conf.py" and "poky.yaml".
(From yocto-docs rev: 64b2e83bddf6af0439ac7089ac95e60faa696cfc)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the license identifier tags were added, an incorrect string was used
and the Share-Alike clause was lost. Fix this to match the license
description in the files and add back the lost piece (its clear from
the history it should be there)
(From yocto-docs rev: 8d30c3d792755a7bfdb74b331dad98f51d3516af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following changes were required after a review of trademark and
branding guidelines:
1. add (R) to 'Yocto Project' on the top left (above the logo)
2. Fix up the capitalization of the main page title
3. Add the copyright/legal blurb at the bottom of the page
For 3. it turned out to be simpler to override the whole footer.html
template, and maintain our own version. Also I took the liberty to
remove the 'next' and 'previous' buttons since I believe they are not
especially useful, given the navigation bar on the left side.
Reported-by: Tracey Erway <tracey.m.erway@intel.com>
(From yocto-docs rev: 6bae1372218e0b10258e4fa6fef72fc1708a329c)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To build the Sphinx documentation, we have the following dependencies:
* sphinx
* sphinx_rtd_theme
* pyyaml
If any of these dependencies are missing, we might end up with some
cryptic error messages. This patch adds better error reporting when
dependencies are not met.
(From yocto-docs rev: 19df8d1ec56dc2ecb44122288cc53e84237fab69)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ADT Manual is deprecated, and was removed from the documentation
set in 2.2, until we remove it completely, let's make sure it's
excluded from Sphinx build.
(From yocto-docs rev: 5fa20d6afb1be56cbb2a012357f8ccff4b91d585)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
html_css_files and html_js_files exist since Sphinx 1.8, and it's the
proper (documented) mechanism to include custom CSS and JS files in
the documentation.
(From yocto-docs rev: 4ae9c426654e33fed4185e5d6e0de76b4a430d84)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Remove the 'generated by Sphinx' text on each page
* Add a 'last updated timestamp' on each page
* Remove the trailing 'dot' in TOC numbering
(From yocto-docs rev: 3fa6cf149b3dbbd88b3aa75b6ce1f8bd12817c91)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enhance the sphinx experience/nagivation with:
* Remove the pointless looking parts of breadcrumb navigtation
* Add a document type switcher to the breadcrumb navigation
* Add a version selection switch to the breadcrumb navigation
(From yocto-docs rev: 1823624bdb9ea002d44c9e6d0fd4cd662bff36ad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This extension can generate automatic links to the documentation of
objects in other projects. We will use it to use cross references with
the Bibtake manual, for example.
(From yocto-docs rev: 5add9854b112f93acba982f237fbfa83aee80d77)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This extension generates the labels for each section, so that we can
reference section by their title.
(From yocto-docs rev: 910bdad33819116f00fd4f849dcf7484fbebb465)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A certain amount of boilerplate is added at the beginning of all
documents. In DocBook this is copy/pasted in each file. Let's create a
boilerplate ReST file, which we will include in each document,
wherever it's required.
(From yocto-docs rev: 37e0d5f246c614e62a7c0d4d72a5d6ce9ec5325e)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Yocto Project documentation makes heavy use of 'global'
variables. In Docbook these 'variables' are stored in the file
poky.ent. This Docbook feature is not handled automatically with
Pandoc. Sphinx has builtin support for substitutions however they are
local to each reST file by default. They can be made global by using
rst_prolog:
rst_prolog
A string of reStructuredText that will be included at the
beginning of every source file that is read.
However Sphinx substitution feature has several important limitations. For
example, substitution does not work in code-block section.
yocto-vars.py is an extension that processes .rst file to find and
replace 'variables'. This plugin will do variables substitutions
whenever a rst file is read, so it happens before sphinx parses the
content.
All variables are set in poky.yaml. It's a simple YAML file with pairs
of variable/value, and the file is parsed once during setup. It's
important to note that variables can reference other
variables. poky.yaml was generated by converting poky.ent into a YAML
format.
To use a variable in the Yocto Project .rst files, make sure it is
defined in poky.yaml, and then you can use : &DISTRO_NAME;
For external links, Sphinx has a specific extension called extlinks,
let's use it instead of variable substituions. Note that we
intentionnally did not put the trailing '/' in the URL, this is to
allow us to use :yocto_git:`/` trick to get the actual URL displayed
in the HTML.
(From yocto-docs rev: dc5f53fae8fdfdda04285869dd1419107b920bfe)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to override CSS settings from the theme, by providing
custom snippets of CSS stylesheet. Support for that is added in
conf.py file.
The following changes are made:
* remove the overall text width which (set to 800px by default)
* improve the visual output, and colors of links and admonition
(From yocto-docs rev: 0c1e108bc6c452f7cc8c665bee984bd7da281666)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SPDX headers have been added to each file, and match the headers used
in the DocBook files.
(From yocto-docs rev: 79dbb0007ae24da4a3689a23e921f2a2638757f7)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To install this additional theme:
pip3 install sphinx_rtd_theme
(From yocto-docs rev: 9121dbd0a457451d7f7cdffe8fa2717d5e5959ec)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sphinx-quickstart to generate top level config and
Makefile.sphinx, to allow side by side DocBook and Sphinx
co-existence.
(From yocto-docs rev: 01dd5af7954e24552aca022917669b27bb0541ed)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
