We were using this one temporarily to use the new Sphinx version and
rsvg-tools. Update the URL now that 5.1.1 is released.
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a modified 5.1 buildtools tarball for docs builds which has support for svg -> png
conversion using rsvg-convert.
Older releases need older docs buildtools. Add support for this in the build
script for docs, installing both and choosing depending on how old the main
release branch is. For now I've put the changeover at scarthgap, it may be
possible to make it older. I know kirkstone fails with the new tools.
Bitbake doesn't need this since it's docs work fine with both versions.
The actual docs build is done in a subshell so we can choose the environment.
We may need to improve the selection logic in future since some branches may
start to build with the newer tools after backports.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tweak the tex install command to include latexmk which is needed
by the docs build process.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To build the latexpdf output for the docs, we need Xetex (which can handle
international characters). Rather than require it on all autobuilder workers,
along with dependencies, we install a copy onto NFS which the workers can
use. The full install is 6GB so just install the pieces we've found we
need via tlmgr on top of a small install. This script creates the install
if it isn't present.
For local docs builds, we assume the user has the tools available from
the host OS. It is unlikely most users using this script would have the
NAS path present so this seems like a reasonable compromise.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pub/non-release directories have aged out and no longer work.
Switch to a stable location and bump the version to 4.1.2 as well.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
It happened that the git repositories were dirty and resulted in
incorrect files being used. Let's use git clean -ffdx to force a
completely clean git repositories before and after checking out a branch
so that nothing is left from or to another branch build. Also git reset
--hard to undo changes to tracked files.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Reported-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> #https://lore.kernel.org/yocto-docs/e50abe3c777e4a23a752a3ec25ad0b2a@axis.com/
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Master branch of the yocto-docs now autogenerates the releases.rst file
and the file therefore does not exist anymore.
Signed-off-by: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`git checkout <pathspec>` without <tree-ish> only undo the changes made
to pathspec compared to HEAD. It is not guaranteed that HEAD is on
latest commit of master branch from upstream when running this script,
therefore let's add the <tree-ish> argument to specify which branch to
use to make sure set_versions.py is the latest, otherwise getlatest
might return the wrong information.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We may not have fetched the master revision if a different one was being
built so using origin/master works consistently in all cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script which adds banners to the old docs and outdated dunfell docs
so that the user knows they're using old documentation.
Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason, the old docs tarball includes many eclipse
subdirectories which are just cluttering the docs website up.
Therefore, let's just not extract eclipse directories from the tarball.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent here is to use the latest release branch name instead of a tagged version.
For the 4.0 case, this means the full migration guide will be shown instead of
the reduced information in the 4.0 tag.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the latest tag already has a specific handling in the forloop,
let's just move the symlink creation inside the forloop.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the master branch is the default branch being displayed by
default when reaching the Yocto docs website. When big changes are
implemented for the next release, these are shown immediately to the
user, even though there is currently no release available for those
changes. This is an issue when e.g. behaviors are changed, new features
are added, variables get renamed or some syntaxes change because the
user might try to use things that aren't available to them yet.
I believe more people are using released version of Yocto
Project/Bitbake than people working on latest master. So let's make the
default version of the docs the latest tag (in terms of version number,
not date of tagging) to avoid too much confusion.
The master branch of the docs is now available at /dev subpath.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
set_versions.py and switchers.js.in need to be up-to-date so that a
consistent behavior is kept between different branches and tags of the
documentation.
Right now, kirkstone branch is lagging behind master and therefore does
not have the latest changes from master (e.g. the new obsolete
algorithm, which obviously isn't an issue right now, but will be in two
years). Using master version for those scripts also lightens the
maintenance burden.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rsync only makes sense for the Autobuilder as most people don't have
access to docs@docs.yoctoproject.org. Therefore, to allow for easier
contribution to this script, let's allow to skip the rsync so the script
can continue to run.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the path to buildtools script and docbook tarball are
hardcoded to work on Yocto Project Autobuilder. However, this makes it
harder to contribute to this script because it is very unlikely those
paths exist on a developer PC.
Instead, let's allow to override variables by using the environment and
make the current hardcoded values the default ones.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script is written in such a way that absolute paths are expected.
Instead of failing weirdly at some point in the script, let's just make
all paths passed to this script absolute by calling realpath on them.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses the set_versions.py script from the docs master branch to handle
creation of the swictchers.js version information as well as the data in
poky.yaml.
This allows patches to those sections of the docs to be dropped.
It is assumed the patch to use set_versions is applied to the docs transitions
branch so that it's switchers.js files no longer need to be tweaked.
This does lead to user visible changes on the website:
- Older versioned released docs gain their current version and the
latest version listed in the switcher rather than being unlisted
- The list of releases is normally filtered down to our active ones
(hardknott, honister and dev right now)
- 3.3's doc references to gatesgath are corrected to hardknott
- Docs for unversioned release branches (not linked on the website) now
use 3.1.999 versioning instead of confusing them with the last relased version
- 3.1.13 refers to 3.1.13 instead of 3.1.12
all of which seem to be reasonable improvements.
The big advantage of these changes is that with a single change to the script
in the master branch, the right thing should happen for all the current docs
and at release time, only a docs rebuild should need to be triggered after tags
are pushed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Except patching which is specific to tags and yocto- tag prefix
stripping, the logic is identical, so let's merge both loops together.
Cc: Quentin Schuls <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit that introduced Sphinx support in yocto-docs is
01dd5af7954e24552aca022917669b27bb0541ed. Any tag containing this commit
is buildable by sphinx.
Dunfell tags don't all have Sphinx support. However, all tags containing
the introducing commit c25fe058b88b893b0d146f3ed27320b47cdec236 are
buildable by sphinx.
Therefore, let's just list all tags which contains either of those two
commits instead of the complex series of pipes and shell commands.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 01dd5af7954e24552aca022917669b27bb0541ed, all later
releases of yocto-docs can be built with Sphinx. Instead of manually
updating this list, let's have git return the list of remote branches
which contains the commit.
dunfell branch was initially released without Sphinx support but was
later patched, hence why it's explicitly listed.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
master, master-next and transition only differ from other branches by
their output directory name. Let's put everything in common and only
have a check on whether the branch is master, master-next or transition
and modify the output dir in those cases.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 84ccba0f4aff91528f764523fe1205a354c889ed, docs of all later
releases can be built with Sphinx. Instead of manually updating this
list, let's have git return the list of remote branches which contains
this commit.
1.46 branch was initially released without Sphinx support but was later
patched, hence why it's explicitly listed.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
master and master-next only differ from other branches by their output
directory name. Let's put everything in common and only have a check on
whether the branch is master or master-next and modify the output dir in
those cases.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch was meant as a quick temporary work-around to have the docs
built and published.
Now that releases where -W flag is set (turning warnings into errors)
are appropriately patched to make those warnings disappear (on Sphinx
v3.2.1 which is the one used on the builder), this patch can be reverted
so that next time a warning appears the doc building will fail but will
not destroy the doc website (because of commit
6a4e6ef18d "scripts: run-docs-build: make
the script fail hard ASAP when there's an error", since rsync will not
be run if any error happens before).
This reverts commit 931d409b25.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to keep a list of tags requiring to be patched since the
tag is part of the path where patches are stored.
Therefore, let's only check if there's a patch directory for a given tag
and if so, apply all patches in there.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add patch for version 3.3.5 and move patches into a dedicated
subdirectrory to keep the scripts directory tidy.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
When current_version and all_versions are not updated before the release
build the tag will not build working docs. Make docs builds pull in
needed updates after the fact.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Default flags for sphinx-build contain -W which turns warnings into
errors. This is really helpful during debugging and could be used for
continuous integration though we do not have such a thing separate from
the continuous delivery in place currently. Nowadays, the docs files
served at docs.yoctoproject.com are actually removed before being
updated from the newly built docs. If the `html` target, a dependency of
`publish` target is failing, the docs aren't copied over to the `final`
temporary directory. Therefore it'll be missing in the rsync upload to
docs.yoctoproject.org.
Instead, let's disable the turning of warnings into errors so that the
`html` make target can finish successfully and the `publish` make target
to finish successfully too.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some intermittent issues with the script not publishing all
versions. So let's go extreme and fail the script if any error happens:
- a command returns a non-zero code, even if piped,
- a variable is used uninitialized,
This also makes the script print each and every command being run so we
have a better idea where the script struggles.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A wrong path was given given the working directory.
Also revert the changes with "git reset --hard" to
have a clean state before further branch switches.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows all versions of Bitbake and Yocto Project manuals
to see the manuals for the latest versions.
This also simplifies the release process, not having to update the
releases.rst file for all releases every time a new release is made.
Note that such synchronization is already done for the
switchers.js file (but in a different way). This way, advertised
releases are in sync with switchers.js.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both releases are missing an important patch that changes the displayed
version from dev to the appropriate release number.
This is confusing to the user and probably breaks some assumptions in
some scripts.
Ideally, the tags should have been moved with those patches applied to
their respective branch but that is not a git best practice so we're
stuck with this "hack" instead.
3.3.x releases aren't impacted as they got the patch applied.
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
honister and 1.52 Bitbake branch were recently released, so let's build
those too.
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure that https://docs.yoctoproject.org/current will always link
to the most recent release documentation.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Build Sphinx docs for all versions newer than yocto-3.1.5 inclusive.
Integrate suggestions from Quentin Schulz and Nicolas Dechesne.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The branches exist now, let's publish the corresponding docs.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
