MIRRORS/poky
1
0
Fork 0
mirror of git://git.yoctoproject.org/poky.git synced 2025-07-19 12:59:02 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
c390b2e615
poky/meta/lib/oeqa/selftest/cases/bblayers.py
Alexander Kanavin c390b2e615 oe-setup-build: add a tool for discovering config templates and setting up builds 
This is another piece of the puzzle in setting up builds from nothing
without having to write custom scripts or use external tools.

After layers have been fetched and placed into their respective locations by
oe-setup-layers, one would surely want to proceed to the actual build, and here's how:

1. Without arguments the tool reads available layers
from .oe-layers.json file (written out by oe-setup-layers or a fallback under scripts/),
prints what templates it has found, and asks the user to select one, as seen below.
This will land the user in a shell ready to run bitbake:

=============================================
alex@Zen2:/srv/work/alex$ ./setup-build
Available build configurations:

1. alex-configuration-gadget
This configuration will set up a build for the purposes of supporting gadget.

2. alex-configuration-gizmo
This configuration allows building a gizmo.

3. poky-default
This is the default build configuration for the Poky reference distribution.

Re-run with 'list -v' to see additional information.
Please choose a configuration by its number: 1
Running: TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/configuration-gadget . /srv/work/alex/poky/oe-init-build-env /srv/work/alex/build-alex-configuration-gadget && /bin/bash
You had no conf/local.conf file. This configuration file has therefore been
created for you from /srv/work/alex/meta-alex/conf/templates/configuration-gadget/local.conf.sample
You may wish to edit it to, for example, select a different MACHINE (target
hardware).

You had no conf/bblayers.conf file. This configuration file has therefore been
created for you from /srv/work/alex/meta-alex/conf/templates/configuration-gadget/bblayers.conf.sample
To add additional metadata layers into your configuration please add entries
to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
    https://docs.yoctoproject.org

For more information about OpenEmbedded see the website:
    https://www.openembedded.org/

This configuration will set up a build for the purposes of supporting gadget.
Please refer to meta-alex/README for additional details and available bitbake targets.
==============================================

2. It is also possible to list available configurations without selecting one using
'setup-build list' or to select and setup one non-interactively with 'setup-build setup'.

3. The full set of command line options is:

$ ./setup-build --help
usage: setup-build [-h] [--layerlist LAYERLIST] {list,setup} ...

A script that discovers available build configurations and sets up a build environment based on one of them. Run without arguments to choose one interactively.

positional arguments:
  {list,setup}
    list                List available configurations
    setup               Set up a build environment and open a shell session with it, ready to run builds.

optional arguments:
  -h, --help            show this help message and exit
  --layerlist LAYERLIST
                        Where to look for available layers (as written out by setup-layers script) (default is /srv/work/alex/.oe-layers.json).

$ ./setup-build list --help
usage: setup-build list [-h] [-v]

optional arguments:
  -h, --help  show this help message and exit
  -v          Print detailed information and usage notes for each available build configuration.

$ ./setup-build setup --help
usage: setup-build setup [-h] [-c configuration_name] [-b build_path] [--no-shell]

optional arguments:
  -h, --help            show this help message and exit
  -c configuration_name
                        Use a build configuration configuration_name to set up a build environment (run this script with 'list' to see what is available)
  -b build_path         Set up a build directory in build_path (run this script with 'list -v' to see where it would be by default)
  --no-shell            Create a build directory but do not start a shell session with the build environment from it.

4. There's an an added hint in oe-setup-layers about how to proceed (as it is really not user-friendly
to fetch the layer repos successfully and then exit without a word), and a symlink to the script
from the top level layer checkout directory.

5. The selftest to check layer setup has been adjusted to run a basic check for template
discovery and build setup. The revision of poky to be cloned has been bumped to 4.1,
as that's the first version with a default template in a standard location.

(From OE-Core rev: 1360b64e88cda7dddfb0eca6a64f70c13dafb890)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 11:47:53 +00:00

190 lines
10 KiB
Python
Raw Blame History

#
# Copyright OpenEmbedded Contributors
#
# SPDX-License-Identifier: MIT
#
import os
import re
import oeqa.utils.ftools as ftools
from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars, bitbake
from oeqa.selftest.case import OESelftestTestCase
class BitbakeLayers(OESelftestTestCase):
@classmethod
def setUpClass(cls):
super(BitbakeLayers, cls).setUpClass()
bitbake("python3-jsonschema-native")
bitbake("-c addto_recipe_sysroot python3-jsonschema-native")
def test_bitbakelayers_layerindexshowdepends(self):
result = runCmd('bitbake-layers layerindex-show-depends meta-poky')
find_in_contents = re.search("openembedded-core", result.output)
self.assertTrue(find_in_contents, msg = "openembedded-core should have been listed at this step. bitbake-layers layerindex-show-depends meta-poky output: %s" % result.output)
def test_bitbakelayers_showcrossdepends(self):
result = runCmd('bitbake-layers show-cross-depends')
self.assertIn('aspell', result.output)
def test_bitbakelayers_showlayers(self):
result = runCmd('bitbake-layers show-layers')
self.assertIn('meta-selftest', result.output)
def test_bitbakelayers_showappends(self):
recipe = "xcursor-transparent-theme"
bb_file = self.get_recipe_basename(recipe)
result = runCmd('bitbake-layers show-appends')
self.assertIn(bb_file, result.output)
def test_bitbakelayers_showoverlayed(self):
result = runCmd('bitbake-layers show-overlayed')
self.assertIn('aspell', result.output)
def test_bitbakelayers_flatten(self):
recipe = "xcursor-transparent-theme"
recipe_path = "recipes-graphics/xcursor-transparent-theme"
recipe_file = self.get_recipe_basename(recipe)
testoutdir = os.path.join(self.builddir, 'test_bitbakelayers_flatten')
self.assertFalse(os.path.isdir(testoutdir), msg = "test_bitbakelayers_flatten should not exist at this point in time")
self.track_for_cleanup(testoutdir)
result = runCmd('bitbake-layers flatten %s' % testoutdir)
bb_file = os.path.join(testoutdir, recipe_path, recipe_file)
self.assertTrue(os.path.isfile(bb_file), msg = "Cannot find xcursor-transparent-theme_0.1.1.bb in the test_bitbakelayers_flatten local dir.")
contents = ftools.read_file(bb_file)
find_in_contents = re.search(r"##### bbappended from meta-selftest #####\n(.*\n)*include test_recipe.inc", contents)
self.assertTrue(find_in_contents, msg = "Flattening layers did not work. bitbake-layers flatten output: %s" % result.output)
def test_bitbakelayers_add_remove(self):
test_layer = os.path.join(get_bb_var('COREBASE'), 'meta-skeleton')
result = runCmd('bitbake-layers show-layers')
self.assertNotIn('meta-skeleton', result.output, "This test cannot run with meta-skeleton in bblayers.conf. bitbake-layers show-layers output: %s" % result.output)
result = runCmd('bitbake-layers add-layer %s' % test_layer)
result = runCmd('bitbake-layers show-layers')
self.assertIn('meta-skeleton', result.output, msg = "Something wrong happened. meta-skeleton layer was not added to conf/bblayers.conf. bitbake-layers show-layers output: %s" % result.output)
result = runCmd('bitbake-layers remove-layer %s' % test_layer)
result = runCmd('bitbake-layers show-layers')
self.assertNotIn('meta-skeleton', result.output, msg = "meta-skeleton should have been removed at this step. bitbake-layers show-layers output: %s" % result.output)
result = runCmd('bitbake-layers add-layer %s' % test_layer)
result = runCmd('bitbake-layers show-layers')
self.assertIn('meta-skeleton', result.output, msg = "Something wrong happened. meta-skeleton layer was not added to conf/bblayers.conf. bitbake-layers show-layers output: %s" % result.output)
result = runCmd('bitbake-layers remove-layer */meta-skeleton')
result = runCmd('bitbake-layers show-layers')
self.assertNotIn('meta-skeleton', result.output, msg = "meta-skeleton should have been removed at this step. bitbake-layers show-layers output: %s" % result.output)
def test_bitbakelayers_showrecipes(self):
result = runCmd('bitbake-layers show-recipes')
self.assertIn('aspell:', result.output)
self.assertIn('mtd-utils:', result.output)
self.assertIn('core-image-minimal:', result.output)
result = runCmd('bitbake-layers show-recipes mtd-utils')
self.assertIn('mtd-utils:', result.output)
self.assertNotIn('aspell:', result.output)
result = runCmd('bitbake-layers show-recipes -i image')
self.assertIn('core-image-minimal', result.output)
self.assertNotIn('mtd-utils:', result.output)
result = runCmd('bitbake-layers show-recipes -i meson,pkgconfig')
self.assertIn('libproxy:', result.output)
result = runCmd('bitbake-layers show-recipes -i cmake,pkgconfig')
self.assertNotIn('mtd-utils:', result.output) # doesn't inherit either
self.assertNotIn('wget:', result.output) # doesn't inherit cmake
self.assertNotIn('waffle:', result.output) # doesn't inherit pkgconfig
result = runCmd('bitbake-layers show-recipes -i nonexistentclass', ignore_status=True)
self.assertNotEqual(result.status, 0, 'bitbake-layers show-recipes -i nonexistentclass should have failed')
self.assertIn('ERROR:', result.output)
def test_bitbakelayers_createlayer(self):
priority = 10
layername = 'test-bitbakelayer-layercreate'
layerpath = os.path.join(self.builddir, layername)
self.assertFalse(os.path.exists(layerpath), '%s should not exist at this point in time' % layerpath)
result = runCmd('bitbake-layers create-layer --priority=%d %s' % (priority, layerpath))
self.track_for_cleanup(layerpath)
result = runCmd('bitbake-layers add-layer %s' % layerpath)
self.add_command_to_tearDown('bitbake-layers remove-layer %s' % layerpath)
result = runCmd('bitbake-layers show-layers')
find_in_contents = re.search(re.escape(layername) + r'\s+' + re.escape(layerpath) + r'\s+' + re.escape(str(priority)), result.output)
self.assertTrue(find_in_contents, "%s not found in layers\n%s" % (layername, result.output))
layervars = ['BBFILE_PRIORITY', 'BBFILE_PATTERN', 'LAYERDEPENDS', 'LAYERSERIES_COMPAT']
bb_vars = get_bb_vars(['BBFILE_COLLECTIONS'] + ['%s_%s' % (v, layername) for v in layervars])
for v in layervars:
varname = '%s_%s' % (v, layername)
self.assertIsNotNone(bb_vars[varname], "%s not found" % varname)
find_in_contents = re.search(r'(^|\s)' + re.escape(layername) + r'($|\s)', bb_vars['BBFILE_COLLECTIONS'])
self.assertTrue(find_in_contents, "%s not in BBFILE_COLLECTIONS" % layername)
self.assertEqual(bb_vars['BBFILE_PRIORITY_%s' % layername], str(priority), 'BBFILE_PRIORITY_%s != %d' % (layername, priority))
result = runCmd('bitbake-layers save-build-conf {} {}'.format(layerpath, "buildconf-1"))
for f in ('local.conf.sample', 'bblayers.conf.sample', 'conf-summary.txt', 'conf-notes.txt'):
fullpath = os.path.join(layerpath, "conf", "templates", "buildconf-1", f)
self.assertTrue(os.path.exists(fullpath), "Template configuration file {} not found".format(fullpath))
def get_recipe_basename(self, recipe):
recipe_file = ""
result = runCmd("bitbake-layers show-recipes -f %s" % recipe)
for line in result.output.splitlines():
if recipe in line:
recipe_file = line
break
self.assertTrue(os.path.isfile(recipe_file), msg = "Can't find recipe file for %s" % recipe)
return os.path.basename(recipe_file)
def validate_layersjson(self, json):
python = os.path.join(get_bb_var('STAGING_BINDIR', 'python3-jsonschema-native'), 'nativepython3')
jsonvalidator = os.path.join(get_bb_var('STAGING_BINDIR', 'python3-jsonschema-native'), 'jsonschema')
jsonschema = os.path.join(get_bb_var('COREBASE'), 'meta/files/layers.schema.json')
result = runCmd("{} {} -i {} {}".format(python, jsonvalidator, json, jsonschema))
def test_validate_examplelayersjson(self):
json = os.path.join(get_bb_var('COREBASE'), "meta/files/layers.example.json")
self.validate_layersjson(json)
def test_bitbakelayers_setup(self):
result = runCmd('bitbake-layers create-layers-setup {}'.format(self.testlayer_path))
jsonfile = os.path.join(self.testlayer_path, "setup-layers.json")
self.validate_layersjson(jsonfile)
# The revision-under-test may not necessarily be available on the remote server,
# so replace it with a revision that has a yocto-4.1 tag.
import json
with open(jsonfile) as f:
data = json.load(f)
for s in data['sources']:
data['sources'][s]['git-remote']['rev'] = '5200799866b92259e855051112520006e1aaaac0'
with open(jsonfile, 'w') as f:
json.dump(data, f)
testcheckoutdir = os.path.join(self.builddir, 'test-layer-checkout')
result = runCmd('{}/setup-layers --destdir {}'.format(self.testlayer_path, testcheckoutdir))
layers_json = os.path.join(testcheckoutdir, ".oe-layers.json")
self.assertTrue(os.path.exists(layers_json), "File {} not found in test layer checkout".format(layers_json))
# As setup-layers checkout out an old revision of poky, there is no setup-build symlink,
# and we need to run oe-setup-build directly from the current poky tree under test
oe_setup_build = os.path.join(get_bb_var('COREBASE'), 'scripts/oe-setup-build')
oe_setup_build_l = os.path.join(testcheckoutdir, 'setup-build')
os.symlink(oe_setup_build,oe_setup_build_l)
cmd = '{} --layerlist {} list -v'.format(oe_setup_build_l, layers_json)
result = runCmd(cmd)
cond = "conf/templates/default" in result.output
self.assertTrue(cond, "Incorrect output from {}: {}".format(cmd, result.output))
# rather than hardcode the build setup cmdline here, let's actually run what the tool suggests to the user
conf = None
if 'poky-default' in result.output:
conf = 'poky-default'
elif 'meta-default' in result.output:
conf = 'meta-default'
self.assertIsNotNone(conf, "Could not find the configuration to set up a build in the output: {}".format(result.output))
cmd = '{} --layerlist {} setup -c {} --no-shell'.format(oe_setup_build_l, layers_json, conf)
result = runCmd(cmd)
Reference in New Issue View Git Blame Copy Permalink