When converting variable history file names to relative
paths, keep the layer directory's name so that the user
can distinguish between conf files with the same name.
[YOCTO #8188]
(Bitbake rev: 59561d652af91c2099b735084f0e44275d68e637)
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because some image_license.manifest files contain multiple
FILES lines, and because those lines can sometimes not contain
a list of files (i.e. they look like "FILES:\n"), we were
resetting the list of kernel artifacts when we hit the second
"empty" line.
Fix by ignoring any FILES line which doesn't list files, and by
appending any files found in a valid FILES line, rather than
overwriting the existing list.
[YOCTO #10107]
(Bitbake rev: 927ec3524625ac731326b3c1c1361c2a4d2bd9e1)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* unify the format how the task is described
* don't show taskid followed by taskstring as the taskstring is
different only for setscene tasks (by _setscene suffix)
* the duplicated output was introduced by:
2c88afb taskdata/runqueue: Rewrite without use of ID indirection
as reported and confirmed as a bug here:
http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123148.html
* show:
NOTE: Running task 541 of 548 (/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
instead of much longer:
NOTE: Running task 541 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package, /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package)
and similarly for failed tasks:
ERROR: Task (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
instead of much longer:
ERROR: Task virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1'
(Bitbake rev: 696693d45f5eff1226866ed79dbfb67161d8cd3f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's not a whole lot of point showing how many tasks are running when
we're in quiet mode, it just looks a bit strange particularly when it's
not running any tasks.
(Bitbake rev: 5317200d9cd73c6f971bc1b0cfe8692749e27e3a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we have the task number here we need to subtract 1 to get the number
of tasks completed.
(Bitbake rev: 7c78a1cd3f0638ae76f7c7a469b7f667c7c58090)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds handling of the non-git layers to create and update the
corresponding layer objects in Toaster.
(Bitbake rev: 0a9b5d7d9655dbb09d458fc6e330e932f0f9dab6)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Many of the methods in toasterui and buildinfohelper rely
on the internal state of the buildinfohelper; in particular, they
need a Build object to have been created on the buildinfohelper.
If the creation of this Build object is tied to an event which
may or may not occur, there's no guarantee that it will exist.
This then causes assertion errors in those methods.
To prevent this from happening, add an _ensure_build() method
to buildinfohelper. This ensures that a minimal Build object
is always available whenever it is needed, either by retrieving
it from the BuildRequest or creating it; it also ensures that
the Build object is up to date with whatever data is available
on the bitbake server (DISTRO, MACHINE etc.).
This method is then called by any other method which relies on
a Build object being in the internal state, ensuring that the
object is either available, or creating it.
(Bitbake rev: 0990b4c73f194ec0be1762e4e48b1a525d8349fb)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify buildinfohelper and toasterui so that they record the
recipe parse progress (from ParseProgress events in bitbake)
on the Build object.
Note that because the Build object is now created at the
point when ParseStarted occurs, it is necessary to set the
build name to the empty string initially (hence the migration).
The build name can be set when the build properly starts,
i.e. at the BuildStarted event.
Then use this additional data to determine whether a Build
is in a "Parsing" state, and report this in the JSON API.
This enables the most recent builds area to show the recipe
parse progress.
Add additional logic to update the progress bar if the progress
for a build object changes.
[YOCTO #9631]
(Bitbake rev: f33d51d46d70e73e04e325807c1bc4eb68462f7b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to keep track of layerindex data in our database. And
using branch==release is very confusing in the schema. Instead use the
existing Release definition to keep track of which release a
layer_version is for.
Remove the Branch model and all references to it.
Create a migration path to convert from up_branches to their
corresponding releases.
(Bitbake rev: f8f4cffe6fd371f3a7e63690c68f3fcb5dc1f297)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that progress information we can extract from a task is
rarely apportioned closely enough to the time taken for the ETA to be
accurate, so showing it is going to be misleading most of the time for
anything but the most basic of examples. Let's just remove it and avoid
misleading (or worse, annoying) the user.
Fixes [YOCTO #9986].
(Bitbake rev: 235db4870b11db97250979e647b54cdb5ce4fbb6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify custom widgets then we don't want to assume where the
"extra" position is - you should have to specify it, and if it isn't
specified it shouldn't just wipe out the last widget or you can start to
see odd behaviour if you're modifying the code.
(Bitbake rev: 19e33c10feb1637589ceb05b5e8d58b1e012ccb8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Store the path to the *.rootfs.manifest file for targets which
generate images.
A link to the package manifest is displayed in the build dashboard
for targets which produce image files.
Like the license manifest path, if a target would have produced
the package manifest (but didn't, because it already existed), that
path is copied from the target which did produce the package
manifest.
(Bitbake rev: 79b8e349a0da2ea6b97ad82daa5837e6dfffe0af)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a target is built which is classified as an "image" target
(e.g. "core-image-minimal"), Toaster reads the list of files in
the image (from the files-in-image.txt file).
However, Toaster continues to do this for builds which don't
produce images, if the recipe providing the target is an
image recipe. This can result in a list of files in the image
being attached to a target which didn't produce an image (e.g.
rootfs).
When associating files with an image, ensure that only targets
with a task which produces an image have "files in the image"
associated with them.
[YOCTO #9784]
(Bitbake rev: 44375d0c2a88e0070b8067c9285b89c54eaf3152)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SDK artifacts were previously picked up by toaster.bbclass and
notified to buildinfohelper (via toasterui). The artifacts
were then added to the Build object, so that it wasn't clear
which artifact went with which target; we were also unable
to attach SDK artifacts to a Build if they had already been
attached to a previous build.
Now, toaster.bbclass just notifies the TOOLCHAIN_OUTPUTNAME when
a populate_sdk* target completes. The scan is moved to buildinfohelper,
where we search the SDK deploy directory for files matching
TOOLCHAIN_OUTPUTNAME and attach them to targets (not builds).
If an SDK file is not produced by a target, we now look for a
similar, previously-run target which did produce artifacts.
If there is one, we clone the SDK artifacts from that target
onto the current one.
This all means that we can show SDK artifacts by target, and should
always get artifacts associated with a target, regardless of whether
it really build them.
This requires an additional model, TargetSDKFile, which tracks
the size and path of SDK artifact files with respect to Target
objects.
[YOCTO #8556]
(Bitbake rev: 5e650c611605507e1e0d1588cd5eb6535c2d34fc)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When buildinfohelper records the targets for a build, it looks
up any existing targets for a build and creates them if they
are not present. This is because in the case of Toaster-triggered
builds, the Target objects have already been created (inside
triggerBuild()) and don't need to be recreated; but in the case
of cli builds, the Target objects have to be created by
buildinfohelper.
The issue is that the code for retrieving an existing target for
a build only looks for Targets with a matching target and build,
e.g. Targets for build X with target "core-image-minimal". But it
is perfectly legitimate to call bitbake with a command like
"bitbake core-image-minimal:do_populate_sdk
core-image-minimal:do_populate_sdk_ext". In such a case, the
code which looks for matching targets finds two objects, as it
doesn't filter by task.
Add the task into the filter for the Target so that only one
Target object is be returned. Note that a command
line like "bitbake recipe:task recipe:task" will still cause an
error as bitbake doesn't de-duplicate the command line arguments
and will run the recipe:task combination twice.
(Bitbake rev: 1c0a689fdaae6469d4afb98583161073d32ea50b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bzImage and modules files were previously attached to a build,
rather than to the target which produced them. This meant it was
not possible to determine which kernel artifact produced by a
build came from which target; which in turn made it difficult to
associate existing kernel artifact with targets when those
targets didn't produce artifacts (e.g. if the same machine + target
combination was built again and didn't produce a bzImage or modules
file because those files already existed).
By associating kernel artifacts with the target (via a new
TargetArtifactFile model), we make it possible to find all
the artifacts for a given machine + target combination. Then, in
cases where a build is completed but its targets don't produce
any artifacts, we can find a previous Target object with the same
machine + target and copy its artifacts to the targets for a
just-completed build.
Note that this doesn't cover SDK artifacts yet, which are still
retrieved in toaster.bbclass and show up as "Other artifacts",
lumped together for the whole build rather than by target.
[YOCTO #8556]
(Bitbake rev: 9b151416e428c2565a27d89116439f9a8d578e3d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the image and artifact scan code from toaster.bbclass and
consolidate its logic with the existing logic in buildinfohelper.
Remove handler setup for events which used to be fired from
toaster.bbclass but which are now handled directly by buildinfohelper.
[YOCTO #8556]
(Bitbake rev: f0085cd554604cfff4a3f40a34825fbb6878004f)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a few places we use the fetcher code to fetch files outside of a
task, for example uninative in OE. In that case the pid of the event is
0 and that was causing an error in BBUIHelper.eventHandler(). Check the
pid and do nothing if it's 0.
(Bitbake rev: 59cb919e5cd5c653fb4d69b2d6a4320648443e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using toaster-eventreplay to run a bitbake event file
through toasterui/buildinfohelper, errors occur when the
tasks are updated with buildstats info:
RuntimeWarning: DateTimeField Task.started received a naive
datetime (2016-07-06 09:15:22.070000) while time zone support
is active.
This is because a method in buildinfohelper returns a naive
datetime, but Django is expecting timezone-aware datetimes.
Ensure that datetimes used to set the started/ended times on
tasks are converted to timezone-aware datetimes.
(Bitbake rev: df9f4337bec87024ea6a43138c6080a755eb7fab)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mistakes can happen with the generation of the progress events, change
knotty to be more tolerant of this rather than crashing, reporting to the
user when something unexpected happens. I haven't debugged why multiple
finish events appear to be triggered.
(Bitbake rev: 7dd06b1016b36420a9c55a45ff29dd64ae1dbcda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.
Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.
(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to the "currently running n tasks (x of y)" message, show a
progress bar for another view on how much of the build is left. We have
to take care to reset it when moving from the scenequeue to the
runqueue, and explicitly don't include an ETA since not all tasks take
equal time and thus it isn't possible to estimate the time remaining
with the information available.
(Bitbake rev: de682015a3fefeff36ddc4197641a700f3fb558d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support code on the BitBake side to allow sstate.bbclass in
OpenEmbedded to report progress when it is checking for availability of
artifacts from shared state mirrors.
Part of the implementation for [YOCTO #5853].
(Bitbake rev: 070ae856da0715dbaf4c560c837ea796ffc29f00)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.
Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
* "percent" to just look for a number followed by a % sign
* "percent:<regex>" to specify your own regex matching a percentage
value (must have a single group which matches the percentage number)
* "outof:<regex>" to look for the specified regex matching x out of y
items completed (must have two groups - first group needs to be x,
second y).
We can potentially extend this in future but this should be a good
start.
Part of the implementation for [YOCTO #5383].
(Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.
(Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.
(Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code is still a bit icky (and should be refactored to not use
Gdk.threads_enter/leave) but it should work about as reliably as
it did with Gtk+2.
Based on earlier patches by Maxin and Joshua.
(Bitbake rev: 8eee64a64144e27b5b8c2aca88e138882c3deab7)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
First parameter of traceback.format_exc is a 'limit' - a number
of stracktraces to format.
Passing exception object to format_exc is incorrect, but it works in
Python 2 as this code from traceback module works:
while tb is not None and (limit is None or n < limit):
Comparing integer counter n with the exception object in Python 2
always results in True. However, in Python 3 it throws exception:
TypeError: unorderable types: int() < <Exception type>()
As format_exc is used in except block of handling another
exception this can cause hard to find and debug bugs.
(Bitbake rev: a9509949d7e2adba6e3cd89f97daa19a955855b5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit causes buildinfohelper to crash when run on python 3
as python 3 doesn't have unicode builtin function.
This reverts commit 7a309d964a.
(Bitbake rev: 750ca5c8d5a25fc519b75c56352dec7823c7e240)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The upgrade to python3 is the final nail in the coffin for image-writer
and the goggle UI. Neither seem used or recieve patches and are based
on old versions of GTK+ so drop them, and the remaining crumbs support
pieces.
(Bitbake rev: ee7df1ca00c76f755057c157c093294efb9078d8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disable the problematic gtk usage for use with pygtkcompat. The following
commit removes these tools/UIs entirely but we may as well leave this
piece in the history in case anyone does want a starting point for reusing
them.
(Bitbake rev: c53c7418d392452450352ca2175667dbdbd92401)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.
(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to pass the DATABASE_URL around and read it back if we
setup the django framework in the correct way.
We make the default sqlite database path a full path so that the
database isn't being assumed to be in CWD.
Also add some more useful comments on the database settings.
This is preparation work to migrate the build tests and be able to
trigger builds on differently configured databases.
(Bitbake rev: 973c740404ca6a09feea250d3433075995067fe0)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to flush the footer removal, else it may not be outputted until
the buffer is flushed as part of StreamHandler and this would lead to
it removing the ERROR output just printed which is extremely confusing.
Also ensure the footer is cleared before printing a summary as in
some cases it wasn't being removed, also leading to user confusion.
(Bitbake rev: 0e030c4d074c41859608dab5f3ad26b05f56b306)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
knotty captures two levels of keyboard interrupt: a single interrupt
or two interrupts in a row. These then trigger stateShutdown
and stateForceShutdown respectively.
toasterui doesn't have an equivalent way of capturing interrupts and
using them to shut down bitbake. Now that we are no longer using
knotty + XMLRPCServer for our command line builds (since switching to
per-project build directories), we see some odd side effects of this,
such as builds continuing after they have been interrupted on the
command line.
Bring toasterui in line with knotty (copy-paste most of the code
in knotty.py which deals with interrupts) so that a keyboard
interrupt actually shuts down the bitbake server (if not in
observe only mode).
Additionally use the cancel_cli_build() method to set the Build
status to CANCELLED in Toaster's db when we get keyboard interrupts.
This means that builds interrupted on the command line show as
cancelled (same as if they'd been cancelled from the Toaster UI),
as specified in the UI designs.
[YOCTO #8515]
(Bitbake rev: d39d2edca95900da433074ee95a192d7bfe7090d)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be used from toasterui to cancel the current command-line
build when a keyboard interrupt is captured.
[YOCTO #8515]
(Bitbake rev: 1486c770327b53bb5e04baa5f3ea26d8154aed63)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now the event is sent unconditionally we can drop this feature
as its no longer needed.
(Bitbake rev: 473deeb0fc6065693e1fcfcbb8b79753103db537)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't close the console log file handle, python prints a warning
about unclosed file handles upon exit which is annoying.
(Bitbake rev: 624dd92952b2fc736fd86abe5f2390b87b3a7dd3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python deprecated logger.warn() in favour of logger.warning(). This is only
used in bitbake code so we may as well just translate everything to avoid
warnings under python 3. Its safe for python 2.7.
(Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.
An example of this is bluez where you could end up in a situation where
for example:
NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump
The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.
I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.
Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.
[YOCTO #5044]
(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.
(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
(Bitbake rev: f50fff03b3de02e73a3cc2eb9935f7c345dbddc4)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper stores current Build object in its internal
state. Any changes to Build object will be lost if internal
state is not updated as current buildinfohelper code
saves Build object from internal state when build is
completed.
This bug causes incorrect build state when build is cancelled.
Updating internal state should fix it.
Note, that this commit updates internal state after status of
the build is changed to Build.CANCELLED. There are several other
places in the code where Build object is updated without updating
internal state. They should be carefully analyzed and fixed.
(Bitbake rev: d056cf40fc55530cb1736aedfb9a3c355884991e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When bitbake doesn't need to build anything it still sends
ImagePkgList event with empty 'pkgdata', 'imgdata' and 'filedata'
fields. This causes crash in buildinfohelper code as it's assumed
that above mentioned fields always have data keyed by build target:
ERROR: u'core-image-minimal'
Traceback (most recent call last):
File "toasterui.py", line 423, in main
buildinfohelper.store_target_package_data(event)
File "buildinfohelper.py", line 1218, in store_target_package_data
imgdata = BuildInfoHelper._get_data_from_event(event)['imgdata'][target.target]
KeyError: u'core-image-minimal'
Fixed this by using dict.get method with empty dictionary as default
return value instead of trying to get value without checking if target
key is in the data.
(Bitbake rev: c39cc463e6d9594bf2c5ac8bb74e834f6f2cf7c8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.
(Bitbake rev: d94d12914d351bf560b06d6f4e45c294b04ecaa3)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
toasterui exits event loop on one of the following events:
CommandCompleted, CommandFailed or CommandExit.
Unfortunately none of them come from bitbake when build fails.
This is normai if toasterui runs in observer mode. However, if it's
in build mode this causes toasterui to stuck in the infinite loop
waiting for new events.
The only event we can rely on is BuildCompleted as it always
comes from bitbake unlike 3 above mentioned events.
Modified the code to always shutdown toasterui in build mode
on BuildCompleted event.
(Bitbake rev: 9cd60f98b13cf7b1c518851a51e1cbaa596d8f81)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The keys 'started', 'ended', 'cpu_time_user', 'disk_io_read' and
'disk_io_write' were added to the event recently, so they don't
exist in the events generated by bitbake server from older releases.
Checking if task_to_update structure has these keys before using
them should fix build of older releases.
(Bitbake rev: 79611d0ea742263074fbb0bf5f1e39df75fd9f55)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.
Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.
Got rid of useless and confusing 'ToasterUI build done 1'
log message.
(Bitbake rev: 5d7cce0d0ed70f6b3ebd6cbad300d86964a13398)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.
(Bitbake rev: f6dcb1c9967f042beae024146781cb8235a9e1f2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.
(Bitbake rev: f356c154016c428a3b53af61a075de6f14d9d1d9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.
Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.
Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.
(Bitbake rev: a0c8e2b309055e5927a8ff729d292ccaa69d0575)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional metadata to the layer created for build history to be
able to identify the layer and recipe later on. Specifically this is the
branch and release to which the recipe and layer are associated with
enabling differentiation of two recipes which are local release and
master and 'master' release.
[YOCTO #8528]
[YOCTO #8545]
(Bitbake rev: 3deebd887bddbbd02fd9829a180aab494b1af7c4)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.event.ParseStarted event is not processed by toasterui, but
present in event list. This causes the following error:
WARNING: Unknown event: <bb.event.ParseStarted object at ...
and non-zero return code:
WARNING: Return value is 1
(Bitbake rev: 1cc102f3d83d9467a3a3c422254333796ba95605)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the very verbose log dump from toasterui. This generates several
megabytes of not that useful debug information and actually hinders
finding the original exception.
(Bitbake rev: a21dc134bdce2c9eb5e47c770094660f0c45c398)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.
(Bitbake rev: 2540969ec71612af7f9041cadcc401513e9b357b)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:
* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.
There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.
[YOCTO #8842]
(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>
(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.
Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.
(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.
[YOCTO #7880]
(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.
Checked if setEventMask succeeded. Print error message and
exit if it's not.
(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed events not used in the code from the list.
Added events that are used in the code.
(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.
(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ui does not work in master, nor has it been updated for several years.
[YOCTO #9178]
(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.
To recap, the reasons why Hob is being removed include:
- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.
- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.
- It's GTK+ 2 based, not the current GTK+ 3.
- Toaster is now a much more capable UI and is being actively maintained
The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html
(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.
My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image
Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.
[YOCTO #8940]
(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)
Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An example prefix: `perl-5.22.1-r0 do_compile:`
(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.
(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.
However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.
Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).
Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.
[YOCTO #9021]
(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.
(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.
(Bitbake rev: f81bb65883baa6c0f8a4d48a4de3291a10543992)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.
This deals with errors causing problems on our autobuilders amongst
other issues.
(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.
(Bitbake rev: 46871f769db13ccd36deedc5b6f3dbc0a3d31c4b)
Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that when you view the process tree, the processes
have meaningful names, aiding debugging:
$ pstree -p 30021
bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593)
├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634)
└─{ProcessEQueue}(115591)
$ pstree -p 30021
bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335)
├─PRServ(117332)───{PRServ Handler}(117333)
├─Parser-1:2(117336)
└─{ProcessEQueue}(117331)
Applies to parse threads, PR Server, cooker, the workers and execution
threads, working within the 16 character limit as best we can.
Needed to tweak the bitbake-worker magic values to tell the
workers apart.
(Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OSErrors occurring in toaster.bbclass are converted to
OSErrorException metadata events. They were then being swallowed
as unprocessed events by toasterui, which made them difficult
to spot.
Explicitly catch OSErrorException events and log them so they
are easier to spot and debug.
(Bitbake rev: 69f2b2bc373ce114609600b59a6b6ccef20771c9)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event mask for toasterui doesn't include MetadataEvents.
This means that we're missing the ArtifactFileSize event
(among others), which is the one we use to populate the SDK
artifact table.
Add that event type to the toasterui event mask so we can
record SDK artifacts as they are created.
[YOCTO #7603]
(Bitbake rev: d0276a831bb8cffd42c8367895633eaa1fa1ed30)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new entries to Provides model and link them to
Recipe_Dependency using 'via' field.
This data will be used by Toaster UI to show 'Provides:'
information for the recipes.
[YOCTO #6169]
(Bitbake rev: 336ddc8df611d4c8f1c3d3a06d0a85bb544c38bc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used providermap in store_dependency_information function
to find virtual dependencies. This should fix annoying
warnings "stpd: KeyError saving recipe dependency"
[YOCTO #6169]
(Bitbake rev: 85c416ca338c886db6e79651e44727482df9fb07)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
TERM=dumb bitbake X
shows no output for task status which is suboptimal. Use the non-interactive
mode if the terminal doesn't support what we need for interactive mode giving
a better user experience. Also print a note to the console to say this has
happened.
[YOCTO #8768]
(Bitbake rev: 6f84cf4bd77f35fcd07e0b2f5149f1d6866a414d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced 'while' loop with 'for' loop.
Made the code more compact and hopefully more understandable.
(Bitbake rev: 4e1e497c8432536b3522295e5b1284844ccea056)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This attribute was introduced by mistake. EventHandle is used in the
code for the same purpose.
(Bitbake rev: 8d505ec8913a7d51de48b4f52bb64c5d6a0bb08e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current code throws Exception("Could not register UI event handler")
if event handler can't be registered. The real reason of this is that
cooker is in busy state. Error message lacks information about this.
Added error message to the return value of registerEventHandler.
Included returned error message into the log message and exception
text.
(Bitbake rev: 07de1ca7d57dcd0cc37406feae2949da12a3fa7a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of:
"""
can only concatenate tuple (not "int") to tuple
"""
we now see:
"""
Traceback (most recent call last):
File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 324, in main
termfilter.updateFooter()
File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 210, in updateFooter
lines = 1 + int(len(content) / (self.columns + 1))
TypeError: can only concatenate tuple (not "int") to tuple
"""
which makes tacking down and fixing the problem much easier.
Also ensure we set an error exit code.
(Bitbake rev: d965bcae6cfd268406a3bd1ef77c5bb6c6e1c6d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the row handling was introduced, one of the callbacks was
missed resulting in:
TypeError: can only concatenate tuple (not "int") to tuple
Fix it.
(Bitbake rev: 0b77cea2bf5b5f5704e2650fb0332f5d78037781)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates buildinfo helper for the recent buildstats layout change
(Bitbake rev: 30311bbe667e9f22de17fae00ff58da06a7c3e23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper, with the new import paths for our Django models
and Django 1.8, was not getting an active connection to the database.
In buildinfohelper, call django.setup() explicitly to make sure
that the database connection is ready and models can be queried
and saved.
[YOCTO #8364]
(Bitbake rev: 671aaab8cb7c494cd5c7621b45a6f41a203d8bb5)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than maintain data as part of the migrations (as was
done for the default project previously), create the default
(cli builds) project on demand as a by-product of getting
it from the database.
[YOCTO #8364]
(Bitbake rev: 5fd8e90ab9b81d1bd0d301bc1c91228ecbbea74b)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildinfohelper runs outside of Django, but still needs
access to our Django app classes.
Previously, the imports referenced the toaster.* app, which worked
fine. But in Django 1.8, this causes an error about the same
module being loaded multiple times from different paths.
Change the paths to our Django modules so they don't cause
this error to be thrown. We can do this as we've added our
application libraries to sys.path in the buildinfohelper anyway.
[YOCTO #8364]
(Bitbake rev: 070da64cf32c32b5ffc34d611b463c3a3960b419)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently BRBE is known to toasterui only when build is
started. It's passed to it with BuildStarted event. This is
too late as if build fails earilier than build starts toasterui
can not inform Toaster about the failure.
Set BRBE as soon as it's provided by Toaster.
This should make toasterui to be able to inform Toaster
about early build failures, e.g. failures during recipe parsing.
(Bitbake rev: d7819508dac488a64be3caec88db285cda9599ab)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster is not able to see ParseStarted and ParseProgress events
for command-line builds. This means it's not possible for Toaster
to detect failed builds, if the failure occurs at a point before
the BuildStarted event, as the build won't show up at all.
Add these events to the event mask, so that Toaster's toasterui
can detect and respond to them.
(Bitbake rev: 16bfd3e3d145705a2b3a05648ddbcacc7a338dfa)
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are more tasks running than there are lines on the terminal, the data
scrolls in ways the UI wasn't designed for. This patch adjusts the UI just to
show the processes which fit onto the number of rows in the terminal window.
You can see the total number running from the counter in the top left as usual
and this makes warning and errors messages scrolling from the top of the window
work as designed.
Ultimately, scrolling would be nice but is for another time, this fixes the
biggest UI issue on highly parallel machines.
(Bitbake rev: 67b77658e2bfa849f6f55c9c262cb11d6bfdb399)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed buildinfohelper code which was trying to guess layer version of
the recipe using build request information. The code caused creation of
duplicated recipes as it resulted in layer version from layer index
instead of returning build layer version. As a result of this Toaster
UI was not showing any information about recipes.
Default approach used to find layer version seems to work much better as
it finds proper layer version. Now toaster will use it as the only
way to find layer version.
(Bitbake rev: 101690bda7ad55dc0657483233c90c374713755b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When build information is the 'original' source of the information we
need to return the recipe that was created rather than the copy of the
recipe that is taken for keeping build history. We do this already for
command line triggered builds, but we also have this case for custom
images. We can simply check if the built_recipe exists instead of
special casing this.
(Bitbake rev: 9a8653bf602b2111dee7ee6a459682a68a695b22)
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildinfohelper code expects only one HelpText object per
build/variable/description.
Current code creates more than one such an object, which causes
toastergui to crash with this exception:
MultipleObjectsReturned: get() returned more than one HelpText -- it returned 2!
Used git_or_create API to ensure that only one HelpText object is
created.
(Bitbake rev: e9b46803eb6f1f4044919abf90c8aeb3536e73ed)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster needs bb server to be running all the time due
to merged analysis and managed modes. Server gets restarted
before every build triggered by UI, but it shouldn't be
terminated as it will influence command line builds.
[YOCTO #8279]
(Bitbake rev: e69c87c7842e796ffcd7193ecde22c8f688498f5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated attributes of buildinfohelper object as they can
be changed for every build. For example brbe is set by
runbuilds for every build triggered by Toaster UI.
(Bitbake rev: ea3bc8d01704dc64f6cb7b4f5fe66c312a575174)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than relying on bug 8411, which is conveniently creating
separate log files for each of our builds, create our own
log file for each build.
The log files are created in the same tmp directories that
bitbake users, but are timestamped to the millisecond to avoid
name collisions.
Each log file is opened on a ParseStarted event (for builds
triggered by Toaster) or BuildStarted event (for builds on the
command line: Toaster doesn't get the ParseStarted event
for command-line builds).
The log file is closed on the BuildCompleted event, or if the
build fails.
Because we start logging on ParseStarted for Toaster builds,
we're able to capture the "Build Configuration" section which
bitbake writes to output.
[YOCTO #8373]
(Bitbake rev: 7974203cd8bc66dff1fcc55f8723dedefaf72840)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Metadata can define BB_CONSOLELOG as containing ${DATETIME} and
this can get expanded to a different value each time the variable
is read. In the case of BB_CONSOLELOG, this behaviour is not
desireable.
The values of DATE/TIME are locked down at build time but this is too
late for the purposes of ensuring the system can figure out the real
value of BB_CONSOLELOG.
The best way to do this is to set the variable into the datastore, thereby
preserving its value.
[YOCTO #8411]
(Bitbake rev: 021f2eb55ab5863b57ed1b3f19f1b329bc1ad477)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Target objects are created before the build if build is
started from UI in build mode. However, in analysis mode Target
objects don't exist and need to be created using information
from bitbake events.
Added new API call get_or_create_targets to retrive existing
target objects or create them if they don't exist yet.
(Bitbake rev: ef69be31d133696bde54605f5a18da660099734c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Critical errors (where a build failed for reasons of
misconfiguration, such as a machine being specified which is not
in a project's layers) were being ignored (only log records
up to ERROR level were being logged to Toaster's db). This meant that
the build would fail but would not correctly report why.
Add support for CRITICAL error levels to the LogMessage model,
include errors at this level in the errors property for a build,
and show errors at this level in the build dashboard.
[YOCTO #8320]
(Bitbake rev: b6eacbca9cacb607de864ab7d093deb296da8226)
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
