This patch adds support for parsing and visualizing network interface statistics from /proc/net/dev in pybootchartgui. It introduces a new NetSample class to hold per-interface metrics, including received/transmitted bytes and their deltas over time.
The data is drawn using line and box charts in draw.py and helps to monitor
network usage during the boot process for each interface individually.
(From OE-Core rev: 9e640022c83a627bd05c23b66b658bd644b2f0d7)
Signed-off-by: denisova-ok <denisova.olga.k@yandex.ru>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will match other deviation subtask of the same main task,
a couple of them can be found on oe-core layer:
do_compile_kernelmodules
do_compile_ptest
cmake_do_configure
setuptools3_do_configure
cargo_common_do_configure
python_pyo3_do_configure
python_setuptools3_rust_do_configure
This task will be also painted with the same color of the main task
but using alpha blending.
(From OE-Core rev: f10582b1c9a5639b48a4663453d201652facb179)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function draw_label_in_box chooses the correct side based on
the maximum width length argument 'maxx'.
Currently the labels are always drawn on the left side and this
is because we are inadvertently passing the 'maxx' argument wrong.
(From OE-Core rev: b1cea47c52899a85428e0bddf82ba6a07689673b)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is another variable named 'w' on the function which is redefined.
The full width is needed in the function so rename it as this must be unique.
(From OE-Core rev: 81389ccdf22d3ea845ae8d894a0e4e0cdf589f2a)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function draw_label_in_box have logic to chose the rigth side
to call the draw_text, so use it instaed of forcing the cordinates
of the right side.
(From OE-Core rev: b369e9bb0136fecc65f49e0965e5506aa50f489e)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instaed of calculate the elapsed time over the loop when needed
do it on the loop startup.
(From OE-Core rev: 63a19626f10873cd7a4aabee7c9777fdede83bcb)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, finding the elapsed time of each task in buildtimes.svg
is a manual effort of checking the top axis and finding and subtracting
the end and start time of the task.
This change adds the elapsed time for each task, so that
manual effort of comparing start/end time is avoided.
(From OE-Core rev: 3efebd3404de548f0757863da237f2d18ce60013)
Signed-off-by: Mauro Queiros <Mauro.Queiros@criticaltechworks.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous addition of pressure values to the chart didn't fix the extents
function which meant the bottom of the chart was cut off. Fix that.
(From OE-Core rev: cc8cef69e717e08f80d10f775f0fffc644267b59)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix:
scripts/pybootchartgui/pybootchartgui/parsing.py:134: SyntaxWarning: "is" with a literal. Did you mean "=="?
if pid is 0:
(From OE-Core rev: c9a6511ae618035b8efad01646e37ba28ce1e3f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* easier to find than searching for the highest value in the graph, when the graph is really wide
(From OE-Core rev: 615e04554aa8442d89ea186b30f23adeb1e64762)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/pybootchartgui/pybootchartgui/draw.py:820: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (OPTIONS.show_pid or OPTIONS.show_all) and ipid is not 0:
scripts/pybootchartgui/pybootchartgui/draw.py:918: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if i is not 0:
(From OE-Core rev: 8d996616f0ca57220d939a41ca9ba6d696ea2a4f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* memory pressure is already collected in buildstats, render it as well
when available
(From OE-Core rev: 42010d0812246a418f30b4f1d9fbd3f374a3bbe9)
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add two new, separate charts showing the avg10 and delta
total pressure over time for the CPU and IO resources. The height of
the avg10 data in each chart represents the percentage of time "some"
task was delayed over the specific resource during the last 10
seconds of the build. The height of the delta total data in each chart
represents the total time "some" task was delayed since the last sample
was collected. If the reduced_proc_pressure data is not present in the
buildstats log, then the new charts are not shown at all rather than
being present but unpopulated.
Note that the delta total graphs may appear "spikey",
oscillating from high values to low. This behaviour is fixed in a
subsequent commit.
(From OE-Core rev: fb9ff46dc3059cb3f4c8df8e4654184c3eab1571)
Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy MacLeod <randy.macleod@windriver.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid a rare divide by zero error if there isn't data point spread.
[YOCTO #14547]
(From OE-Core rev: d7e36d01e87ddf89f76f164a0b7d98f597a53fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When disk stats don't run frequenctly enough, we see divide by zero
errors. The code already has a fallback path so ensure we use it
for this case too.
[YOCTO #14360]
(From OE-Core rev: b71d30aef5dc2c360432c0dd4147859dd303ea48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
time.clock() was removed in python 3.8, use one of its recommended replacements
to fix failures on python 3.8 systems.
(From OE-Core rev: f1d4ecce7d3968c5e08c268e3329041d4b647cf6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Corrections:
- environment
- accommodate
- conversion
- compatible
(From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The chart size extents were being incorrectly reported, not accounting for the
width of the legend. Set a minimum width to account for that (its fixed size).
Also stop printing the chart background off the bottom of the chart extents.
(From OE-Core rev: ef257afb772daf721f12cb3ff9708d39da06f2c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This tweaks some intermediate variable names to make it clearer what
is being done.
(From OE-Core rev: 50d9af7bcb05823db36e86170ef9ba3a1d6cfeca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the pybootchart code (used for viewing build timing profiles)
to use python3. The bulk of the changes are to use gi instead of pygtk, i.e.
port from gtk+2 to gtk+3.
The main change is to make the bootchart widget inherit gtk.Scrollable
and change the way the scrollbars are implemented to match the new method
upstream. The drawing code used cairo already so can remain unchanged,
(From OE-Core rev: 949144681ad7f536732169351cab6d0612e9c566)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script had a toxic mix of tabs and spaces, fix this.
(From OE-Core rev: 5eebe246e9444a5a7dbf4d8683ae08c468cfc401)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pre-processing /proc data during the build considerably reduces the
amount of data written to disk: 176KB instead of 4.7MB for a 20
minuted build. Parsing also becomes faster.
buildstats.bbclass only writes the reduced logs now, but support for
the full /proc files is kept around as reference.
(From OE-Core rev: b5e47df9af1ebbb477074587fdeae17eb2f55582)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The internal representation after parsing now matches exactly
what the drawing code needs, thus speeding up drawing a bit.
However, the main motivation is to store exactly that required
information in a more compact file.
(From OE-Core rev: ca06e67a0bb5820b38fda4c8dfee20764c1e59ae)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a new, separate chart showing the amount of disk space used
over time for each volume monitored during the build. The hight of the
graph entries represents the delta between current usage and minimal
usage during the build.
That's more useful than showing just the current usage, because then a
graph showing changes in the order of MBs in a volume that is several
GB large would be just flat.
The legend shows the maximum of those deltas, i.e. maximum amount of
space needed for the build. Minor caveat: sampling of disk space usage
starts a bit later than the initial task, so the displayed value may
be slightly lower than the actual amount of space needed because
sampling does not record the actual initial state.
(From OE-Core rev: 263d189d066b578debf08b2bd07494a69b70f70d)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When matching fails, m.group(0) is invalid and can't be used in the
error message.
(From OE-Core rev: ddfea21e06a2e6b1a1e766969f8c134a6de7388a)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The only real change is the addition of two if checks that skips the
corresponding drawing code when there is no data.
(From OE-Core rev: 1658fd5e9ca1ba793cae604c2a395d54e3ec9056)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This enables rendering of the original bootchart charts for CPU, disk
and memory usage. It depends on the /proc samples recorded by the
updated buildstats.bbclass. Currently, empty charts CPU and disk usage
charts are drawn if that data is not present; the memory chart already
gets skipped when there's no data, which will also have to be added
for the other two.
(From OE-Core rev: 233d3e50b361feea07803a9c0f2a691e687c6cd5)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code did not handle x scaling correctly when drawing starts at
some time larger than zero, i.e. it worked for normal bootchart data,
but not for the system statistics recorded by buildstats.bbclass.
(From OE-Core rev: 166f8f9aaa1f01fc6d6a5451f8f06b815c51ffae)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Substracting curr_y when determining the hight of the process chart is
wrong because the height is independent of the position where the
chart is about to be drawn. It happens to work at the moment because
curr_y is always 10 when render_processes_chart() gets called. But it
leads to a negative height when other charts are drawn above it, and
then the grid gets drawn on top of those other charts.
Substracting some constant is relevant because otherwise the box is
slightly larger than the process bars. Not sure exactly where that
comes from (text height?); leg_s seems a suitable constant and happens
to be 10, so everything still gets rendered exactly as before.
(From OE-Core rev: b6bb690728c329ae448f89a1b68298c6dd8a573a)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was broken with the python3 fixes.
(From OE-Core rev: 0eadd008d08f9a6bb17a1f641b4735d0d2012580)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed print_function and with_statement imports from __future__
as they're supported by python 3 by default.
(From OE-Core rev: 77ae2daad5d775d710b953cf0c623ce74cb2c274)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To support yocto on systems with python3 as default version, scripts
should use /usr/bin/env python in the shebang, as this allows the use of
a fake env to mimic python2 as default version.
This patch simply replaces occurrences of #!/usr/bin/python with
#!/usr/bin/env python and was done with this oneliner:
git grep -lE '^#!/usr/bin/python' | xargs \
sed -i 's|/usr/bin/python|/usr/bin/env python|'
(From OE-Core rev: 6d3de22a19657a413e01d7bb5fd74d16c00dc696)
Signed-off-by: Martin Hundebøll <martin@hundeboll.net>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When --full-time (or -T) is used, the graph allways shows the full
time regardless of which processes are currently shown. This is
especially useful in combinationm with the -s flag when outputting to
multiple files.
(From OE-Core rev: c6e88199ddf2c4ae243d42afc403d28ab56f00f0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add minimum width zero-padding to the index used in split output files
with -s and -o. I.e., if -s 200 is used, then the index will be
zero-padded to three digits width.
(From OE-Core rev: 45565b24651ab502ae49dc49261dc3ad5634191f)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously they were transparent.
(From OE-Core rev: 4ebdd8672cc5589a3e2f8d1b75cde7fae9fd6c99)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While "Show more" is enabled, all processes are shown, regardless of
--mintime.
This also has the added benefit of making the first shown bar start at
its correct offset from the start time, rather than always starting at
0.
(From OE-Core rev: 5f1b8730f90099c0f73a6b08599990ee71e831b5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With this, one second ticks are only enabled if the width of a second is
five pixels or more. It is also possible to distinguish 1, 5 and 30
second ticks.
(From OE-Core rev: bd0bde6d04fd6cd9f8e7773d68da127144afa7de)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will make the first bar actually start within the graph. It will
also move the graph to the right so the names of the first tasks are
more likely to be visible.
(From OE-Core rev: 388daa9a8ce7f2216fb55ce65cab1d4060f6c41d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
