mirror of git://git.yoctoproject.org/yocto-autobuilder-helper.git synced 2025-07-19 20:59:02 +02:00

Ross Burton 2954d78759 Rewrite metrics-gathering scripts

Rewrite the scripts that gather the metrics to be more generic.

Extract the metrics repository cloning out so that we don't have to
repeatedly clone it.

Make the scripts parse their arguments using getopt and be more specific
about what they're passed.  In particular, this means that for the patch
review run we pass the _repository_ that we're scanning so we can do git
operations on it, and the base of the _layers_ (either a layer, or a
directory containing layers) so we know what to scan.

Be more clever when identifying what commits we need to analyse for
patch review: instead of iterating through a set randomly, we can keep
the revision list sorted and the checkout operations are a lot faster.

Remove the commit/file count metric addition as patchreview itself does
that now.

Add an explicit --push option so it's easy to test the scripts in
isolation without pushing.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

2023-10-31 15:01:57 +00:00

2.4 KiB

Executable File

Raw Blame History

#!/bin/bash

SPDX-License-Identifier: GPL-2.0-only

set -eu

ARGS=$(getopt -o '' --long 'metrics:,branch:,results:,push' -n 'run-cvecheck' -- "$@") if [ $? -ne 0 ]; then echo 'Cannot parse arguments...' >&2 exit 1 fi eval set -- "$ARGS" unset ARGS

Location of the yocto-autobuilder-helper scripts

OURDIR=$(dirname $0)

The metrics repository to use

METRICSDIR=""

Where to copy results to

RESULTSDIR=""

The branch we're building

BRANCH=""

Whether to push the metrics

PUSH=0

while true; do case "$1" in '--metrics') METRICSDIR=$(realpath $2) shift 2 continue ;; '--branch') BRANCH=$2 shift 2 continue ;; '--results') RESULTSDIR=$(realpath -m $2) shift 2 continue ;; '--push') PUSH=1 shift continue ;; '--') shift break ;; *) echo "Unexpected value $1" >&2 exit 1 ;; esac done

TIMESTAMP=date +"%s"

if ! test "$METRICSDIR" -a "$BRANCH" -a "$RESULTSDIR"; then echo "Not all required options specified" exit 1 fi

CVE Checks

if [ ! -d $RESULTSDIR ]; then mkdir $RESULTSDIR fi

cd .. set +u . oe-init-build-env build set -u bitbake world --runall cve_check -R conf/distro/include/cve-extra-exclusions.inc

if [ -e tmp/log/cve/cve-summary.json ]; then git -C $METRICSDIR rm --ignore-unmatch cve-check/$BRANCH/*.json mkdir -p $METRICSDIR/cve-check/$BRANCH/ cp tmp/log/cve/cve-summary.json $METRICSDIR/cve-check/$BRANCH/$TIMESTAMP.json git -C $METRICSDIR add cve-check/$BRANCH/$TIMESTAMP.json git -C $METRICSDIR commit -asm "Autobuilder adding new CVE data for branch $BRANCH" || true if [ "$PUSH" = "1" ]; then git -C $METRICSDIR push fi $OURDIR/cve-report.py tmp/log/cve/cve-summary.json > $RESULTSDIR/cve-status-$BRANCH.txt fi

if [ "$BRANCH" = "master" ]; then mkdir -p $METRICSDIR/cve-check/$BRANCH/ $OURDIR/cve-generate-chartdata --json $METRICSDIR/cve-count-byday.json --resultsdir $METRICSDIR/cve-check/ git -C $METRICSDIR add cve-count-byday.json git -C $METRICSDIR commit -asm "Autobuilder updating CVE counts" || true if [ "$PUSH" = "1" ]; then git -C $METRICSDIR push fi

cp $METRICSDIR/cve-count-byday.json $RESULTSDIR
cp $METRICSDIR/cve-count-byday-lastyear.json $RESULTSDIR

2.4 KiB Executable File Raw Blame History