yocto-autobuilder-helper/scripts/run-cvecheck
Ross Burton 2954d78759 Rewrite metrics-gathering scripts
Rewrite the scripts that gather the metrics to be more generic.

Extract the metrics repository cloning out so that we don't have to
repeatedly clone it.

Make the scripts parse their arguments using getopt and be more specific
about what they're passed.  In particular, this means that for the patch
review run we pass the _repository_ that we're scanning so we can do git
operations on it, and the base of the _layers_ (either a layer, or a
directory containing layers) so we know what to scan.

Be more clever when identifying what commits we need to analyse for
patch review: instead of iterating through a set randomly, we can keep
the revision list sorted and the checkout operations are a lot faster.

Remove the commit/file count metric addition as patchreview itself does
that now.

Add an explicit --push option so it's easy to test the scripts in
isolation without pushing.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-31 15:01:57 +00:00

104 lines
2.4 KiB
Bash
Executable File

#!/bin/bash
#
# SPDX-License-Identifier: GPL-2.0-only
#
set -eu
ARGS=$(getopt -o '' --long 'metrics:,branch:,results:,push' -n 'run-cvecheck' -- "$@")
if [ $? -ne 0 ]; then
echo 'Cannot parse arguments...' >&2
exit 1
fi
eval set -- "$ARGS"
unset ARGS
# Location of the yocto-autobuilder-helper scripts
OURDIR=$(dirname $0)
# The metrics repository to use
METRICSDIR=""
# Where to copy results to
RESULTSDIR=""
# The branch we're building
BRANCH=""
# Whether to push the metrics
PUSH=0
while true; do
case "$1" in
'--metrics')
METRICSDIR=$(realpath $2)
shift 2
continue
;;
'--branch')
BRANCH=$2
shift 2
continue
;;
'--results')
RESULTSDIR=$(realpath -m $2)
shift 2
continue
;;
'--push')
PUSH=1
shift
continue
;;
'--')
shift
break
;;
*)
echo "Unexpected value $1" >&2
exit 1
;;
esac
done
TIMESTAMP=`date +"%s"`
if ! test "$METRICSDIR" -a "$BRANCH" -a "$RESULTSDIR"; then
echo "Not all required options specified"
exit 1
fi
#
# CVE Checks
#
if [ ! -d $RESULTSDIR ]; then
mkdir $RESULTSDIR
fi
cd ..
set +u
. oe-init-build-env build
set -u
bitbake world --runall cve_check -R conf/distro/include/cve-extra-exclusions.inc
if [ -e tmp/log/cve/cve-summary.json ]; then
git -C $METRICSDIR rm --ignore-unmatch cve-check/$BRANCH/*.json
mkdir -p $METRICSDIR/cve-check/$BRANCH/
cp tmp/log/cve/cve-summary.json $METRICSDIR/cve-check/$BRANCH/$TIMESTAMP.json
git -C $METRICSDIR add cve-check/$BRANCH/$TIMESTAMP.json
git -C $METRICSDIR commit -asm "Autobuilder adding new CVE data for branch $BRANCH" || true
if [ "$PUSH" = "1" ]; then
git -C $METRICSDIR push
fi
$OURDIR/cve-report.py tmp/log/cve/cve-summary.json > $RESULTSDIR/cve-status-$BRANCH.txt
fi
if [ "$BRANCH" = "master" ]; then
mkdir -p $METRICSDIR/cve-check/$BRANCH/
$OURDIR/cve-generate-chartdata --json $METRICSDIR/cve-count-byday.json --resultsdir $METRICSDIR/cve-check/
git -C $METRICSDIR add cve-count-byday.json
git -C $METRICSDIR commit -asm "Autobuilder updating CVE counts" || true
if [ "$PUSH" = "1" ]; then
git -C $METRICSDIR push
fi
cp $METRICSDIR/cve-count-byday.json $RESULTSDIR
cp $METRICSDIR/cve-count-byday-lastyear.json $RESULTSDIR
fi