Commit Graph

10 Commits

Author SHA1 Message Date
Trevor Gamblin
dab2dcae89 patchtest: add test_commit_message_user_tags
This test makes patchtest check to ensure that there aren't any
GitHub-style user account names being tagged in the commit message, e.g.
it should catch lines like:

"fix added by @threexc"

This is desired so that if (for example) we add upstream changelogs in
recipe upgrade commit messages verbatim, we don't end up subscribing any
associated maintainers to our repo mirrors' updates by accident.

There is a small possibility of a false positive with this test, where
if someone is mentioning Python decorators in their commit message (or
similar syntax from other languages), it will fail when it should pass.
However, having this test in place to guard against username inclusion
is more important that the occasional false positive for that reason.

With this addition, a failure will look like:

|FAIL: test commit message user tags: Mbox includes one or more GitHub-style username tags. Ensure that any "@" symbols are stripped out of usernames (test_mbox.TestMbox.test_commit_message_user_tags)

(From OE-Core rev: 8278d82d8203212bb159eb3805d4a5617c5370df)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-23 17:35:23 +01:00
Trevor Gamblin
4c378fc895 patchtest: simplify, rename modules
- simplify base.py, data.py
- move some leftover regex patterns to patterns.py
- remove pyparsing path logic, since this is no longer needed
- rename PatchTestInput class to PatchtestParser
- data.py: rename to patchtest_parser.py
- patterns.py: rename to patchtest_patterns.py
- move PatchTestDataStore to test_metadata.py since that's the only
  place it's used
- remove unused logger code

(From OE-Core rev: 1e971b05b036b0b1eb0bdbd9b26b54d06e74294c)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Trevor Gamblin
18a65c77c0 patchtest: test_mbox: change ismerged -> canbemerged
The merge logic is a bit weird as-is. Remove the ismerged() method and
replace it where appropriate with canbemerged().

(From OE-Core rev: c5f5686f20b13d635a461d4cc353563f9c886f5f)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Trevor Gamblin
bb0f1625d7 patchtest: patterns: add module, refactor
Currently, patchtest has a lot of spread-out definitions for patterns
used in various setup and test functions. Organize these by putting them
all into a new patterns.py module. This allows the tests/pyparsing
directory to be removed, as it is now redundant. Also remove some
definitions where they were duplicated or unused, and perform some
renames to improve readability and avoid collisions. Many of these
variables are composed from others, so the file is only partially
sorted.

(From OE-Core rev: 1ab55d495957918be532a36224b5598c9955a44d)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Trevor Gamblin
bd2159719c patchtest: test_non_auh_upgrade: improve parse logic
The AUH email address used for matching was outdated. Fix it so that it
correctly identifies emails using the new one.

Also make sure to only scan the commit message and not the body, since
it's possible (like in this patch) that the user may be editing actual
code that checks for AUH-related strings.

Fixes [YOCTO #15390].

(From OE-Core rev: 557400648b6f4f31176847f8a068d2e199b7793d)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-01 12:32:37 +01:00
Trevor Gamblin
3802372a2a patchtest: test_shortlog_length: omit shortlog prefixes
In certain cases, the shortlog for a patch would include branch and
'PATCH' tag information even after the repo tag got stripped out,
padding it with data that wouldn't actually be included with a merge.
This caused the patchtest test_shortlog_length test to erroneously
report a failure.

For example, a patch whose shortlog tags were:

[OE-core][master/scarthgap][PATCH]

would leave this behind in the shortlog, along with the actual content:

[master/scarthgap][PATCH]

Add a re.sub() call in the test to fix this by removing all square
bracket pairs that appear at the beginning of the string.

Fixes [YOCTO #15575].

(From OE-Core rev: 6e1ddadfe779291cf46fb88fb01769c022b733ac)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-01 12:28:10 +01:00
Trevor Gamblin
6f66cf7694 patchtest: test_mbox: remove duplicate regex definition
(From OE-Core rev: b350239ce1350ed61410431d30e938365d406db1)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-01 12:28:10 +01:00
Trevor Gamblin
06a7e7a14d patchtest: remove test for CVE tag in mbox
After patchtest went live it was determined that testing for a CVE tag
in the mbox commit message is unnecessary, since it will already be in
the shortlog and in any carried patches. Remove the test and the
associated selftest files so that its absence isn't flagged in future
test results.

(From OE-Core rev: 54690f18f04a2ab993a85d551ce4f8d0fa56618a)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-01 19:19:40 +00:00
Trevor Gamblin
9b22a5aa8c patchtest: shorten test result outputs
Some test result lines in TestMbox and TestPatch are still too long to
avoid being flagged by the mailer script. Clean them up by removing
redundant information, so that they are all under the length limit of
220 characters.

(From OE-Core rev: c10d0bb542b23fbdc14d76dfa8e5885aa4d33083)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-29 22:39:17 +00:00
Trevor Gamblin
207a5a21d3 patchtest: simplify test directory structure
Consolidate the various mbox tests into a new TestMbox class, metadata
tests into TestMetadata, and patch tests into TestPatch. Also update the
selftest filenames to match the changes. The test contents are not
significantly changed (other than to reference the new class names).
While this doesn't improve overall readability, it does result in more
obvious categorization, and more importantly reduces the number of calls
to setup tinfoil in the tests, resulting in a roughly 25% reduction in
runtime.

Before:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass)
XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail)
XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass)
XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail)
XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass)
XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass)
XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass)
XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail)
XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip)
XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail)
XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail)
XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip)
XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass)
XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail)
XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass)
XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass)
XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass)
XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail)
XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail)
XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail)
XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================

real    24m14.386s
user    1m13.599s
sys     0m21.477s

After:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail)
XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail)
XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip)
XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass)
XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail)
XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass)
XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail)
XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail)
XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass)
XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail)
XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail)
XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass)
XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass)
XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass)
XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass)
XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass)
XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail)
XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail)
XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass)
XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass)
XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail)
XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail)
XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================
real    18m39.749s
user    0m41.857s
sys     0m14.708s

(From OE-Core rev: f788592da2fd0e21638ce2c3326675a060ba51cf)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00