Commit Graph

2 Commits

Author SHA1 Message Date
Trevor Gamblin
7040223b09 patchtest: selftest: fix patch files
Recent changes to the selftest-hello recipe caused the patchtest
selftest patches to not apply cleanly, which in turn meant several tests
weren't being fully executed or were reporting unexpected PASS/SKIP.

Update all of the test patch files to use a newer baseline and apply
cleanly again.

(From OE-Core rev: d73373d8a590d68e448af2e9ac600adc85777604)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-05 16:47:58 +01:00
Trevor Gamblin
207a5a21d3 patchtest: simplify test directory structure
Consolidate the various mbox tests into a new TestMbox class, metadata
tests into TestMetadata, and patch tests into TestPatch. Also update the
selftest filenames to match the changes. The test contents are not
significantly changed (other than to reference the new class names).
While this doesn't improve overall readability, it does result in more
obvious categorization, and more importantly reduces the number of calls
to setup tinfoil in the tests, resulting in a roughly 25% reduction in
runtime.

Before:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XPASS: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.pass)
XFAIL: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.1.fail)
XPASS: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.pass)
XFAIL: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.fail)
XFAIL: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.1.pass)
XFAIL: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.pass)
XPASS: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.pass)
XFAIL: CVE.test_cve_tag_format (file: CVE.test_cve_tag_format.fail)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.1.fail)
XFAIL: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.fail)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.2.skip)
XPASS: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.1.fail)
XPASS: Shortlog.test_shortlog_format (file: Shortlog.test_shortlog_format.pass)
XFAIL: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.2.fail)
XFAIL: MboxFormat.test_mbox_format (file: MboxFormat.test_mbox_format.2.fail)
XFAIL: Summary.test_summary_presence (file: Summary.test_summary_presence.fail)
XPASS: Author.test_author_valid (file: Author.test_author_valid.2.pass)
XSKIP: Merge.test_series_merge_on_head (file: Merge.test_series_merge_on_head.1.skip)
XPASS: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.pass)
XFAIL: CVE.test_cve_presence_in_commit_message (file: CVE.test_cve_presence_in_commit_message.fail)
XPASS: SignedOffBy.test_signed_off_by_presence (file: SignedOffBy.test_signed_off_by_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_presence (file: LicFilesChkSum.test_lic_files_chksum_presence.pass)
XPASS: CommitMessage.test_commit_message_presence (file: CommitMessage.test_commit_message_presence.pass)
XPASS: Summary.test_summary_presence (file: Summary.test_summary_presence.pass)
XPASS: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned (file: LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.pass)
XFAIL: Shortlog.test_shortlog_length (file: Shortlog.test_shortlog_length.fail)
XFAIL: PatchSignedOffBy.test_signed_off_by_presence (file: PatchSignedOffBy.test_signed_off_by_presence.fail)
XFAIL: Bugzilla.test_bugzilla_entry_format (file: Bugzilla.test_bugzilla_entry_format.fail)
XPASS: SrcUri.test_src_uri_left_files (file: SrcUri.test_src_uri_left_files.pass)
XFAIL: Author.test_author_valid (file: Author.test_author_valid.2.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================

real    24m14.386s
user    1m13.599s
sys     0m21.477s

After:

[tgamblin@megalith poky]$ time ./meta/lib/patchtest/selftest/selftest
XFAIL: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.fail)
XPASS: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.pass)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.1.fail)
XFAIL: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.fail)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.2.skip)
XPASS: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.pass)
XFAIL: TestMbox.test_commit_message_presence (file: TestMbox.test_commit_message_presence.fail)
XPASS: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.pass)
XFAIL: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.fail)
XFAIL: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.fail)
XPASS: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.pass)
XFAIL: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.fail)
XFAIL: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.fail)
XFAIL: TestMbox.test_mbox_format (file: TestMbox.test_mbox_format.2.fail)
XPASS: TestPatch.test_cve_tag_format (file: TestPatch.test_cve_tag_format.pass)
XSKIP: TestMbox.test_series_merge_on_head (file: TestMbox.test_series_merge_on_head.1.skip)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.pass)
XPASS: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.pass)
XPASS: TestMbox.test_bugzilla_entry_format (file: TestMbox.test_bugzilla_entry_format.pass)
XPASS: TestMetadata.test_src_uri_left_files (file: TestMetadata.test_src_uri_left_files.pass)
XPASS: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.pass)
XPASS: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.2.fail)
XFAIL: TestMbox.test_author_valid (file: TestMbox.test_author_valid.2.fail)
XFAIL: TestMetadata.test_lic_files_chksum_presence (file: TestMetadata.test_lic_files_chksum_presence.fail)
XPASS: TestMbox.test_shortlog_format (file: TestMbox.test_shortlog_format.pass)
XPASS: TestMbox.test_author_valid (file: TestMbox.test_author_valid.1.pass)
XPASS: TestPatch.test_signed_off_by_presence (file: TestPatch.test_signed_off_by_presence.pass)
XFAIL: TestMetadata.test_lic_files_chksum_modified_not_mentioned (file: TestMetadata.test_lic_files_chksum_modified_not_mentioned.fail)
XPASS: TestMbox.test_shortlog_length (file: TestMbox.test_shortlog_length.pass)
XFAIL: TestMbox.test_signed_off_by_presence (file: TestMbox.test_signed_off_by_presence.1.fail)
XFAIL: TestMbox.test_cve_presence_in_commit_message (file: TestMbox.test_cve_presence_in_commit_message.fail)
XFAIL: TestMetadata.test_summary_presence (file: TestMetadata.test_summary_presence.fail)
============================================================================
Testsuite summary for patchtest
============================================================================
============================================================================
real    18m39.749s
user    0m41.857s
sys     0m14.708s

(From OE-Core rev: f788592da2fd0e21638ce2c3326675a060ba51cf)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00