This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mandriva is no longer maintained, with its last release having been in
2011. It's no longer useful as a yard-stick distro, therefore drop it from
distro_check.create_distro_packages_list()
(From OE-Core rev: 4e1bef31e04044d26a948a9682a5122b46d15c0f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since openSUSE switched to their dual development model of Leap and
Tumbleweed the urls we need to check for package lists has changed. Update
get_latest_released_opensuse_source_package_list() to use the new Leap
urls.
(From OE-Core rev: 263da717ba689eb7efe99234566e2b474e8ce95a)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.
(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of checking against a file that represents a distribution that hasn't
existed for years, fetch package names for Clear Linux instead.
[ YOCTO #10601 ]
(From OE-Core rev: 006c4db0974c42ff0f6950dd24e61c008f801679)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This library suffered as part of the Python 2 to Python 3 migration and stopped
working entirely.
Fix all the migration problems such as files being treated as strings but opened
in binary mode, insufficient use of with on files, and so on.
Rewrite large amounts to be Pythonic instead of C-in-Python.
Update OpenSuse and Fedora URLs.
Fedora now splits the archive alphabetically so handle that.
[ YOCTO #10562 ]
(From OE-Core rev: 58de12eaaac9c60bb8fc84a3a965ef86d2a39ae0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
create_socket: Use urllib because urllib2 is now urllib in python3
and proxies as argument are deprecated so export them in the environ
instead.
get_links_from_url: Change usage of sgmllib for parsing HTML because
is deprecated in python 3, use instead bs4 that is already imported
in the bitbake tree.
[YOCTO #9744]
(From OE-Core rev: ee26ecf58277560459dd01992bb3f486f92c1531)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the proxies dictionary has a proxy set to None urllib will throw an exception
instead of not using a proxy (abridged stack):
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 43, function: get_links_from_url
*** 0043: with create_socket(url,d) as sock:
0044: webpage = sock.read()
File: '/home/ross/Yocto/poky/meta/lib/oe/distro_check.py', lineno: 5, function: create_socket
0003:def create_socket(url, d):
0004: import urllib
*** 0005: socket = urllib.urlopen(url, proxies=get_proxies(d))
File: '/usr/lib/python2.7/urllib.py', lineno: 87, function: urlopen
0086: if data is None:
*** 0087: return opener.open(url)
File: '/usr/lib/python2.7/urllib.py', lineno: 203, function: open
0201: else:
0202: proxy = None
*** 0203: name = 'open_' + urltype
0204: self.type = urltype
0205: name = name.replace('-', '_')
Exception: TypeError: cannot concatenate 'str' and 'NoneType' objects
Filter out unset values so that the dictionary only has valid assignments in.
(From OE-Core rev: 2d91290ab5608dd1297d1c26ab807fc4574a8a6b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two ways to pass the proxies values into urllib.open: exporting
the environment variables *_proxy and passing as dictionary into the
urllib.open call. The latter is currenty used and values should not have
the '_proxy' string, so removing it. Also, ignore uppercase proxies, these
are not taken into account by the library.
Tested on a network with proxy, adding 'inherit +="distrodata"' on local.conf
with the following commands
for distrotask in distrodataall distro_checkall checklicenseall; do
bitbake universe -c $distrotask
done
[YOCTO #7567]
(From OE-Core rev: 92fb327b08e4f9cfb2400610c2983afe579b3592)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Proxies defined in the enviroment where not taken into account
on the distrodata tasks. This commit implied passing the datastore
into the distro_check library and context manager for the
urllib.urlopen function.
One way to run distrodata tasks is using 'universe' as target and the
'all' distrodata task:
$ bitbake universe -c distrodataall
$ bitbake universe -c distro_checkall
$ bitbake universe -c checklicenseall
Logs are located under TMPDIR/log
[YOCTO #7567]
(From OE-Core rev: 7d1c3470bb06e43245ccb7f067121de606506430)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the distro check functions for the change of nativesdk
being a suffix to a prefix. Also added crosssdk as another
case for converting to PN for matching in the distro_tracking
(From OE-Core rev: ae9dbd0e1e26ba2b35cbd08ec731aee62adedc23)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace os.system with subprocess.call since the older function would
fail (more or less) silently if the executed program cannot be found
More info:
http://docs.python.org/library/subprocess.html#subprocess-replacements
[YOCTO #2454]
(From OE-Core rev: a07d03cc6f67c88feb9813ae7deb6e4a93552dfe)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Complete the bb.data.getVar/setVar replacements with accesses
directly to the data store object.
(From OE-Core rev: 2864ff6a4b3c3f9b3bbb6d2597243cc5d3715939)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a problem with package names from inherits showing up
(From OE-Core rev: edb2a11994ac21e790155ea519bded4b37ef9307)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distro_check.py: Create a new function called create_log_file to reduce a lot of repeat code in distrodata.bbclass.
We needn't to create log file in function save_distro_check_result, because the log file has been generated in check_eventhandler.
Another bug is that we maybe access the /tmp/Meego-1.0 before we create this file.
Add a judge statement to decide whether we need to create this file firstly.
distrodata.bbclass: Add a new task checklicense to collect missing text license information.
This can help package-report system to know how many recipes are missing license text.
(From OE-Core rev: b41148cda9f0cc292b662a8473f26bc1ee0148f3)
Signed-off-by: Mei Lei <lei.mei@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipe meta-ide-support's alias is Poky, but not defined in distro_exceptions, this will lead an error.
The compare_in_distro_packages_list function omit a split process, this will lead to a form issue.
Change the log file generate way, use symbolic links link to the latest log file, instead of appending every log file in old log.
(From OE-Core rev: 123a7f49753aef4d5ccb5f5a5590c3c88775c6d0)
Signed-off-by: Mei Lei <lei.mei@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distro_check.py: Add DISTRO_PN_ALIAS check for Poky, OpenedHand,
Intel and Upstream, to mark packages that are
not in known upstreams
distro_tracking_fields.inc:
Merged private files into 1 master file
Added additional Maintainer Info
Added Distro Alias information for Poky and OpenedHand
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
datetime checking is changed to be once per day
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
distro_check: fix for natives, cross, and initial recipe types
Signed-off-by: Saul Wold <Saul.Wold@intel.com>
This adds a new task (distro_check) for each recipe.
The task generates the source package list for Fedora OpenSuSE
Ubuntu Debian & Mandriva Linux distros.
As one recipe or source package can generate multiple target packages
the recipe name is compared with the source package name list of LInux
distributions.
Thread locking is used to avoid multiple threads racing for the
package list update.
Then the recipe name (PN) is checked if it exists in the package
list of distros. And if the DISTRO_PN_ALIAS then it is used to copmare
pacakge_name instead of the PN variable. Just for example the
DISTRO_PN_ALIAS can be defined in the recipe (.bb) files like this
In the file xset_1.0.4.bb:
DISTRO_PN_ALIAS = "Fedora=xorg-x11-server-utils;\
Ubuntu=x11-xserver-utils; Debian=x11-xserver-utils;Opensuse=xorg-x11"
The final results are stored in the tmp/log/distro_check-${DATETIME}.result
file.
FYI this command will generate the results for all recipies:
bitbake world -f -c distro_check
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
