There are much better ways to handle this and most editors shouldn't need this
in modern times, drop the noise from the files. Its not consitently applied
anyway.
(Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following commands are not supported, but they were ignored silently, that
may suprise users:
* addtask task1 task2
task2 is ignored
* addtask task1 before task2 before task3
Should be: addtask task1 before task2 task3
* addtask task1 after task2 after task3
Should be: addtask task1 after task2 task3
* deltask task1 task2
task2 is ignore
This patch can check and warn for them.
[YOCTO #13282]
(Bitbake rev: 675689aa7cc7287efecf8ef775ca2059369167f1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.
(Bitbake rev: c1fcc46e2498ddd41425d8756754f814d682aba3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous check was in data.py which only can check code like "python
funcname()" in the dependency chain, but there are 3 kinds of python functions:
- python()
- def py_funcname()
- python funcname()
Add the checking to BBHandler to check and warn for all of them.
The warning looks like:
WARNING: /path/to/recipes-core/busybox/busybox_1.29.2.bb: python should use 4 spaces indentation, but found tabs in busybox.inc, line 75
(Bitbake rev: 0cdc5b81fc1f5e5281a525a657e420ebc3bb9e90)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
- Add a comment in base.bbclass:
def oe_import(d):
import sys
# Comment
bbpath = d.getVar("BBPATH").split(":")
[snip]
Note, '# Comment' is started with '#', it is legal in python's syntax
(though maybe not a good style), but bitbake reported errors:
$ bitbake -p
ERROR: ParseError at /path/to/base.bbclass:20: unparsed line: ' bbpath = d.getVar("BBPATH").split(":")'
This error report would mislead people, the real problem is that '# Comment'
is not supported, but it reports the next line, this may make it hard to debug
the code are complicated.
We can make __python_func_regexp__ handle '^#' to fix the problem, since it
already can handle blank line "^$" in a python function, so it would be pretty
safe to handle "^#" as well.
(Bitbake rev: 79e62eef1c93f742bf71e9f25db57fdd2ffedd02)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The resolve_file() calls mark_dependency(), so the one which calls
resolve_file() doesn't need call mark_dependency() again.
(Bitbake rev: 4682571107323a39b42cd9ec8ee67419e7f15acc)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is more pythonic and can handle unclosed file warnings better
than the previous code structure.
(Bitbake rev: 50633012a64a3b5f0662145e29ff426374fb7683)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The functionality overlap between these two functions is significant and
its clearer to handle both things together since they are intimately
linked. There should be no behaviour change, just clearer code.
(Bitbake rev: 391aa4afc91be90d8d3ee47e1bf797d6ebe61a71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we're actively using the line numbers for other thins, having
magic values like IN_PYTHON_EOF causes problems, in particular, 32
bit overflow on 32 bit machines.
There is a neater way to signal eof to feeder(), just using an extra
parameter so use this instead and drop the IN_PYTHON_EOF magic values.
This has the added bonus that line numbers are then correct for
python functions at the end of files.
(Bitbake rev: e0f05871c2a6f1e86ae19ad343c7c6f822ddb67e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
(Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're not going to change the value of FILE, or we know it isn't
going to have changed (ext == bbclass), don't set FILE.
This avoids messy looking history of the variable as well as optimises
parsing speed slightly.
(Bitbake rev: 88e4600aa66dda2e6c807f9d97af8982bcd8817b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst bitbake has done this for a long time, the behaviour of resolving
class files against cwd is not desirable. This can be seen during
base configuration parsing when looking for base.bbclass where a dependency
on cwd is added. If cwd then changes, the cache is invalid and triggers a
re-parse.
The only real option is to drop this entry and if files can't be found, we
fix BBPATH in the cases where it needs fixing. I didn't find any in the
random selection of layers I tested parsing locally.
(Bitbake rev: 508aad9d5db7e51328b1fd6ee53b4bc3720a30b7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some sanity checks on the parsing state engine when returning data
so that incomplete functions raise parse errors.
This means a recipe doing:
do_somefunction {
echo 1
VAR = "1"
will now raise a ParseError. To get the right file/line information,
__infunc__ was changed to a list.
[YOCTO #7633]
(Bitbake rev: 6b54a72638f57882d4fd5aab96b2752a09e065af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the depths of time we were rather confused about naming. bb files
are recipes, the event to skip parsing them should be SkipRecipe,
not SkipPackage. This changes bitbake to use the better name but
leaves the other around for now. We can therefore start removing
references to it from the metadata.
(Bitbake rev: 98d9e6e0f514a7cb7da1d99bf4bd5602b89426d6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The file inheritance logging has been here since the dawn of time. It
duplicates output many times over and logs to debug level 2. When running
with the debug option, its understandable the user may want to see the
paths of files included in the build.
These changes remove pointless/duplicate output and print the
include paths clearly at debug level one in a form which
users should be able to more easily parse.
(Bitbake rev: 633a56987df639295fe9128418da5634050063e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Back in the depths of time we did support task removal. In the pre
AST days it was nearly impossible to continue supporting it, it wasn't
used so it was dropped. With the modern codebase we can easily now support
deltask and it would be very useful within the metadata since it can
massively simplify dependency trees.
As an example, a core-image-sato had 47703 inter task dependencies before
this patch and a patch to native.bbclass, afterwards with the noexec tasks
deleted, we had 29883. Such a significant simplification is worthwhile
and justifies adding a deltask operation to the system.
(Bitbake rev: acecbde6fb70ff3c96deab3cdf819d8442e87ed4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bitbake only adds files to its dependency list if they exist.
If you add 'include foo.inc' to your recipe and the file doesn't exist,
then later you add the file, the cache will not be invalidated.
This leads to another bug which is that if files don't exist and then
you add them and they should be found first due to BBPATH, again the
cache won't invalidate.
This patch adds in tracking of files we check for the existence of so
that if they are added later, the cache correctly invalidates. This
necessitated a new version of bb.utils.which which returns a list of
files tested for.
The patch also adds in checks for duplicate file includes and for now
prints a warning about this. That will likely become a fatal error at
some point since its never usually desired to include a file twice.
The same issue is also fixed for class inheritance. Now when a class
is added which would be found in the usual search path, it will cause
the cache to be invalidated.
Unfortunately this is old code in bitbake and the patch isn't the
neatest since we have to work within that framework.
[YOCTO #5611]
[YOCTO #4425]
(Bitbake rev: 78d285871e4b8c54ccc4602d571e85f922e37ccd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having a global method scope confuses users and with the introduction
of parallel parsing, its not even possible to correctly detect conflicting
functions. Rather than try and fix that, its simpler to retire the global
method scope and restrict functions to those locations they're defined
within. This is more what users actually expect too.
If we remove the global function scope, the need for methodpool is reduced
to the point we may as well retire it. There is some small loss of caching
of parsed functions but timing measurements so the impact to be neglibile
in the overall parsing time.
(Bitbake rev: 4d50690489ee8dc329a9b0c7bc4ceb29b71e95e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 is stricter about how files are accessed. Specficially:
* Use open(), not file()
* Use binary mode for binary files (when checksumming)
* Use with statements to ensure files get closed
* Add missing file close statements
(Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was meant to be squashed into the previous commit for multiline comment
handling. It fixes the case the commented multiline is followed by an empty
line which was resulting in a traceback instead of a sane error message.
(Bitbake rev: 7e7d692e244fe8dca533f842ca143b9c821e317c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Faced with an expression like:
# Some comment \
FOO = "bar"
what should bitbake do? Technically, the \ character means its multiline and
currently the code treats this as a continuation of the comment. This can
surprise some people and is not intuitive.
This patch makes bitbake simply error and asks the user to be clearer
about what they mean.
(Bitbake rev: 589d31ce41e019ee6a7cb6527d67bc76c0b6382a)
(Bitbake rev: 79c00fabe08b4c210a3bd81cfaffbc47ffdc2e2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SkipParse is raised from something which isn't anonymous python, it wasn't
being handled correctly. This improves the handling for example from within inline
python.
(Bitbake rev: 7467d7d66b24cc8f43ab168e65895e7c4aee6092)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The inherits cache contains duplicate entries, some with the full patch, some
just starting classes/xxx. This is a waste of parse time and potentially
error prone. This patch fixes various pieces of code so the absolute paths are
always preferred and work correctly. The inherits_class function did not work
with full paths so the patch fixes this.
(Bitbake rev: f3a71e509af196391ec126d079cf1bd178e62ad5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current usage of EXPORT_FUNCTIONS is rather problematic since a class
list (classes) is passed into the ast statement and cached as it was
when first parsed. This class list may be different in other cases but
is locked once in the cache.
Worse, the construction of classes can be broken by exceptions during parsing
at the wrong moments since the state of the parser is not always reset
correctly. This can lead to leakage of other classes into the classes list.
The current EXPORT_FUNCTIONS implementation looks at the last two currently
inherited classes and sets up an indirect function call view the second last
class inherited, e.g.:
do_configure calls gnomebase_do_configure
gnomebase_do_configure calls autotools_do_configure
This intermediary doesn't seem to serve a useful purpose.
This patch therefore makes builds deterministic and fixes various cache problems
and indirection by removing the intermediaries and simply performing
directly mapping for the cases where its needed.
(Bitbake rev: 9fc98f96f0e0320beda0ce9546275a99336732c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The classes variable in particular could be in an odd state if parsing of a
previous recipe had abruptly ended.
(Bitbake rev: 3dc5e896a97154914cee6c47900e3bb2a2627cdb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This removes some dubious functions and replaces them with a simpler, cleaner
API which better describes what the code is doing. Unused code/variables are
removed and comments tweaked.
(Bitbake rev: f1e943aebfb84e98253f3f21646d6765c4fa1d66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that statements that expand to more then one entry
such as:
CLASSES = "a b"
inherit ${CLASSES}
work correctly instead of trying to inherit a class called "a b".
(Bitbake rev: 2568e9ace6e6f483e1bf2a9ef2f4d8318d6c85b7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that a file and line number are reported for ParseError where
possible. This helps particularly in the case of inherit and require
which previously did not report either of these upon failure.
(Bitbake rev: f588ba69622a2df35417ced184e56c79ac1b40d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible for relative paths to creep into FILE. These confuse the
build system no end as its not clear where they might be releative to.
This patch ensures we always use resolved absolute paths for FILE
so that things behave in a deterministic way.
(Bitbake rev: 658d7daa70e46c2b20973b90ee53f0bbadc8bf5d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If metadata contains:
"""
FOO = "bar"
"""
The variable FOO should get set to "bar" but doesn't due to the empty lines
be swallowed by the parser and FOO becomming part of the multiline comment.
This patch corrects that behaviour so FOO is set as expected.
[YOCTO #1377]
This patch fixes parsing of multiline comments so lines ending with \
behave consistently and we warn users where there is something happening
they likely don't expect.
(Bitbake rev: 30eaef7f50fff855cf8830772a7088dd83a4240e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We will be needing this information to improve the tracebacks of python code
from the metadata, as well as to give the user information about where
variables were defined, so they know how it ended up the way it is.
(Bitbake rev: 9615c538b894f71a2d1a0ba6b3f260db91e75786)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use a custom Logger subclass for our loggers
This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug
(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
