package_custom_regex_comp is built with the current package name and
then used to search upstream version this reduces custom regex'es in
sites that have different packages in the same directory.
(Bitbake rev: a69d5d48c4bf9a3df3121cc4e69ba5d7e947ad8d)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Latest version string only try to find latest directory when REGEX_URI
isn't specified to avoid unnecessary processing and makes code easier
(Bitbake rev: afc33ec7cdb7d8ee3602a23fa973551ca5510ac4)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to query whether updated versions of a url are available
is useful, not least for the package reporting system. Since such code
is closely linked to the url type and the url itself, the fetcher
makes a locical place to contain this code.
For wget based urls this means taking upstream directory listings
and searching those for later versions, returning those that are
found.
The patch also adds unittests for this function so that if
improvements are made, the original test urls can be used
to evaulate the those changes.
This is based on code from Irina Patru <irina.patru@intel.com>.
(Bitbake rev: a8272e22b7819e0e8afd8e291d276f5f28fc0007)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the previous cleanups broke the downloadfilename functionality.
This change fixes the code to ensure the commandline is correctly built.
Thanks Kristof Robot <krirobo@gmail.com> for reporting the issue.
(Bitbake rev: e008d9bb07e5d1a3584cc04ca2cd3dd906fd5759)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These two functions have little in common, separate them out.
(Bitbake rev: 7413e0fa4dca9571ea98f32dab87d4fd60bc8de9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Start to clean up wget fetcher command construction to allow clearer
and more extensible code structure. Drops support for ${URI} and
${FILE} directly in the commands.
(Bitbake rev: 4e59fe45be2088996abc21e9a631a32b9a9642c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These variables were dropped from OE-Core some time ago, drop their
usage from the fetcher as well.
(Bitbake rev: bd33e709ab65d6966b234010641861834d170e2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I can't think of a reason we'd download zero sized files however there are
reasons zero length files can accidently make it onto source mirrors.
This check allows us to ignore the broken files and switch to another
mirror rather than fail with odd checksum failures.
(Bitbake rev: 300cba2e1a720dba4b83b0c76208ea93c608c1de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to keep passing around the url parameter when
its contained within urldata (ud). This is left around due to
legacy reasons, some functions take it, some don't and its time
to cleanup.
This is fetcher internal API, there are a tiny number of external users
of the internal API (buildhistory and distrodata) which can be fixed up
after this change.
(Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there there isn't a subdirectory to the downloadfilename, this was
failing. This patch avoids that issue.
(Bitbake rev: 58bfd8d88495d4cae808e23b7af40e65ad05450f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we pass the -O option whilst spidering, empty files are created which is not
desired. We also need to ensure any subdirectories are created when using the
downloadfilename parameter.
(Bitbake rev: d5f78e98d5aba36c95288fbaac267c2d54537b02)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wget commands for check and resume were
mixed-up, leading to the following issues:
1. long running "NOTE: Preparing runqueue"
reason: objects were downloaded, not spidered on the mirror
2. Failing network test in Build Appliance, because wget 1.14
(in use in BA) will fail if a file already exists.
During the network connectivity test, index.php file was
actually downloaded, not spidered (checked for existence on
yoctoproject.org website), leading to wget failure.
(Bitbake rev: d7a5185cae975eaca50a9785c6605e895dc7bb51)
Signed-off-by: Cristian Iorga <cristian.iorga@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is a common mistake to use http:// and protocol=git when attempting
to fetch from a git repository using the http protocol; if this is
detected then throw an error explaining that you need to use git:// with
protocol=http instead.
(Bitbake rev: 5bc4930c1638db16bcd5f9c8cfc4081f9ffc192b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
this allows wget fetcher to store the downloaded file in a specified custom filename in ${DL_DIR}
Exmaple:
SRC_URI = "https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz"
This fixes bug:
[YOCTO #2570]
(Bitbake rev: ceb5871007f221c4d86a7bee421d4dd8d9100aaf)
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This also changes to use -t 2 -T 30 as the defaults which are more sane for
a modern fetcher and already specified in OpenEmbedded metadata.
(Bitbake rev: bf0e5dddf0f63cdb0648fb6d872af5ceef6fbfb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were some hardcoded behaviours in the system for which backends
support checksums verses which backends recommend them verses which
don't recommend them.
This moves the functionality into specific fetchers and then makes the
general code generic. This cleans up the codebase and fixes some corner
cases such as trying to checksum directories returned by the git fetcher.
(Bitbake rev: ef6d268f7b8527541a7fb044cf95a973be4097f4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows some simplifications of the function and is slightly faster.
(Bitbake rev: 5999dc9985ad087c036611bfaa59b090a08781a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the wget fetcher to not emit the commands output when calling checkstatus, this
matches the behaviour of the git fetchers checkstatus() method.
(Bitbake rev: 5896d8ff7bcf85b3bab20fc27c4c15e87f86b4d9)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch2 is copied from bb.fetch, and has many bb.fetch referrence.
Fix these referrence with bb.fetch2 referrence
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
