Commit Graph

1056 Commits

Author SHA1 Message Date
Tim Orling
271107f024 recipe{desc,parse}.py: BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE from the shell environment has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Exiting to allow enviroment variables to be corrected

Replace BB_ENV_EXTRAWHITE with new variable BB_ENV_PASSTHROUGH_ADDITIONS

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-10 11:52:10 -07:00
Tim Orling
8293eee252 layerindex/models.py: add Inactive-Upstream
Add the newish Inactive-Upstream upstream status.

Add 0046_alter_patch_status.py migration.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>

layerindex/migrations: update patch status

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-10 11:51:04 -07:00
Tim Orling
75f052d19d layerindex/urls.py: fix about url pattern
The url pattern was not including the trailing /

[YOCTO #14445]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-09 18:58:09 -07:00
Tim Orling
796d2455bb templates/*: staticfiles -> static
In Django 3.0 the staticfiles tag was changed to static

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 23:36:22 -08:00
Tim Orling
f074aece70 requirements.txt: bump all to latest
Run pip-upgrade and update all to latest

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:16:46 -08:00
Tim Orling
88b13592c3 Dockerfile: add back python2 dependencies
We have some layers and especially older releases that still support
python2. Add python2 dependencies back to the container so that the
older releases still function.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:38 -08:00
Tim Orling
722f1f3a37 rrs: add 0030_alter_recipeupgrade_maintainer.py miagration
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:36 -08:00
Tim Orling
c99a1ed4e7 Dockerfile: fix warnings
* Define LANGUAGE to quiet perl warnings
* Set DEBIAN_FRONTEND=noninteractive before apt call to quiet TERM/Dialog
  warnings

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:33 -08:00
Tim Orling
743c8f449d rrs/admin.py: drop curry import
Dropped in Django 3.0 [1]
Suggested replacements are functools.partial() or
functools.partialmethod() [2]

[1] https://docs.djangoproject.com/en/4.0/releases/3.0/
[2] 5b1c389603

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:14 -08:00
Tim Orling
8f648093c7 README.devel: update versions
This updated code-base should be compatible with
Django 3.1 and obviously 3.2 (but not 3.0 or earlier).

Django 3.2 requires Python 3.6+.

With the upgrade to Celery 5, it is safer to recommend
RabbitMQ 3.8.x, since that is what we are using.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:03 -08:00
Tim Orling
f1dbcde4c4 docker-compose: fix celery
In celery 5, the --workdir argument must come
before the 'worker' subcommand.

Without this, celery cannot load the layerindex
module and this causes the celery container to
continually restart.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:54 -08:00
Tim Orling
67679306b0 settings: set DEFAULT_AUTO_FIELD
New in Django 3.2

To quiet warnings, set DEFAULT_AUTO_FIELD to the
default value 'django.db.models.AutoField'

NOTE: The default value for newly created Django 3.2
      projects is django.db.models.BigAutoField, but
      this causes the need for a migration in 'captcha'.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:44 -08:00
Tim Orling
314d29deda requirements.txt: bump to Django 3.2 LTS
Upgrade to the latest Django 3.2.x LTS for
extended support up until April 2024.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:31 -08:00
Tim Orling
08d7f96e10 requirements.txt: update all to latest
Since we are on a "modern" version of Python in
Debian buster container (3.7) and a modern version
of Django (3.1.x) we should be able to run with
the latest of all dependencies.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:22 -08:00
Tim Orling
ecd1005ae2 docker-compose: bump mariadb to 10.3
10.2 was based on Ubuntu bionic 18.04,
bumping to 10.3 as it is the first release
on Ubuntu focal 20.04

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:10 -08:00
Tim Orling
49b41802e4 requirements.txt: bump to Django 3.1
Another incremental upgrade towards 3.2 LTS

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:59:02 -08:00
Tim Orling
a027959343 Updates for Django 3.0
* Update requirements.txt versions
  - Mostly update to latest pre-Django 3.1 versions
* Fix deprecated axes.backends.AxesModelBackend
  - settings.py
  - docker/settings.py
* Fix template syntax 'staticfiles' -> 'static'
  - base.html
* Add migrations for layerbranch classicrecipe

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:58:52 -08:00
Tim Orling
094c89bc70 requirements.txt: bump to Django 3.0
Incremental upgrade towards 3.2 LTS version.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:58:40 -08:00
Tim Orling
4e857f68f4 layerindex/*: make all shebangs python3
Several scripts still had /usr/bin/env python

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:58:28 -08:00
Tim Orling
1f41512758 dockersetup.py: letsencrypt 2048 bit rsa
The minimum length for an RSA pem is 2048 for the dummy cert.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:58:14 -08:00
Tim Orling
f9c640a7c0 Dockerfile: cleanup, install python3-wheel
Remove duplicate python3-pip, etc. lines
Need python3-wheel for bdist_wheel command

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 20:57:42 -08:00
Tim Orling
df4a764d59 dockersetup.py: fix EMAIL_USE_SSL/TLS
Need to concatenate str(boolean), not bool

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
2022-01-13 09:15:50 -08:00
Paul Eggleton
ccc1fa775b Report charset for text & CSV views
I noticed in the branch comparison plain text view ("Plain text" button
in the Tools -> Branch Comparison page) that in current versions of
Firefox the ellipses were coming through corrupted, though they looked
fine in the HTML version, and it turns out this is because I wasn't
specifying a character set encoding. It should be UTF-8, so add a
charset to the content type stating as such for this and other similar
views.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 11:07:46 +13:00
Paul Eggleton
308805f04a about: add Amber & Tim to contributors list
Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:20:31 +13:00
Paul Eggleton
fdc8209b29 rrs: make changes to support Django 2.2
Removed obsolete references to django.core.urlresolvers. Added the
newly required on_delete parameters to foreign key relationships in
models.py and in all migrations.

(Amber Elliot's 2d526f9b0d served as a
reference for these changes).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:58 +13:00
Paul Eggleton
e01254dd0d utils: ensure we show error messages on server startup
If a BBHandledException occurs that means some error was logged, so we
need to handle any pending events so that we can actually have the error
logged. Tinfoil should really be doing this for us but at this stage in
the release we can't really fix this there, so do it here for now.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:56 +13:00
Paul Eggleton
38e6288c7d update: fix setup of BBLAYERS
So with honister / current master we can no longer get away with
bypassing BBLAYERS - it now needs to point to the core layer at
minimum. This is fine, we just need to skip parsing layer.conf if we're
parsing the core layer or we get some extra warnings we don't need.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:53 +13:00
Paul Eggleton
fe0bd7ddf5 README.devel: update versions
Update version requirements in line with recent changes.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:50 +13:00
Paul Eggleton
848ca10920 requirements.txt: set proper version restriction for Django
Without a comma, pip3 will currently install Django 4.0 which does not
work.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:46 +13:00
Meh Mbeh Ida Delphine
2314fb2811 Add SPDX license headers to layerindex-web source files
Added SPDX identifiers to all .py files except those in migrations directory.

Fixes: [YOCTO #13527]

Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2021-04-21 08:22:51 +12:00
Tim Orling
3b2b1f4948 docker/settings.py: MIDDLEWARE
The docker/settings.py file also needs the change from
"Changing MIDDLEWARE_CLASSES to MIDDLEWARE."

Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Tim Orling
2da03f4711 Dockerfile: debian 10 "buster" and drop python2
Debian 10 "buster" actually has Django 2.2 available
python2 is now EOL, so drop all usage

Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Amber Elliot
6c2f153784 requirements.txt: Updating Django requirement to the 2.2 LTS release.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Amber Elliot
7482d57175 Replacing is_authenticated() with is_authenticated for Django upgrade.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Amber Elliot
4e35c18f94 settings.py: Replacing MIDDLEWARE_CLASSES with MIDDLEWARE for Django 2.2 upgrade.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Amber Elliot
2d526f9b0d Updating models and imports for Django 2.2 upgrade.
Removed all obsolete references to django.core.urlresolvers. Added the
newly required on_delete fields to foreign key relationships in models.py
and in all migrations.

Signed-off-by: Amber Elliot  <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Amber Elliot
9b9961a683 README: Correcting mailing list.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2020-06-30 05:37:09 +12:00
Paul Eggleton
a3b1696cfc Fix backup instructions when large logs are recorded
If the database contains task logs that are over the default max packet
size for MariaDB, then attempting to dump the database will fail, but it
won't be immediately obvious that that has happened - the gzipped sql
file will simply be truncated and have an error in it. To fix the
underlying issue, add an option to the example command for database
backup to increase the max packet size to 512MB. (Restoring the database
doesn't seem to suffer the same issue).

In future we should probably look at writing a script to do this and
in it properly check the return of mysqldump so that any problems get
noticed immediately.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2020-02-05 17:08:41 +13:00
Paul Eggleton
9cf0829f41 Fix parsing after AVAILABLE_LICENSES change
In OE-Core commit 8c9ef587fe499c612a878a1ab42092eb79b334ef an
AVAILABLE_LICENSES variable was added in license.bbclass where its value
is assigned with immediate expansion ( := ) with the result that it
looks in LICENSE_PATH for licenses; in turn LICENSE_PATH refers to
COREBASE. You might assume that COREBASE is always set, and normally it
is (since it's set OE-Core's layer.conf) - but in the layer index
context we do not parse layer.conf until a bit later, so it immediately
fails. The quick way to fix this is just to set our own (correct) value
for COREBASE and then AVAILABLE_LICENSES can be expanded successfully.

You might ask why we don't instead just set BBLAYERS such that we *do*
parse OE-Core's layer.conf - the answer is that that can have other
effects such as BBFILE_COLLECTIONS being set, and at least at the moment
the rest of the code isn't expecting that.

Fixes [YOCTO #13723].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2020-01-06 13:36:35 +13:00
Paul Eggleton
8e3cbb7038 requirements.txt: bump Django version to fix CVE-2019-19844
Fixes a vulnerability in the password reset process due to
insufficiently stringent validation of unicode email addresses.

https://www.djangoproject.com/weblog/2019/dec/18/security-releases/
https://nvd.nist.gov/vuln/detail/CVE-2019-19844

(The existing version specification would have selected the fixed
version of Django already for new installs, but bumping the minimum
ensures that it will be installed for upgrades with
./dockersetup.py -u as well.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-20 10:29:11 +13:00
Paul Eggleton
29dd3afa66 tasks: squash out CRs in task logs to avoid huge transfers
Reloading an existing update task page was taking an extremely long time
to fetch down the task log and then pegging the client CPU such that the
browser gave a warning. Digging into it, logs from a Clear Linux update
task can be of the order of 500MB in size (with all of the line
refreshing using CRs that happens during downloads), causing (a) the
transfer to take a long time and (b) the JS code that updates the log
text box to be extremely busy. If we're loading the entire log from
scratch (as we are when we refresh the page) rather than just getting an
update since the last poll, we don't need any of those line refreshes -
so squash them out before returning the data.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09 11:40:12 +13:00
Paul Eggleton
4fd3e9c923 update: ensure removed recipe dependencies are removed from db
If a recipe dependency (either static or dynamic) is removed from the
recipe when it is parsed, then we should ensure it gets removed from the
database as well.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09 11:40:12 +13:00
Paul Eggleton
365cfde783 Upgrade Chart.js to 2.9.3
Quite a few bugs fixed, as far as I know none that we observed, but good
to have. Details can be found here:

https://github.com/chartjs/Chart.js/releases

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09 11:39:47 +13:00
Paul Eggleton
20544033cd Upgrade jquery to 3.4.1
Minor bugfix releases, but we should have them anyway:

https://blog.jquery.com/2019/04/10/jquery-3-4-0-released/
https://blog.jquery.com/2019/05/01/jquery-3-4-1-triggering-focus-events-in-ie-and-finding-root-elements-in-ios-10/

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09 11:35:37 +13:00
Paul Eggleton
b4529f4b43 Upgrade Bootstrap to 3.4.1
https://blog.getbootstrap.com/2019/02/13/bootstrap-4-3-1-and-3-4-1/
https://blog.getbootstrap.com/2018/12/13/bootstrap-3-4-0/

These include a couple of CVE fixes, but this application already
sanitises data when rendering so it is unlikely to have been vulnerable.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-12-09 11:30:46 +13:00
Paul Eggleton
f84223f473 README.devel: mention python SMTP server
For debugging email functionality, python's SMTP test server is useful,
so add brief instructions on how to use it for convenience.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-26 10:44:37 +13:00
Paul Eggleton
60b03648b7 recipes: add help button to explain search terms
There's a bit of advanced functionality by now in recipe searching, so
add a link that shows a popup with information on how it works.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:51:30 +13:00
Paul Eggleton
bc653605fc recipes: allow searching for layer:oe-core
When using the layer: advanced query term, if you want to match on
OE-Core, its actual layer name is "openembedded-core", but people will
naturally assume that "oe-core" should work, so make it so (case
insensitive).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:51:30 +13:00
Paul Eggleton
745ffd6958 recipes: support pn: query prefix
Allow filtering only on recipe name - i.e., searching for "git" finds
any recipe with "git" in the name or description. Now, you can search
for "pn:git" which will return only recipes with the name "git".

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:51:30 +13:00
Paul Eggleton
7eea41de30 recipes: improved support for queries containing quotes
Use shlex.split() to split the query string so that quotes can be used
to delimit strings containing spaces to be matched as a whole. This
worked with the previous code, but it did not support single quotes -
these caused an error in Django's filter code and thus an internal
server error (as did querying for ""). Add some additional checks for
single quotes as it is still possible to get them past shlex.split()
e.g. with something like "'hello'" (with quotes).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:51:30 +13:00