The url returned by the reverse lookup ends in /submit/thanks/ but
the regex was improperly searching for a string without the terminating "/".
Thank you to Pawel Zalewski for reporting the 404 error.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
I noticed in the branch comparison plain text view ("Plain text" button
in the Tools -> Branch Comparison page) that in current versions of
Firefox the ellipses were coming through corrupted, though they looked
fine in the HTML version, and it turns out this is because I wasn't
specifying a character set encoding. It should be UTF-8, so add a
charset to the content type stating as such for this and other similar
views.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
Added SPDX identifiers to all .py files except those in migrations directory.
Fixes: [YOCTO #13527]
Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
Removed all obsolete references to django.core.urlresolvers. Added the
newly required on_delete fields to foreign key relationships in models.py
and in all migrations.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
Add an extra tool that lets you view all of the recipe dependencies in
a layer. There is also a mode that shows only cross-layer dependencies,
which can be useful to find dependencies on recipes in other layers
that aren't declared in the layer's dependencies (or conversely where a
layer dependency is no longer necessary).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to compare available recipes and their versions between
two branches for a selection of layers (default is just OE-Core). This
was mainly intended to help us with the Yocto Project release notes
preparation (hence the "Plain text" button at the bottom of the page)
but is also useful in its own right.
Note: for readability, SRCREVs are only shown when PV has not changed.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Without this change the system will fail parsing various URL components
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I forgot the lesson of f5922091b4 which
was that djangorestframework has a very silly default of registering
the model name as the key for finding the ViewSet, with the result that
the recently created RecipesExtendedViewSet was also being used for the
recipes URL which we expect to point to RecipeViewSet; this broke the
bitbake-layers layerindex-* commands.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Adding these extra child items to the standard "recipes" viewset (which
we did recently in 684a06a383) means that
some current usage is impractical due to the size of the returned list
of items. Instead, create a recipesExtended viewset, move the new child
items to that and add pagination to avoid result size issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These are a little less useful than the other items but if we want to be
able to create a clone of a layer index then we need them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Indentation, spacing, and wrapping of a long import line.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When you make changes to the infrastructure it can be useful to test
that email sending is working, since for that to work that involves the
code, Celery, RabbitMQ and SMTP being functional. However, up until now
to run a test you needed to submit a fake layer which is a bit annoying.
Add an explicit "Test email" option to the Tools drop-down for staff
users to allow them to send an email to themselves.
Note: the page will come back when the Celery job has been created, it
does not check and report on the job status - you need to look on the
server side to see that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For situations where the user launches a distro comparison update
process and then shortly afterwards realises it is operating with the
wrong configuration (or is otherwise broken) and is going to take a long
time to finish, add a button to the task page to stop the task. This was
tricky to get working, since the default behaviour of Celery's revoke()
would either terminate both the Celery task process along with the update
process (leaving us with no log saved to the database) or worse not even
kill the update process, depending on the signal sent. To avoid this,
send SIGUSR2, trap it in the task process and kill the child process,
returning gracefully. To make that possible I had to rewrite runcmd() to
use subprocess.Popen() instead of subprocess.check_call() as otherwise
we can't get the child's PID.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were refreshing the page constantly in order to show output while
a task was running, which basically worked but is horrible. Instead,
write the task output to a file and then use AJAX calls to request
whatever output has been written to the file since the last call
and call this roughly every second. Put the output in a scrollable <pre>
element instead of making it the length of the page, and auto-scroll
to the end (unless the user grabs the scrollbar and pulls it upwards -
it may not be immediately obvious that you can do this if there is a lot
of output since you have to pull it up when the scrolling animation is
not running, but it is possible).
An alternative would be to have used some kind of long-lived HTTP
session or a websocket, but those come with their own set of problems
so I elected to use this much simpler method.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Specifying the covering recipe in the comparison recipe detail page was
always a bit awkward - you could only type the name, if you wanted to
actually find a recipe or look up the currently selected one's details
then you had to open another browser tab/window. To fix this, replace
the form on the comparison recipe detail page with a side-by-side
display of the covering recipe's information, along with a button that
lets you search and then select the covering recipe and at the same time
enter comments or set any of the other cover fields.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Comparison updates might involve some custom fetch process, so provide a
mechanism to register these via settings.py on a per-branch basis. If
an update command is defined for a branch and the logged-in user has the
new "update_comparison_branch" permission, an "Update" button will show
up on the recipes page for the comparison branch for authenticated
users that will trigger the command in the background (as a celery job)
and then show a page that displays the status. The status isn't shown in
real-time since that requires quite a lot of plumbing, but the page at
least auto-refreshes.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to export the comparison search results to CSV format in
order to allow importing the data into external tools.
Note: I implemented this in a different way than the earlier recipe CSV
export, i.e. it uses a template to render the CSV instead of a
function-based view with the Python csv module - the reason for this is
we can reuse the same view as we use for producing the search, with all
of the flexibility that gives us.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When doing the reversed query it's often desirable to exclude recipes
by inherited class, for example those that inherit the packagegroup,
image and meta classes as they don't actually build anything and thus
aren't going to match up with anything in the other distribution.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a lazy-loaded popup for selecting layers to include in the query
instead of having it as a simple drop-down, so you can select more than
one layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Turn the existing OE-Classic support into something a bit more
generic so we can import data from other distributions and compare it to
what we have in layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We're about to replace this with a proper CSV export function, so we
don't need this dead code hanging around anymore.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The patterns() function is deprecated in Django 1.8 and gone in 1.10, so
we should switch over to the new list format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Rename publish() to publish_view() and call it directly from the view
rather than using a string name (which is not possible in Django 1.10+).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a page with basic statistics for the index - number of layers,
recipes, classes, machines and distros on an overall basis (distinct
names) and per branch, since I've been asked a few times for this kind
of information. It's currently only linked from the Tools menu for
logged-in users, but the URL will work for anyone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For reasons that are not immediately clear to me, if you don't specify a
name when adding the ViewSet it takes one from the model used in the
queryset in the ViewSet. The new layers view uses LayerBranch and so it
silently replaced the layerBranches URL in the router, even though the
URL itself was still present. Unfortunately that's broken Toaster.
Specify a name to restore the old URL.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It was a bit awkward to query layers externally - you had to do multiple
queries and you couldn't get the YP Compatible version info at all. Add
an additional LayerBranch-based view that exposes the branch name,
layer fields, YP Compatible Version and active maintainer information
with just one call.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Layer maintainer and layer note information wasn't available through the
REST API since it wasn't needed for Toaster, but for other uses it is
useful.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add another tab to search for classes.
Fixes [YOCTO #11207]
Signed-off by: Amanda Brindle <amanda.r.brindle@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
At the moment it's a bit difficult to get update logs out of the
environment in which the update script is being run. In order to make
the logs more accessible, create a LayerUpdate model to record the
output of update_layer.py separately for each layerbranch and tie the
created LayerUpdates together with a single Update model per session.
We provide two ways to look at this - a Tools->Updates page for
logged-in users, and there's also an "Updates" tab on each layer that is
accessible to anyone; which one is useful depends on whether you are
looking at the index as a whole or an individual layer.
Update records older than 30 days are deleted automatically by default.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the distros to the index. This looks a lot like the machines
and allows users to search for a particular distro.
Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com>
Added associated migration.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Django 1.8 warns that the default for the "permanent" parameter to
RedirectView is changing in 1.9 from True to False, but I believe we
should be specifying False for these redirections - these are not just
redirections from old URLs and may in fact change in the future if the
site structure changes.
Part of the implementation for [YOCTO #9620].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We need to be able to support Python 3 so that we can parse master of
OE-Core with bitbake (which now requires it). This now means the
interface itself and the update script require Python 3.4+.
Part of the implementation for [YOCTO #9704].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I'd like to be upgrading to 1.8 but that causes problems with South, and
we're not quite ready to dispense with our existing migrations yet.
Part of the implementation for [YOCTO #9620].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This patch enables a read-only REST API for the layerindex
application using Django REST Framework.
The objects of types Branch, LayerBranch and LayerItem are
exposed to queries so that the layerindex application can
function as a Layer Source in Toaster.
The library dependencies are documented in the requirements.txt
file.
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
It's not really completely functional and puts quite a bit of load on
the server when used, so just disable it for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Drop the front page - this just gets in the way. Redirect to the
layers list instead. This has meant adding a touch more text to the
about page and adding the FAQ link to the footer.
* Use a separate navbar to hold the branch selector and the main
top-level pages (Layers, Recipes, Machines) instead of tabs
* All pages depending on a branch selection are now under
branch/<branchname>/ so we don't need to have the branch selection on
every page.
* Use breadcrumbs on recipe detail and layer detail pages instead of
tabs
* Add title to recipe detail page
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a script for doing a one-time import of OE-Classic recipe
information, so comparisons against OE-Core can be performed; this
is stored using a new ClassicRecipe model supporting additional fields
for tracking migration status. The migration status fields can be
updated as well as viewed and summarised in graph format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This provides a way to set "meta" fields (SUMMARY, DESCRIPTION,
HOMEPAGE, BUGTRACKER, SECTION, and LICENSE) for a number of recipes at
once, and then download those changes in the form of one or more patch
files which can be submitted for merging into the layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The recently added global redirect was also redirecting incorrect URLs
within the application to the front page rather than showing 404, which
is not desirable.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Give the user a friendly introduction instead of plunging them straight
into the layers list. (Of course if people want to link directly to the
layers list they still can.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This cuts out a lot of the elements that aren't needed for the review
list, shows fields in a more suitable way for review purposes than the
standard detail (and includes some fields that don't currently get shown
on the standard detail e.g. layer type and short description).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The simple views for about and the submit thanks pages don't need
special views, so the standard TemplateView can be used.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
