Recently the -r/--reload option was changed to preserve existing recipe
data, so that it could be used to load values into newly added fields;
however we still need an option to load recipe data from scratch for
testing purposes (e.g. in combination with --nocheckout) so add a
--fullreload option to do this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In order to test updating we need to be able to check out a previous
revision of the repository and have the update script load from that
without checking out the latest revision, so fix the --nocheckout
option to work properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We always need to replace %path% in the URL, not just when there is a
non-blank path to replace it with.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Change Submit layer link into a button
* Ensure tools dropdown menu doesn't show on layer submit/edit page
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not really completely functional and puts quite a bit of load on
the server when used, so just disable it for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Since it's possible that the user will try to get patches while the
update script is running, ensure we show a sensible error if the script
times out waiting for a lock.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Avoid the possibility of these two clashing especially when multiple
branches are being used.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Drop the front page - this just gets in the way. Redirect to the
layers list instead. This has meant adding a touch more text to the
about page and adding the FAQ link to the footer.
* Use a separate navbar to hold the branch selector and the main
top-level pages (Layers, Recipes, Machines) instead of tabs
* All pages depending on a branch selection are now under
branch/<branchname>/ so we don't need to have the branch selection on
every page.
* Use breadcrumbs on recipe detail and layer detail pages instead of
tabs
* Add title to recipe detail page
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't start with all records being shown - only do this if the search
button is pressed with no keyword.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a script for doing a one-time import of OE-Classic recipe
information, so comparisons against OE-Core can be performed; this
is stored using a new ClassicRecipe model supporting additional fields
for tracking migration status. The migration status fields can be
updated as well as viewed and summarised in graph format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Having the page content dance about when switching between pages just
because one has a scrollbar and the other doesn't is annoying, so just
force the scrollbar to show all of the time to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The recipe detail pages use the recipe record id, so in case users
bookmark links to these pages, try to preserve the id when a recipe
upgrade occurs (which usually shows up as a delete and an add). This
also works when doing a full update with -r so that we can capture the
newly recorded items (bbclasses, bbappends) without having all the
recipe id values change.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This provides a way to set "meta" fields (SUMMARY, DESCRIPTION,
HOMEPAGE, BUGTRACKER, SECTION, and LICENSE) for a number of recipes at
once, and then download those changes in the form of one or more patch
files which can be submitted for merging into the layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
To allow re-use outside of the update script, split out parsing setup
code to a new recipeparse module. Also split out runcmd, get_layer,
get_branch and logger_create functions to a separate utils module.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Some layers don't use the conventional branch name (i.e. the same name
as the core branch). Add a field so we can record the actual branch name
and use this within the update script.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Collect bbappend/bbclass info during the update process and display it
on the layer detail page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Show BUGTRACKER in the recipe detail page if set
* Track the other two variables for possible future use
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an index_preference field to enable control over which layer's
duplicate recipes get de-emphasised in the recipe search results.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Save individual field changes into revision comment and display this
comment on the history page. Now we're ready to add a link at the
bottom of every page so the history is easily visible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This allows disabling the checkout of the appropriate branch so that we
can test the update script with fixes to bitbake or the metadata if
needed.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not really appropriate to say "Hi Joe Bloggs" at the start of the
email - we only need the user's first name or the login name if the
first name hasn't been filled in.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The cooker module has been refactored in BitBake master and this moved
and renamed the _parse() function; change the update script to handle
that (whilst still handling older versions).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Report layer which failed to fetch in error message
* Don't retry fetching a repo if it already failed for another layer
(where more than one layer is in the same repository)
* Exit immediately if all fetches failed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to forget you need to use -l to specify the list of layers to
update and previously we just ignored unused arguments, so check for
them and error out if present.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This includes the layer name in layer note additions/changes in the
history page, among other places.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Somehow due to some CSS shenanigans these checkboxes had 0 width in
Chromium and thus were invisible. A style on the td had to be used
because a style on the input doesn't seem to fix it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We shouldn't be looking for this attribute unless the user has logged in
- this was a regression in the recent email case fix.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This avoids showing a 403 error when a user clicks on a review link in
the layer submission notification email but hasn't logged in yet.
Also protect the review list view with a permission check; it's not that
it's sensitive, but we should be consistent with the detail here.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This actually works for complex URLs since we normalise just the path
portion that we are adding instead of trying it on the URL after
composing it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
gitweb, ever vigilant for the slightest transgression in URL syntax,
fails to find the specified path if it is a directory and has a trailing
slash, so we need to avoid adding this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow using e.g. ../README to point to README file in the parent of the
repository subdirectory.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Avoid truncation for slightly longer fields (as happened with PN and
SUMMARY when importing some existing layers).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Avoid picking up machine files in sub-layers
* Skip scanning of .git directory when doing full refresh
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fall back to NAME if DESCRIPTION is not specified and remove more
preamble from the string.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If another recipe exists with the same name in a different layer and
that other layer is "older" (by layer ID) and is a software or base
layer then lighten the recipe entry in the search results as it may
not be the preferred version (e.g. recipes in BSP or distro layers may
have been customised specifically for the machine or distro).
This has had a performance impact on the recipes list; as a result
showing all recipes by default has been disabled. If the user really
wants to see all recipes they can just leave the search box blank and
hit the search button.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the user edits the layer and changes the repository URL,
subdirectory, or dependencies, ensure that the entire layer gets
refreshed the next time the update script runs by clearing out the last
fetched revision field.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This ensures recipes that failed to parse sort properly instead of all
appearing at the start of the search results.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't go through the normal code path where this directory would be
created, and if it doesn't exist an error will be thrown in
insane.bbclass when any QA issue is logged; so create it manually.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Rather than letting BitBake error out, just check for existence of this
file and print a single error line if it doesn't exist.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should help the initial population of the database and act as a
simple example of how to populate the database programmatically.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're creating a LayerBranch record, it's helpful to copy the
maintainer and dependency records in from master as they're usually
going to be the same; if they're not the user can always change them
afterwards.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
So it turns out that one or two layers have changed in structure between
branches, so we need to be able to specify this on a per-branch basis.
Good times...
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The recently added global redirect was also redirecting incorrect URLs
within the application to the front page rather than showing 404, which
is not desirable.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This enables sorting the branch drop-down in a configurable manner (e.g.
so that you can have them in order from newest to oldest).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Give the user a friendly introduction instead of plunging them straight
into the layers list. (Of course if people want to link directly to the
layers list they still can.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This cuts out a lot of the elements that aren't needed for the review
list, shows fields in a more suitable way for review purposes than the
standard detail (and includes some fields that don't currently get shown
on the standard detail e.g. layer type and short description).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Newly submitted layers don't have anything other than a master branch,
so they won't display properly unless we have master selected; so just
add a parameter to submission email and review URLs to ensure that is
the case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't assume the current directory contains our conf/local.conf,
explicitly use the directory above where the script is located.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The simple views for about and the submit thanks pages don't need
special views, so the standard TemplateView can be used.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a badge next to the Review link (when shown for users with the
publish permission) on all pages showing how many layers need to be
reviewed, if any.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This means the usage URL can point to a file whose contents may be
different per branch (e.g. a README file).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Change the data structure to support multiple branches. At the top level
there is a set list of Branch objects, and then a LayerBranch object
between each layer and the maintainers, dependencies, recipes and
machines, so that the set of each can be different per branch. The
branch is a session option, and can be selected via a drop-down that is
shown for all pages.
Additionally, with this change we avoid the need to run the update
script within a build environment set up with oe-init-build-env - since
we need a specific version of BitBake per branch we now use our own copy
of BitBake which is fetched by the script itself. The update script will
need to be called multiple times however - once per branch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should prevent spamming even though this is less likely with this
kind of site.
The CAPTCHA does not show when editing, only submitting, and is also not
shown for authenticated users.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Don't specify action on form - this is best practice and avoids the
need to have separate form tags for submit and edit
* Add a short introduction to the submit form
* Focus first field on load only if first field is empty (avoids doing
this focusing on the edit form which is not really useful).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Hide all empty maintainer forms and have a button to "add" another
maintainer which just uses jQuery to show the next hidden form. Up to
10 maintainers can be added which should be more than enough.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we get bogus or duplicate entries we'll want the ability to delete
them easily before publishing (without needing to have access to the
admin interface), so add this ability. Being able to delete a published
layer might be a bit dangerous and is less likely to be needed so that
is disallowed for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This allows adding an important notice to a layer e.g. "this layer is
deprecated, please use layer xyz instead". Only one layer note can be
added through the interface although the data structures allow multiple,
so notes may be added programmatically without disturbing user-added
ones.
With this change we also add a get_absolute_url() function to the
LayerItem model and change the calls to reverse() for layers to use it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If editing rather than submitting we want users to be able to make an
existing maintainer inactive.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Show "unpublished" rather than "new" in dependencies list in submit
form, and mention publishing as the operation in the "thanks for
submitting" page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Add a label making it clear the layer is unpublished
* Hide recipes list header since there should never be any recipes (as
the update script doesn't process unpublished layers)
* Add test links for base URLs
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow users with publish permission to edit any layer, and users with
the same email address as one of the maintainers of a layer to edit that
layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Show button only if user has appropriate permission, and remove the
duplicate button at the end of the page as it's unlikely to be useful.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The number of machines is likely to be quite high and it may not be
immediately clear to users which layer machines can be found in, so add
a searchable index to help with this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Previously I had added a custom widget to handle this, but it turns out
it's not flexible enough - we want to style items individually (not done
yet) and reorder checked items to the top on refresh (done).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we don't do this then clicking on the link when you're already on the
submission form will reload it, blanking out all the values.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the submit layer form, if a user pastes/types in a name/email pair
(e.g. My Name <my.email@example.com>) then this will automatically be
split into the name and email input fields, saving the user from doing
this by hand (since this is a common notation within repositories and
documentation.)
Additionally, if there is any text after the email address then move it
into the responsibility field unless the responsibility has been entered
by the user.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We have django-reversion to handle auditing/history now so we don't need
an explicit field to hold the creation date.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an icon with some javascript to select the URL / subdir text so it
can be easily copied (since dragging can easily select unwanted text
from the rest of the page).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If there is more than one layer in a repository, we want the latest
commit in the subdirectory not the entire repository (since we use it
to determine the last commit date which should show how "fresh" the
layer is).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we aren't doing a full refresh, when changes to an included/required
file or an inherited class occur we need to update all of the files that
depend on them. BitBake records these dependencies, so read them and
record them against each recipe.
At the moment this only handles dependencies within the same layer; that
is probably sufficient to handle most changes for the purposes of
extracting the data this application cares about. A future improved
solution will probably involve making use of BitBake's cache rather than
and parsing all layers and their dependencies rather than individual
recipes.
This change also adds -x/--nofetch and -n/--dry-run options for
debugging as well as some further debug messages.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
"Fatal" errors during parsing a single recipe (such as finding legacy
staging) shouldn't stop us from parsing the rest of the layer. To do
this we need to catch BaseException instead of Exception as fatal errors
end up calling sys.exit.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Parse layer.conf files so that BBPATH is set correctly as well as any
other custom variables. This required the repository fetching to be
split out of the update process and done first so that we're sure we
have the data for the other layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Formsets allow us to have separate fields for name/email for each
maintainer, as well as being able to collect the responsibility field
value. Also split the form to be output field-by-field to allow styling.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Look for conf/machine/*.conf and add a record for each file found. These
are displayed in the layer detail page if any are present.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Set BBPATH so that files from the layer can be found when recipes use
include/require pointing to files with a specific path, or use
inherit to inherit a class in the layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
meta-ti has a recipe in it that causes parsing to fail with a fatal
error due to LICENSE not being set; for the purposes of the layer index
just set a default value for LICENSE so that this doesn't break parsing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the repo URL matches known public hosting sites (i.e.
git.openembedded.org, git.yoctoproject.org, github.com, gitorious.org,
bitbucket.org) then we can set the web URLs automatically and save the
submitter from having to figure them out themselves.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Some git web interfaces use different paths for files (blobs) and
directories when it comes to browsing; this means we need to have a
separate field for the base URL for files.
Additionally, Gitweb typically puts the path within the URL instead of
at the end (although the parameters can be reordered) and cannot handle
the file parameter (f=) being specified with no value. So, to add some
flexibility, add the ability to use a positional macro %path% to
optionally specify where to put the path. If needed, square brackets
can be used in conjunction with %path% to exclude parts of the string
when there is no path, i.e. we want to browse the root. For example, the
following base URL:
http://git.example.com/?p=somerepo.git;a=tree;[;f=%path%];hb=HEAD
would be translated to this with a path of "path/to/dir":
http://git.example.com/?p=somerepo.git;a=tree;f=path/to/dir;hb=HEAD
and this if no path were specified:
http://git.example.com/?p=somerepo.git;a=tree;hb=HEAD
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We want to force selection of a specific layer type, otherwise many
submitters will just leave it at the default value.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These can be used to show important notes (e.g. "This layer is
deprecated, use layer xyz instead.") as eye-catching alerts at the top
of the detail page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Changed to full screen layout to improve the table display
and get rid of the accordion for the layer information.
Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
The layer list has a javascript-based search which won't interact well
with server-based pagination, so disable it for the moment.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a Django snippet to make pagination display a bit nicer (compressed
form instead of showing all page numbers). In order to interact properly
with GET requests, an additional snippet was required to add parameters
to the URL (why doesn't Django provide this out of the box?).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use more traditional GET instead of POST method
* Search in pn, summary, description and filename instead of just pn
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Set custom title for admin pages
* Use reasonable field widths on editing pages
* Allow filtering and searching by useful fields
* Disable editing of recipe records
* Disable editing of layer fields set by the update script
* Show "Layers" instead of "Layer items" for layers
* Show "Layer dependencies" instead of "Layer dependencys"
* Show layer name in maintainers listing
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
By default, if DESCRIPTION is not set it takes the value of SUMMARY.
However, within this tool it is useful for us to be able to tell when
DESCRIPTION has not been set so we want it to be blank if that is the
case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The default value of HOMEPAGE ('unknown', set from bitbake.conf) is not
particularly helpful - we want it to be blank if it's not set, so just
set it to blank before parsing each recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If recipe parsing failed the pn field will be blank, but for display
purposes we can take a guess from the filename.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Just in case two users go to publish the same layer at around the same
time, avoid saving the record for a user who tries to publishes it after
the first time.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Collect history for changes, and allow browsing / reverting / comparison
within the admin interface.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
