summaryrefslogtreecommitdiffstats
path: root/devel
AgeCommit message (Collapse)AuthorFilesLines
2011-11-17Ensure reporead is protected against simultaneous runsDan McGee1-100/+106
This adds a bunch of transaction magic and SELECT FOR UPDATE stuff to reporead to cope with the now-concurrent runs of reporead we get when invoked from our inotify-based updater. The collision occurs with 'any' architecture packages as both repo databases contain the new version, and the updates occur at exactly the same time. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-17reporead_inotify: nice the spawned subprocessesDan McGee1-4/+9
This prevents the reporead job from taking over time from more important processes; this is not a rush task. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-16reporead_inotify: close connection once we are done with itDan McGee1-0/+6
This prevents an otherwise idle connection from sitting around and being totally useless. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-16reporead_inotify: spin up read_repo() in separate threadDan McGee1-2/+9
This prevents memory usage from ballooning to absolutely huge values, such as when multiple threads kick off at the same time. The bulk of our memory allocation obviously comes in these threads and not the main threads, so being able to isolate them in processes helps a lot. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-16reporead: a few small tweaksDan McGee1-3/+4
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-15reporead: clean up some debug loggingDan McGee1-3/+5
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-15reporead_inotify: cancel threads that haven't started yet on shutdownDan McGee1-0/+5
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-15Add new reporead_inotify management commandDan McGee1-0/+188
This is the new on-the-fly updates hotness. Rather than continue to schedule reporead to run once an hour in cron or however else you ran it, this command can be run once and left running, and will automagically pick up on any database file changes and run an import. It operates on the files databases only; this will keep both the packages and files always in sync and remove the delay in updating, especially helpful for new testing packages. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-15Improve primary arch validationDan McGee1-7/+14
Ensure we can accept either a Arch object or an architecture name when passed to read_repo() by moving the validation there and being a bit more careful about typechecking and object lookup. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-15Add ability to cache users by username on the UserFinderDan McGee1-0/+16
This is very useful in the signoff message population script where we are very likely to encounter the same users over and over. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-11Show relevant signoffs on dashboardDan McGee1-0/+5
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-03Use UTC now everywhereDan McGee1-3/+3
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-11-01Really ensure we don't catch any NULL or blank valuesDan McGee1-4/+4
Fuck you too, Django. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-26Ensure PGP signature values are not trimmedDan McGee1-1/+5
This makes them totally unusable for any real purpose down the road. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-21Add a generate_keyring commandDan McGee2-1/+60
This grabs all the PGP keys from the developer profiles and adds them to the keyrings. Obviously we may want to do more in the future such as filter by groups, active status, etc. but this is just a first iteration. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-05Allow developer index to work with a non-authenticated userDan McGee1-1/+6
This is not the normal case given the decorator on the view, but during testing and development it is sometimes useful so others don't have to log in over a non-secure connection to check things out. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-17Add PGP signature package fieldDan McGee1-0/+1
And add eventual display code for it to the details template, but don't show it yet as no packages will have it. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-16Add two new DB fields to reporeadDan McGee1-1/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-28Add a bad compression ratio reportDan McGee1-1/+18
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-28Add order_by to packager count queryDan McGee1-2/+2
No real idea why SQLite is returning wrong results without out this, but it is likely a bug in the ORM layer I'm not interested in digging into. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-23Add (hidden) ability to search by last packagerDan McGee1-0/+2
This is used from the developer dashboard to add a new column to the stats of # of packages for a given developer where they were the last to do the packaging. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-23Add a rematch_packager management commandDan McGee3-4/+80
This allows quick resolution of all unmatched packages, especially after tweaking the way find_user works. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-23Turn find_user into UserFinder classDan McGee3-58/+103
This moves the cache inside an instance. Also add a few more tests. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-23find_user: add tests and fix no email address caseDan McGee2-9/+62
If a packager string was passed in without an email address, we would blow up on the matcher and not try to find a user. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-23Move find_user method to devel utilsDan McGee2-50/+59
This could be handy elsewhere as well, and it is loosely coupled to anything else in reporead. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Fix out of date testrelease_2011-06-15Dan McGee1-4/+1
A version of this view is now publicly available, so it returns 200. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-15Refactor common select_related into manager methodDan McGee1-2/+2
For a Package object query, we almost always did .select_related('arch', 'repo). Refactor this into the manager as a 'normal()' method so we can avoid sprinkling the same logic everywhere. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-14Add named devel URLsDan McGee1-2/+2
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-14reporead: small memory/perf improvementsDan McGee1-3/+5
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-14reporead: two small cleanupsDan McGee1-15/+17
* Parse builddate when reading from repo database file * Use defaultdict where it comes in handy Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-13Add a long out-of-date developer reportDan McGee1-1/+6
This shows packages that have been marked out of date for more than 90 days in the repos. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-09Fix busted batch score on package removalDan McGee1-1/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-09reporead: allow batching of package updatesDan McGee1-9/+49
The real reason I originally added transactions to this code was to prevent half-updates; e.g. a package gets in without the matching depends values. We can safely commit between packages and resume processing the database at a later time. Take advantage of this fact and commit every so often in batch fashion if we have a lot of updates piling up. In the case of updating the files DB, this can really cut down on the need to hold open a long-running, statement heavy transaction and get the information public faster. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-09Management command cleanupDan McGee1-22/+5
Now that we aren't seeing odd segfaults and hung tasks, we can remove the traceback stuff from the scripts. Also use the 'io' module only, it has been long enough. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-06-02Add a PGP key field on the dev profileDan McGee1-0/+7
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-05-26Remove now uneeded pytz bug workaroundDan McGee1-3/+1
2011-05-15Allow screening developer reports by maintainerDan McGee2-4/+19
A simple link is added for each user, but the URLs are flexible enough to screen by any maintainer if you know how they are constructed. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-05-04Make uncompressed info query slightly easier on the databaseDan McGee1-1/+3
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-05-02Add filesizeformat filter to sizes in reports/bigEvangelos Foutras1-1/+8
We also add a new 'filesize' tablesorter parser that handles all the suffixes found in django's filesizeformat filter. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-29Add unneeded orphans reportDan McGee1-1/+8
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-29Change big packages reportDan McGee1-2/+3
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-29Use single quotes only in dev report viewDan McGee1-9/+9
2011-04-25Developer reportsDan McGee2-9/+54
This commit adds four initial developer reports that are hopefully useful to developers and packages in checking up on the state of things. They include: * big : the 100 biggest packages in the repos * old : packages built > 2 years ago * uncompressed-man : self-explanatory * uncompressed-info : self-explanatory There should obviously be some sort of index page to access all of these, so that will be coming soon. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-09Show more info about todolists on developer dashboardrelease_2011-04-09Dan McGee1-2/+6
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-09Rename local variables for clarityDan McGee1-3/+3
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-04-07Use UTC datetime objects everywhereDan McGee1-3/+3
Rather than the twisted mix of local times and UTC times we currently have. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-29Work out kinks in create new user viewDan McGee1-6/+17
We allowed repo selection before, but never actually called save_m2m() so selections would have to be repeated on the next page. Add in group selection as well to the form, and ensure we do all of our database operations in one transaction so it is a lot easier to test and roll back if things go wrong. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-29Log user additions via new user formDan McGee1-0/+15
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-29Show important fields first on new user formDan McGee1-5/+16
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-03-24Add provides to collections listrelease_2011-03-28Dan McGee1-1/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3-55-g3dc8 (git 2.39.1) at 2024-04-20 14:31:41 +0000