There are modules like backup queue that allow for tasks to run in background. I have also used the task queue to implement a few tasks as part of a custom development workflow (e.g. git pull). With a number of team members and lots of sites & platforms I have an aegir setup that is keep busy constantly running tasks during the day often a few tasks waiting in the queue.
There are tasks (like backup/migrate/clone) that can be time consuming/intensive on the server that can block the task queue for other less intensive tasks (git pull - which might take a few seconds to run). Some tasks are also less of a priority (backup) which is fine if they keep getting pushed to when the task queue is quiet.
I am looking to build a feature that will allow for tasks to be weighted and run based on a priority. The idea is for this to be configurable for a system, so depending on your requirements you could set the install's to have a high priority and it would mean that install tasks would always get put into the top of the queue and run ahead of a backup or other lower priority tasks.
Code to make use of this can be found in a sandbox https://www.drupal.org/sandbox/darthsteven/2730949
Comment | File | Size | Author |
---|---|---|---|
#16 | task_priorities-1895580-16.patch | 4.33 KB | helmo |
#11 | hosting-1895580-task-priorities.patch | 4.14 KB | Steven Jones |
Comments
Comment #1
anarcat CreditAttribution: anarcat commentedThe only problem I would see with such a system is that we loose the assumption for the user that tasks are ran serially. For example, a user may queue a "backup" job before a (say) "migrate" job, and assume they would run in that order. How would we deal with that assumption?
Comment #2
univate CreditAttribution: univate commentedMy thought is that by default all tasks will have a weight of 0 and run serially as it does now.
But through a configuration page you would be able to drag task types around to increase their priority - so you would have to actually choose to make a change to the order. This would not be something coded into the task definition.
The implementation would be a new table with the list of tasks types and weights which when the queue runs would join across to pull out tasks based on weight.
I am trying to think of a good example where it would cause a problem (something like backup before migrate might be an issue if migrate didn't do backup first itself - I think there are enough protections in Aegir that it shouldn't cause a problem that can't be fixed) - interested if anyone can think of a use case that will break something.
Comment #3
anarcat CreditAttribution: anarcat commentedFair enough - if it's optional, it makes sense. But it will make the task list a bit more confusing - make sure the task list reflects the task priorities. I think even now the task list display doesn't show clearly the queue...
Comment #4
ergonlogicComment #5
Steven Jones CreditAttribution: Steven Jones commentedSo I was bitten by this yesterday:
We have a platform with 600+ sites on it, and we migrated them from one platform to another.
Part of this process was to perform a backup on all of these sites, and the amount of disk space used grew and grew.
We do have a backup garbage collection system, but it couldn't run its tasks because the queue was running the previously queued migration tasks, which were just trying to use more and more disk space :(
I implemented a quick fix to select non-migration and non-backup tasks first, and then the queue processed normally and the server recovered.
I wonder if we can't use lovely D7 database layer to tag the query that gets the 'next' tasks to run so that other people can hook_db_query_alter the query to alter the definition of what the 'next' task is. That way, we can handle the weights and whatnot in contrib.
Comment #6
helmo CreditAttribution: helmo commentedComment #7
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedGoing to pick this one back up, since it's now totally possible to implement fairly easily.
Rough plan of attack:
_hosting_get_new_tasks
so that it uses a D7 styledb_select
and add a query tag to the query.I suspect I'll do 1 in the hosting module, and then spin the rest out into contrib for now.
Comment #8
ergonlogicSounds like a solid plan.
Comment #9
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedHere's a patch that does part 1, allows the query for 'get me next task' to be altered. I think that that'll be all the changes that are required in Aegir core.
These changes are also in the
dev/1895580-task-priorities
git branch.Comment #10
ergonlogicPerhaps we could document the availability of the tag by adding a
hook_query_TAG_NAME_alter()
example in hosting.api.php.Code looks good, but the implementation in contrib will be useful in testing this. Please post a link to that once it's ready.
Comment #11
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedThanks very much for the review.
Additional documentation for the hook added in the attached patch (and committed to git).
Leaving as Needs Work until the contrib module is ready.
Comment #12
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedSee https://www.drupal.org/sandbox/darthsteven/2730949 for the contrib part of the plan in #7.
Currently points 2, 3 and 5 are implemented. Point 4 is sort of covered in code, where delete and backup delete and login reset tasks are set as high priority.
Tested and working quite nicely :)
Comment #13
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedPoints 4 is done too now, and point 6 has a firm plan in #2731013: Priority at task view time
This should be ready for review again.
Comment #14
Steven Jones CreditAttribution: Steven Jones at ComputerMinds commentedAll the bits are there in the sandbox module now.
Comment #15
helmo CreditAttribution: helmo at Initfour websolutions commentedMentioned in the scrum as Issue of the Week .... who's got time to review?
Comment #16
helmo CreditAttribution: helmo at Initfour websolutions commentedHere's a re-roll ... also on https://github.com/aegir-project/hosting/pull/8
Comment #17
helmo CreditAttribution: helmo at Initfour websolutions commentedI've not used the sandbox code in but as it's not given me trouble either this week I'd say rtbc.
Comment #18
helmo CreditAttribution: helmo at Initfour websolutions commented