Queue API should support item prioritisation

Great job! Patch looks very good. I just have some (minor) comments.

+++ b/core/lib/Drupal/Core/Queue/DatabaseQueue.phpundefined
@@ -72,7 +73,7 @@ public function claimItem($lease_time = 30) {
+      $item = $this->connection->queryRange('SELECT data, item_id FROM {queue} q WHERE expire = 0 AND name = :name ORDER BY priority ASC, created ASC', 0, 1, array(':name' => $this->name))->fetchObject();

My personal preference are higher numbers that have priority before the lower ones. It seems more natural to me. This being just my personal preference please take it just as a well-intentioned and optional comment.

+++ b/core/lib/Drupal/Core/Queue/QueueInterface.phpundefined
@@ -21,13 +21,16 @@
+   * @param $priority
+   *   Priority to be associated with the new task in the queue.
+   *
    * @return
    *   TRUE if the item was successfully created and was (best effort) added
    *   to the queue, otherwise FALSE. We don't guarantee the item was
    *   committed to disk etc, but as far as we know, the item is now in the
    *   queue.
    */
-  public function createItem($data);
+  public function createItem($data, $priority = 0);

Related to the previous comment. No matter which approach we take (lower-highest or high-lowest), it should be clearly documented here. It will prevent a lot of confusion.

+++ b/core/modules/system/system.installundefined
@@ -993,6 +993,12 @@ function system_schema() {
+      'priority' => array(
+        'type' => 'int',
+        'not null' => TRUE,
+        'default' => 0,
+        'description' => 'The priority of the item.',

As chx already mentioned on IRC we need to modify existing index.

+++ b/core/modules/system/system.installundefined
@@ -2095,6 +2101,22 @@ function system_update_8048() {
+function system_update_8049() {
+  // Add the {queue}.priority column if it doesn't exist.
+  if (!db_field_exists('queue', 'priority')) {
+    $column = array(
+      'type' => 'int',
+      'not null' => TRUE,
+      'default' => 0,
+      'description' => 'The priority of the item.',
+    );
+    db_add_field('queue', 'priority', $column);
+  }
+}

You probably do not need this function here, as you already create this field in update_fix_d8_requirements().

Looks good. I am just wondering if we could add just one index. If we'd put name on the first place in multi-column index there would be no need for single-column index on name. I was playing a bit with this idea and I think it could be achievable.

I was getting only "Using where" for query that loads new queue item for execution, which seems OK.

Looks good to me. Great job!

I asked msonnabaum to take a look at this patch. In the meantime, tagging as something that was RTBC in time for feature freeze.

This is not a feature we should include in the interface. Although it is supported by some queues, it's not supported by all (i.e. redis), which will make it very awkward to implement.

I've always solved this problem by putting high priority items in a dedicated queue, which works for most cases. If you need more than that, it can easily be solved in contrib.

I've always solved this problem by putting high priority items in a dedicated queue, which works for most cases.

Yeah I've done the same.

We already have queue features which arent supported by all backends (i.e. countItems() is not supported with any STOMP based queue backends,

I think we could remove that, calling code can track how much it's processed if it needs to, it's only used one place in core outside of tests.

and Beanstalkd doesnt support createQueue or deleteQueue).

It doesn't really need to though. There's a difference between methods that might be needed by certain backends to function properly - i.e. if they have to create a schema, vs. methods that just can't be supported meaningfully.