Change dblog_watchdog to use Drupal Queue instead of DB insert.

So, it sounds like what you're attempting to accomplish is allowing the current execution to continue and not wait for the database insert to complete. This can be accomplished with INSERT DELAYED in MySQL, or it could be accomplished by loading all db log entries into (drupal_static variable? MemoryQueue?) and processing them in a shutdown function.

In that case, the log entries would still have to be queried out of the mongodb queue backend via a worker and inserted into MySQL, so I'm not sure what the gain is over using INSERT DELAYED or inserting during a shutdown function (except that the processing could occur on a different server). If you're wanting to simply log watchdog entries to a different backend (such as mongodb), there are modules available for that, such as mongodb/mongodb_watchdog.

One thing we might do in this case is rely on the Drupal\Core\Queue\Memory queue backend which is essentially a local buffer.

The problem with any such buffering (think Monolog BufferHandler) is that if any page encounters a fatal failure, nothing gets logged at all, so you don't even get any trace that anything has gone wrong. It means we may need to add an ability to force a flush (but this is not present in PSR-3 or Drupal's own logging API), or use some strategy to stop queuing in some conditions (think Monolog FingersCrossedHandler).

It also means there has to be a setting to switch queuing on/off in dblog, and possibly one for a fingerscrossed-like behaviour.

Working on an initial patch at DC Vienna.

Here is an initial stab at what it could look like:

• config now includes queue.enabled and queue.level
• any log below the minimum level logs everything in waiting
• any log above the minimum is just enqueued
• synchronous logs remaining the default for backwards compatibility and for safety

Note that, unlike the summary, it is still written at the end of the page, since this uses the in-memory queue to limit impact and make it available on all sites. It's just buffering, in the end.

Regarding deferred building of the logs from an external queue is something which does not make sense without contrib, since memory queue is empty after the page end, and the default DB queue, well... is written to the DB anyway.

Kwalify expects "integer", not "int". Rerolled.

* DRYed the config name by extracting it to a constant
* Moved the setttings form building to a class
* Synchronized form label/description with the schema.

Gave more thought given to the off-page-cycle writing using an external queue like MongoDB or Redis: one downside would be the fact that this would make the DB writes extremely bursty, with possibly very high quantities of rows being written during these cron/worker jobs in rapide succession, likely to bury the DB server under the write load and disturb the normal page production far more than the spread-over-time behavior we have with the current implementation or with a buffered one like this patch.

Overall, what I've seen over the years is rather the straight use of MongoDB Watchdog, or pushing to syslog or local filesystem buffering, thence Filebeat (Lumberjack, LMogstash forwarder), injecting (directly or not, via Logstash) to Elastic for display using Kibana. In all cases, this gets DB load entirely out of MySQL (which is the main goal), and brings a much improved UI: MongoDB Watchdog remaing on-site and is easy to use and quite convenient, while the BELK stack is considerably more flexible but requires a lot more system administration and configuratoin.

+++ b/core/modules/dblog/src/Logger/DbLog.php
@@ -109,4 +155,59 @@ public function log($level, $message, array $context = []) {
+    if (!$this->useQueue) {
+      // ksm("Not queued write $message");
+      $this->dbLog($level, $message, $message_placeholders, $context);
+    }
+    elseif ($level <= $this->flushLevel) {
+      // ksm("Queued flush write $message");
+      // Log previously stored message.
+      $this->dbLogQueue();
+      // Then log this one to preserve ordering.
+      $this->dbLog($level, $message, $message_placeholders, $context);
+    }
+    else {
+      // ksm("Enqueueing $message");
+      $this->queue->createItem(compact('level', 'message', 'message_placeholders', 'context'));
+    }

The idea of having an alternative backend for dblog is considered here: #2458191: Provide a storage backend for dblog module. I think this will simplify the logic behind the code provided here. Once you be comfortable with the approach of using I queue for dblog, I think your review on that patch could give us and idea of if is flexible enough to support cases like this.

Since this this issue was created 10 years ago, and given contrib module can write to any backend without needing to write to the db, like redis_watchdog or https://git.drupalcode.org/project/mongodb/-/tree/8.x-2.x/modules/mongod... and potentially in the future use the same dblog ui thanks to #2401463: Make dblog entities I'm marking this issue as won't fix.