Problem/Motivation
When the Drupal cron runner is executing a queue, and if a queue worker throws a \Drupal\Core\Queue\SuspendQueueException, a RfcLogLevel::ERROR level log and backtrace are logged, and potentially recorded to a database of some kind. Throwing a SuspendQueueException exception doesnt needed to be treated so severely. For example in the normal course of business I exceed the quotas of an external API, I need to suspend the queue temporarily. This does not warrant logging an 'error' and backtrace.
Proposed resolution
Log a less severe log entry.
Dont log the exception and backtrace.
Remaining tasks
API changes
Data model changes
Original report
In the related issue you can requeue a single item back to the queue.
There already is functionality to suspend the whole queue by throwing a SuspendQueueException. Downside of this is that a watchdog is written. In my case this is not needed. I just want to stop processing the queue in the current cron-run and try again in the next one.
I'm not sure how to go about fixing this. For now, I've made a custom patch which adds a boolean "preventWatchdog" to the SuspendQueueException and in Cron.php, this is checked. I'll add the patch in the first comment.
Side note: watchdogs/exceptions/etc during cron-runs are being emailed, which results in quite some emails which are perfectly fine. I'd rather have an email when there really is a problem.
| Comment | File | Size | Author |
|---|---|---|---|
| #21 | 2867001-suspend-queue-logging-21.patch | 10.89 KB | mstrelan |
| #16 | 2867001-suspend-queue-logging-16.patch | 11 KB | dpi |
| #16 | interdiff-2867001-suspend-queue-logging-15-16.txt | 1.51 KB | dpi |
Issue fork drupal-2867001
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
spadxiii commentedComment #3
idebr commentedComment #11
dpiThis is about the core Cron runner, which is especially noisy:
Right now the runner treats
SuspendQueueExceptionas erroneous, creating a watchdog exception. #2667294: Allow for peaceful requeueing simply changed the logic soRequeueExceptionis caught, which prevented the default\Exceptioncatch from logging as an exception.I think the change here should remove the
watchdog_exceptioncall entirely, and at best log a normal ~debug-level log entry.Comment #12
dpiComment #13
larowlanThat seems reasonable to me.
Comment #14
dpiComment #15
dpiLets try this.
Comment #16
dpiNinja CS fixes
Comment #17
neclimdulNeed to refresh some memory on expectations around this make sure it wouldn't break anyone's expectations but this seems to make sense. I think this may be similar to how drush is handling it already.
One thought skimming the patch, the "broken" test seems like it was probably valid just using the wrong exception. Is there another test that covers the handling of broken queues throwing exceptions or was it really only testing the suspend exception?
Comment #18
dpi\Drupal\Tests\system\Kernel\System\CronQueueTest::testUncaughtExceptionsmakes use of a queue worker (\Drupal\cron_queue_test\Plugin\QueueWorker\CronQueueTestException) dedicated to throwing the generic\Exception.The
// Get the queue to test the specific SuspendQueueException.comment seems to indicate this was just for Suspend.Right, Drush's current behavior in
\Drush\Drupal\Commands\core\QueueCommandsis to terminate by throwing an exception, which may end up in stdout. Nothing is logged.Comment #20
kim.pepperComment #21
mstrelan commentedReroll of #16 against 9.3.x
Comment #29
smustgrave commentedThis issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.
Appears the MR has test failures.
Have not yet reviewed.
Comment #30
acbramley commentedI'm not sure what the old logs were before (with
$this->anything()coming first) but now we just have our log and then the cron run completed notice.Comment #31
smustgrave commentedThanks.
Tagged for a change record to announce this new behavior.
The code changes look good to me so +1 there.
Comment #32
acbramley commentedhttps://www.drupal.org/node/3343512 drafted.
Comment #33
smustgrave commentedperfect and adding the example really helped!
Comment #35
catchMakes sense and the changes look good, more of a task than a feature.
Committed ccae3c0 and pushed to 10.1.x. Thanks!
Comment #37
quietone commentedPublished change record.