I have a feeds importer that is scheduled to run via cron. I set the importer to run "as often as possible" so that I'm sure that when the cron runs at 11am and 5pm, the feed importer will run.

However, there are times that the job scheduler doesn't run because I don't see it in the logs. I experienced this 2 times. Once, it didn't run in the morning cron and another time it didn't run in the afternoon. It happened sporadically.

It is reported that the cron ran but there's no entry about the job scheduler so I thought this is a job scheduler issue rather than a feeds issue.

I'm sorry if I can't provide more details but I thought I should report it just in case someone else is also experiencing this.

CommentFileSizeAuthor
#7 2034973-scheduler-does-not-sometimes-run.patch1.56 KBTorreyToomajanian
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

captainpants’s picture

captainpants CreditAttribution: captainpants commented
Category: Support request » Bug report
Priority: Normal » Major
Issue summary: View changes

I'm getting the same issue as above.

Job Scheduler database records that run as expected have the data column set to "NULL" and scheduled column set to "0". Those job scheduler db records that literally skip a cron run have the data column set to "N;" and scheduled timestamp set to a unix timestamp of when the feeds import was supposed to run.

In addition to this, when a feeds import skips a cron run, there is a corresponding db record in the queue table.

I'm not too sure if this is a problem with the job_scheduler module or the feeds module.

captainpants’s picture

captainpants CreditAttribution: captainpants commented

I think I may know what's causing this.

I believe that if a scheduled job executed and somewhere within the job a notice, warning, error or exception are thrown, the scheduled jobs "skip" a cron run.

Personally, after patching the Disqus module, which was throwing a ton of undefined property notices while unpatched, none of my scheduled jobs were skipping.

Scratch that. What I described above only fixed skipped imports on one of my development environments and not the other. This is incredibly strange considering that there is no skipping on one environment considering that The code and settings are exactly the same between both environments. The only thing that differs between the two environments is the app server that runs the Drupal site.

akosipax’s picture

akosipax CreditAttribution: akosipax commented

I think we figured out what is causing the skipped jobs: Enabling/Disabling modules.

Everytime a module is enabled/disabled, the job_scheduler is flagged to be rebuilt or rescheduled. When the jobs are requested/flagged to be rescheduled/rebuilt, the cron hook skips the portion where jobs are dispatched.

Here is the relevant code responsible for this behavior. I cannot suggest a permanent fix; The quick fix that works for us is to run cron immediately after enabling/disabling a module.

function job_scheduler_cron() {
  // Reschedule all jobs if requested.
  if (variable_get('job_scheduler_rebuild_all', FALSE)) {
    foreach (job_scheduler_info() as $name => $info) {
      job_scheduler_rebuild_scheduler($name, $info);
    }
    variable_set('job_scheduler_rebuild_all', FALSE);
    return;
  }

  // Reschedule stuck periodic jobs after one hour.
  db_update('job_schedule')
    ->fields(array(
      'scheduled' => 0,
    ))
    ->condition('scheduled', REQUEST_TIME - 3600, '<')
    ->condition('periodic', 1)
    ->execute();

  // Query and dispatch scheduled jobs.
  // Process a maximum of 200 jobs in a maximum of 30 seconds.
  $start = time();
  $total =
  $failed = 0;
  $jobs = db_select('job_schedule', NULL, array('fetch' => PDO::FETCH_ASSOC))
            ->fields('job_schedule')
            ->condition('scheduled', 0)
            ->condition('next', REQUEST_TIME, '<=')
            ->orderBy('next', 'ASC')
            ->range(0, 200)
            ->execute();
  foreach ($jobs as $job) {
    try {
      JobScheduler::get($job['name'])->dispatch($job);
    }
    catch (Exception $e) {
      watchdog('job_scheduler', $e->getMessage(), array(), WATCHDOG_ERROR);
      $failed++;
      // Drop jobs that have caused exceptions
      JobScheduler::get($job['name'])->remove($job);
    }
    $total++;
    if (time() > ($start + 30)) {
      break;
    }
  }

  // Leave a note on how much time we spent processing.
  watchdog('job_scheduler', 'Finished processing scheduled jobs (!time s, !total total, !failed failed).', array('!time' => format_interval(time() - $start), '!total' => $total, '!failed' => $failed));
}
captainpants’s picture

captainpants CreditAttribution: captainpants commented

Unfortunately modules being enabled/disabled is not the cause of this given that this behavior persists even though it has been roughly a month since a module has been enabled/disabled and that I'm running cron every 5 minutes.

dineshw’s picture

dineshw CreditAttribution: dineshw as a volunteer and at TATA Consultancy Services for Pfizer, Inc. commented

Hello All,
I feel there is something wrong with in which sequence rebuild scheduler is handled.

I can confirm, for each cron run below code is getting executed and calling job_scheduler_rebuild_scheduler() which in turns sets/updates the value for job_schedule for next column and last column.

if (variable_get('job_scheduler_rebuild_all', FALSE)) {
    foreach (job_scheduler_info() as $name => $info) {
      job_scheduler_rebuild_scheduler($name, $info);
    }
    variable_set('job_scheduler_rebuild_all', FALSE);
    return;
  }

That results into below bold condition will never gets satisfied and for each cron run job_scheduler will kind of keep refreshing value of next column.
Ideally that should happened post dispatch. 

$jobs = db_select('job_schedule', NULL, array('fetch' => PDO::FETCH_ASSOC))
            ->fields('job_schedule')
            ->condition('scheduled', 0)
           <strong> ->condition('next', REQUEST_TIME, '<=')</strong>
            ->orderBy('next', 'ASC')
            ->range(0, 200)
            ->execute();

This is the scenario which I observed. Thoughts please, Its necessary to address this issue as it affects other modules features. Eg: Feeds Job Scheduling

G42’s picture

G42 CreditAttribution: G42 commented

variable_get/set is now deprecated, please update the D8 version accordingly as variable_get will cause a fatal crash.

http://drupal.stackexchange.com/questions/186156/drupal-8-variable-get

In Drupal 7

$data = variable_get('test_content_types');

In Drupal 8

$data = \Drupal::state()->get('test_content_types');

For more information about about "get", "set", "delete" visit - Step 5: How to upgrade D7 variables to D8's state system.

TorreyToomajanian’s picture

TorreyToomajanian CreditAttribution: TorreyToomajanian as a volunteer commented
FileSize
2034973-scheduler-does-not-sometimes-run.patch1.56 KB

I too encountered this error and figured I would spin up the appropriate patch since no one seemed to have gotten to it. It replaces all calls to the now deprecated variable_get() with \Drupal::state()->get() AND all instances of variable_set() with \Drupal::state()->set().

intrafusion’s picture

intrafusion
Primary language English
Location Edinburgh, UK
CreditAttribution: intrafusion commented
Status: Active » Postponed (maintainer needs more info)

Is this a D7 or D8 issue? The version of this is D7 but you're posting D8 code?