I'm currently under active development of a massive migration module. I had some problems with DataSync, in debug mode at least, when I run jobs which break throwing exceptions, DataSync run script does not mark jobs as failed or finished.

For the sake of stability, DataSync should be able to catch any error that user did not handled and definitily mark the job as finished or failed as soon as an error goes through.

Further, is there a method called in case job goes wrong? I need to close some things and send some network signals on fail.

Comments

andrewlevine’s picture

Title: DataSync cli script should catch all exceptions and mark the job as finished » DataSync should catch exceptions and mark the job as failed
Category: bug » feature

This is a great feature request, thanks. I agree that DataSync should handle uncaught exceptions.

For the moment, what you can do is catch exceptions yourself and then return FALSE in your "phase function". DataSync will then call the job's fail_phase() method which of course you can override to do whatever you want.

dataync_scheduler module also has timeout functionality so if a job has been "running" for too long it will automatically call fail_phase() on it.

Please let me know if you think catching exceptions and running fail_phase() on the job would fulfill your request.

andrewlevine’s picture

StatusFileSize
new929 bytes

actually, let me know how this patch works for you.

andrewlevine’s picture

StatusFileSize
new1.01 KB

oops, this one.

pounard’s picture

I did use you patch, bahavior seems to be better, because it did run fail_phase this time, but, when I went to see jobs status in the admin page, my fail job was still "Queued".

EDIT: then I re-ran it, and my job went from "Queued" to "Running". I re-ran it many times to be sure, the job is still "Running".

After looking up a bit, I found that fail_phase() method must return TRUE, I think in case of error, even if jobs does not returns TRUE from fail_phase() method, you should abort it.

Else, the patch is fine, thanks, you should push it to HEAD.

andrewlevine’s picture

StatusFileSize
new4.07 KB

Pierre,

The reason the job restarts is because the default (and currently only) implementation of fail_phase restarts the current phase and queues it for re-run. It was always my intention to have 3 default implementations of fail_phase that you could choose from. Please test this patch with your application. It sounds like you will want to choose the "fail_job" method by overriding fail_phase in your job-type class and having something like:

function fail_phase($method = NULL) {
  return parent::fail_phase('fail_job');
}

Let me know how that works out for you.

pounard’s picture

Oh ok, this is a behavior I can override, I should have searched more!
I'll test it tomorow, right now I don't have my dev box with me.

andrewlevine’s picture

Great. let me know how this works and hopefully we can release another alpha today, unless you have any other issues you've run into.

pounard’s picture

Sorry for taking such time to answer.
Your patch works well on my devel environment, until now, commit it!

I did override the fail_phase to do job_fail in case of error, and it worked well to, thanks for the tip.

pounard’s picture

DataSync does not seems to store exceptions it catched in watchdog, it should (or this is a bug on my environment, as soon as I finished PostgreSQL support, I'll look at)!
This because as custom phase did failed, any watchdog message will disappear at rollback.

andrewlevine’s picture

committed. thanks for testing

andrewlevine’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.