Issue description

node_save() calls db_transaction() in it's first line. Since a transaction is commited when it's object goes out of scope the node will actually be saved at very end of node_save().

We invoke hook_node_insert() and hook_node_update() before the end of node_save() which means, that node has not been saved to DB at that moment. On the other hand documentation for this two hooks states:

This hook is invoked from node_save() after the node is updated in the node table in the database, ....

and

This hook is invoked from node_save() after the node is inserted into the node table in the database, ...

This makes you think, that you already have all data safely saved, when Drupal calls your hook implementations. Assuming this lead me to very strange situations, that were really hard to debug.

Same thing applies to user_save() and could also apply to some other similar function.

EDIT: I've also written a blog post with a bit more background to this: http://janezurevc.name/strange-behaviour-race-condition-during-nodesave

Proposal/solution

I propose that we either change place where this hooks are invoked or update docs to make people aware of this fact. I'd think about exposing another hook at the time after a node has been physically saved into DB even if we decide to only update docs. People might need something like this.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

slashrsm’s picture

slashrsm’s picture

Component: node.module » documentation
Priority: Normal » Major
Status: Active » Needs review

I propose to something like this to hook_node_insert() and hook_node_update() api docs:

Note that at the time, when this hook is invoked, no changes were actually written to the database. This will be done when save operation is entirely completed and node_save() goes out of scope. You should not rely on data in the database at this time as it may be out of date. You should also note, that any write/update database queries, that will be run from this hook, will also not be committed immediately. Check node_save() and db_transaction() for more info.

I will mark this as Major. Not being aware of this can result in really strange bugs and can be really frustrating for a developer.

tim.plunkett’s picture

Version: 7.x-dev » 8.x-dev
Priority: Major » Normal
Issue tags: +Needs backport to D7

Moving to D8 first.

catch’s picture

Priority: Normal » Major

I actually think this is major, if only to add the documentation.

slashrsm’s picture

There might be other places that would need similar notice. hook_user_insert() and hook_user_update() are definitely two candidates.

I can prepare patch, but I'd like to have consensus about the actual text before that.

fago’s picture

hm, this also relates to #1729812: Separate storage operations from reactions.

Given #1729812: Separate storage operations from reactions I think the transaction should be completed *before* the hooks are invoked, thus making the hooks for reactions only. We can't do that for d7, but for d8 we could. I guess this would deserve its own issue though and require field-storage to be moved to the entity controller (and separately re-usable) first.

jhodgdon’s picture

Status: Needs review » Active

There is no patch on this issue, so reverting status to "active".

slashrsm’s picture

I agree with #6. So should we improve docs for D7 here and do D8 changes in #1729812: Separate storage operations from reactions?

jhodgdon’s picture

At the moment, the process would be to fix the D8 docs here, then backport the changes to D7. That other issue may or may not be resolved any time soon.

slashrsm’s picture

Category: bug » task
Status: Active » Needs review
FileSize
3.31 KB

Here comes the patch....

Status: Needs review » Needs work

The last submitted patch, improve_api_docs_1677830_10.patch, failed testing.

slashrsm’s picture

Stupid me....

slashrsm’s picture

Status: Needs work » Needs review
jhodgdon’s picture

Status: Needs review » Needs work

Thanks, looking pretty good!

There shouldn't be a comma here though (the first one):

+ * Note that at the time, when this hook is invoked, no changes were actually
+ * written to the database. This will be done when save operation is entirely

Actually, I think it needs a little rewriting to make more sense grammatically. How about something more like:

Note that when this hook is invoked, the changes will not yet be written to the database, because a database transaction is in progress. The transaction is not finalized until...

Also, this section has some extra commas and needs rewriting:

* the database at this time as it may be out of date. You should also note,
+ * that any write/update database queries, that will be run from this hook,
+ * will also not be committed immediately. Check node_save() and
slashrsm’s picture

Status: Needs work » Needs review
FileSize
3.36 KB

Improved patch. I am wondering if there is any other hook, that also needs this notice?

jhodgdon’s picture

Status: Needs review » Needs work

Much better, thanks! There are still a couple of small comma/article problems to fix:

+ * Note that when this hook is invoked, the changes will not yet be written to
+ * the database, because a database transaction is in progress. The transaction is
+ * not finalized until save operation is entirely completed and node_save() goes
+ * out of scope. You should not rely on data in the database at this time as
+ * it may be out of date. You should also note that any write/update database
+ * queries, executed from this hook, will also not be committed immediately.
+ * Check node_save() and db_transaction() for more info.

- 3rd line: "...until *the* save operation..."
- 5th line: I don't think the information is really "out of date", it's just not updated yet?
- 6th line: Remove both commas

Also... The previous paragraph to this text that is added reads:

This hook is invoked from node_save() after the node is updated in the node table in the database, after the type-specific hook_update() is invoked, and after field_attach_update() is called.

So this second paragraph contradicts the first one and the first one is really incorrect, right? Something needs to be done about that... maybe integrate them together?

slashrsm’s picture

- 3rd line: "...until *the* save operation..."
- 5th line: I don't think the information is really "out of date", it's just not updated yet?
- 6th line: Remove both commas

Fixed.

So this second paragraph contradicts the first one and the first one is really incorrect, right? Something needs to be done about that... maybe integrate them together?

How about this? I do not really like my current proposal, but it was the best I was able to produce right now...

slashrsm’s picture

Status: Needs work » Needs review
jhodgdon’s picture

Status: Needs review » Needs work

I think the new wording is OK. The only thing I think needs to be fixed now is that the verb tenses documentation are inconsistent, which makes it a bit confusing (it switches between past, present, and future). And there is one missing "the" in the new text: "... that will insert *the* node into...". Other that that, I think it's OK.

slashrsm’s picture

Status: Needs work » Needs review
FileSize
4.08 KB

Better?

jhodgdon’s picture

Status: Needs review » Needs work

Better, thanks! There are still a few small errors that need to be fixed:

a) The word wrapping on this line is too long:

+ * insert the node into the node table is scheduled for execution, after the type-specific

I think there are a couple others that are longer than 80 characters too.

b) Grammar problem:

+ * Note that when this hook is invoked, the changes has not yet been written to

changes *have* not yet... ... This grammar needs to be corrected in several places in the patch.

slashrsm’s picture

Status: Needs work » Needs review
FileSize
4.09 KB

Comments from #21 fixed.

jhodgdon’s picture

Assigned: Unassigned » jhodgdon
Status: Needs review » Reviewed & tested by the community

Thanks! Assuming the test bot shows green (which it should), I'll get this committed shortly (patch will most likely work for both 8.x and 7.x).

slashrsm’s picture

That's great. Thank you for your time and effort!

jhodgdon’s picture

Status: Reviewed & tested by the community » Fixed

Committed to 8.x and 7.x. Thanks again for *your* time and effort!

jhodgdon’s picture

Assigned: jhodgdon » Unassigned

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Added link to blog post.