We have a lot of update functions in core that have been removed partially during the D7 cycle (and are now just stubs) or removed completely (so that the numbering is off).

This does have a real effect on site administrators: When they are updating their site and open the fieldset to see the list of updates, it will be confusing to have incorrect or out-of-date updates listed there, or to see updates with incorrect, out-of-sequence numbers (it will make it look like they did something wrong and don't have a complete set of updates to run). For more explanation, see the first part of my comment at #898520-58: Clean-up the upgrade path: comment and the subsequent discussion.

I think we should fix this throughout core, as has already been done for individual modules in some other issues (e.g. #898536: Clean-up the upgrade path: dblog).

Comments

David_Rothstein’s picture

Priority: Normal » Critical
Issue tags: +D7 upgrade path

This is not critical in the traditional sense, but it is critical based on the same logic that led people to mark #898558: Clean-up the upgrade path: node as critical: If we don't renumber these functions now (before the D7 HEAD-to-HEAD upgrade path is officially supported), we won't be able to do it all.

David_Rothstein’s picture

Status: Active » Needs review
StatusFileSize
new41.43 KB

Here's a patch. Kind of mindless, but it seems to work :)

Notes:

  1. Although I removed most stub functions, I deliberately left a few behind - these are ones that had comments like "Update moved to update_fix_d7_requirements". The reason is that things which happen in update_fix_d7_requirements() never get documented for the administrator anywhere else, so we might as well leave them behind to document it. Perhaps for consistency we should kill those too though; there are a lot of things that happen in update_fix_d7_requirements() that we already don't tell people about in the detailed upgrade notes.
  2. This patch necessarily renames the system_schema_cache_7054() helper function to system_schema_cache_7040(), since the update function it is associated with has also been renumbered. This may constitute a (tiny) API change, but I think it's worth doing to keep the numbering intact.
  3. I left behind openid_update_6000(), although I'm not sure why it's in the codebase - aren't we only supporting updates for sites that are running the latest D6? However, it seems to have been added recently in #886982: Incomplete verification of assertions and is related to a security issue, so I didn't remove it for now.
  4. I deliberately left the comment.module updates alone here, since that is soon to be taken care of by #898520: Clean-up the upgrade path: comment anyway.
David_Rothstein’s picture

The HEAD to HEAD module already needs an issue as a result of previous rounds of update function renumbering that were committed to core. It will need it even more if this patch is committed :)

The existing issue for the HEAD to HEAD module is here: #909338: Fix the schema versions in the {system} table for dblog and comment update function renumbering (and now locale too!)

moshe weitzman’s picture

StatusFileSize
new1.47 KB

I'm OK with leaving stubs behind with PHPDoc about what they were. I think thats helpful for folks that track HEAD which some sites actually do and we don't want to discourage them. They are the brave canaries.

So, here is a patch which simply omits empty functions. This just required a fancy array_slice() using the reflelction api (which we used here already).

My patch does the plumbing work. Not sure if we need to edit any update functions as well.

moshe weitzman’s picture

StatusFileSize
new1.67 KB

This one actually works.

moshe weitzman’s picture

StatusFileSize
new1.48 KB

sigh

David_Rothstein’s picture

Interesting. Although if you look at my patch, I'm not sure I removed anything that wasn't just pure cruft at this point - i.e., probably long past the point where it was useful to anyone. And it's consistent with some of the cleanups that have gone in elsewhere, for individual modules.

Also, if we don't do something like this then we'll have weird gaps in the update function numbering that makes it look like core developers don't know how to count :)

Perhaps it would make sense to combine the two approaches, though.

moshe weitzman’s picture

Yeah, probably a combination of both.

Going forward, I'm fine with gaps that look like "core developers don't know how to count". Thats really a symptom of we don't know what we are doing.

chx’s picture

It's completely fine if the list just contains arbitrary numbers IMO. Noone cares :)

catch’s picture

Yeah I don't see a problem with that either.

moshe weitzman’s picture

StatusFileSize
new3.18 KB

David's patch no longer applies cleanly. I went ahead and combined my patch with a cleanup of existing stub functions to match our proposed convention of keeping them around with Doxygen that explains where they went.

moshe weitzman’s picture

StatusFileSize
new4.62 KB

Woops. Still tripping up with git sometimes.

chx’s picture

Instead of rereading the file again and again what about a static $stub = TRUE and use getStaticVariables of the already ran Reflection?

moshe weitzman’s picture

StatusFileSize
new4.94 KB

Here is a patch which only calls file() when the .install file changes. I added some comments as well. I spoke with chx on IRC and he is OK with this approach.

chx’s picture

Status: Needs review » Reviewed & tested by the community

Yes this is a nice patch.

David_Rothstein’s picture

Priority: Critical » Normal
 /**
  * Adds 'delivery_callback' field to the {menu_router} table to allow a custom
  * function to be used for final page rendering and sending to browser.
+ *
+ * Moved to update_fix_d7_requirements().
  */
 function system_update_7041() {
-  // Moved to update_fix_d7_requirements().
 }

Hm, this looks a little like magic, though... Dumb question: Is there a way to check if the function body is empty of code (regardless of whether or not it has comments in it)? Seems like that's ideally what we want to check.

Also, if the consensus is to not bother renumbering the functions, then I think this isn't critical anymore (since it can be done at any time).

chx’s picture

Priority: Normal » Critical

David, after tonight you think you can demote criticals? Especially one that's ready despite you two doing everything to kill the morale of anyone trying to fix criticals? Get real.

And no, there is no way for that short of tokenizing the files which is too slow.

dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD. Thanks.

David_Rothstein’s picture

Status: Fixed » Needs work

Hm, I realized that this actually introduces a worse bug than it fixes, because it introduces a mismatch in the reported number of updates to be run (which everyone sees, not just the 2% of people who open up the fieldset to look at the details).

For example, updating from Drupal 6 to Drupal 7 it tells me "118 pending updates" before I run them, but then the progress bar shows 130 updates once they actually start running - the difference being due to the stub functions. (In an edge case, this could lead to even more confusion - if somehow you wind up with a site where the stub updates are the only ones that need to run, then your status report page will keep telling you your database is out-of-date but when you go to run updates it will tell you that you don't have any.)

I think there is no simple way to fix that, so probably we should roll this back and try something else. I think it's simplest if we just delete the obsolete functions. I don't think there's anywhere else in the code where we deliberately keep old, useless functions around, is there? So no need to do it here. And they're still available via CVS, of course, for developers who want to see the history.

Does that sound right? I'm not proposing renumbering the update functions anymore as I did originally (as it seems like most people think that's too much code churn now), just removing the 10 or so that are empty, as the simplest way to make them not appear anymore.

***

By the way, not that it matters much with the above plan, but I don't understand why tokenizing the file would have been necessary to find out if the function body only consists of code comments? (The code already defines $body as an array containing all lines of code in the function, so it seems like we could just look at the individual lines in that array and see which ones began with a comment marker, etc.)

damien tournoud’s picture

Huh. That makes very little sense to me.

This is our LAST CHANCE to renumber and clean up the installation process, as we should have done *long ago*.

Let's revert this. There is no reason to add more complexity to the upgrade process, while we can (and should) just renumber the functions.

mikejoconnor’s picture

Assigned: Unassigned » mikejoconnor
mikejoconnor’s picture

Assigned: mikejoconnor » Unassigned
Status: Needs work » Needs review
StatusFileSize
new23.09 KB

I reversed the patch in comment #14, removed the stub functions, and renumbered. Also updated the update_dependencies to match the new function names. It would be great if someone could verify that everything matches.

Status: Needs review » Needs work

The last submitted patch, no-stubs.patch, failed testing.

moshe weitzman’s picture

I'm fine with revert if that’s what folks want.

mikejoconnor’s picture

Status: Needs work » Needs review
StatusFileSize
new24.95 KB

Apparently I missed a dependency, lets try this.

David_Rothstein’s picture

@mikejoconnor, thanks, but your patch file looks much smaller than my patch from comment #2. Did you deliberately leave parts of it out?

***

I'm still theoretically in favor of doing a full renumbering, but most other people above sounded opposed. Maybe we should come to consensus before writing more code. The choices are:

a. Revert #14 and just remove the stub functions (but leave the numbering incorrect).
b. Revert #14, remove the stub functions, and renumber the remaining ones.

Note that to do (b) properly, we also need some changes along the lines of those in my original patch such as these:

-    $schema['cache_path'] = system_schema_cache_7054();
+    $schema['cache_path'] = system_schema_cache_7040();

which might be considered a minor pseudo-API change at this point.

flevour’s picture

How can we help on this issue? It seems to be stuck waiting opinionated comments from core developers on a/b choices in #26.

catch’s picture

I like (a). I really don't see what the point of renumbering the functions is, it's purely cosmetic.

damien tournoud’s picture

For the sake of getting a beta out, I can settle on just nuking the stub functions without renumbering.

chx’s picture

StatusFileSize
new6.09 KB

Let's do that then.

dries’s picture

Status: Needs review » Fixed

Yes, let's do (a). Committed to CVS HEAD. Thanks.

Status: Fixed » Closed (fixed)
Issue tags: -D7 upgrade path

Automatically closed -- issue fixed for 2 weeks with no activity.