drush up drupal deletes .svn/.bzr/vcs files within core directories

@quicksketch: Try this for me. It is improved, but I didn't check in my drupal dir to svn for a "real" test. Does drush up somemodulename preserve your .svn files? If you're not using a vcs, then drush up will remove all files in the directory. Prior to this patch, drush up drupal removed all dot files in the drupal root, but should have preserved patch files.

Sub.

@quicksketch: I was more asking about how it was behaving on your system, since files that aren't checked in will cause the whole shebang to screech to a halt before you ever get to the point where drush preserves or loses your files. But I infer now that my 'additional files' you mean 'checked into svn but not part of the drupal release'. I was somehow thinking of files extraneous to both systems.

In any event, here is a new patch that works (*cough*) cleanly.

Okay, I was hoping that no one would notice that cough. Darn. A bit of an explanation.

The way that update core works right now is it moves all of the core files away to a new directory that 'looks' a lot like a project directory. It then uses the regular updatecode routines to do the update, and moves everything back again when it's done. Now, without going into details, the 'sites' folder can't come along; it doesn't work -- and in fact this is the main reason we move everything else away; I could probably also get away with keeping everything else in place, and just move the 'sites' folder somewhere else, but no matter, the end effect is the same.

The missing 'sites' folder causes problems for svn. 'svn status' emits a "! missing" line for the sites folder, and 'svn status -u' is even worse; it emits a "* not up to date" line for everything inside the 'sites' folder. [Edit: The rest of this is wrong -->] There is no flag to pass to svn status to tell it to skip just one file. So I hack it. I run 'svn status' as usual, but then use preg_match to get rid of the one "! missing sites" line. The 'svn status -u' is much too noisy for this sort of tomfoolery, though, so I skip the up-to-date check altogether when doing a core update.[End wrong]

Note that this would also be an issue for bzr, save for the fact that bzr does not do the same checks that svn does at the moment.

Edit: Do not use this patch.

I missed an obvious solution to the update core with svn problem last night. svn status can optionally take a list of files to get status for; so, although there is no 'skip' option, an item can easily be skipped simply by passing in status a list of every item in the folder except the item to skip.

I'll post a cleaner patch shortly.

I kind of worry that we are drifting into lots of special cases for Drupal core, when really we should be able to use the same approach to managing excludes or deltas (either specifying manually, or determining automatically) to deal with modules or themes with local changes - which is an identical problem. The approach of moving the current core out of the way and then trying to merge changes back also seems to be introducing quite a lot of complexity - perhaps we should reconsider the approach of dl-ing the project into a temporary directory and then copying files across from there (we could even do a simple rsync to start).

Either way, this is clearly a pretty major issue right now, so it is fine to commit this (or a simpler band aid, if possible), but I think we should aim for a round of refactoring before we do a stable release, if not to get core vs. contrib more consistent then at least to move somewhat in that direction.

Yes, I agree with the sentiments. This implementation should definitely be re-factored and fixed up, as strategy used by pm-updatecode is not good for updating code. Your suggestions above are better vis-a-vis a long-term solution. However, I did it this way so that the existing, stable strategy used for module updates could also be applied to a core update, so that we could get to a point where we can update core faster, without needing all new code. So, while I agree that this implementation is ugly, it is also based on code that has gone through a lot of testing, and although it needs to wiggle a little to do it, a core update is done in very much the same way that a module update has always been done. I think that this technique is good enough to go into 3.0-stable, and that we should work on refactoring and rewriting it after DrupalCon. It's been six months since the last stable drush release, and I think a rewrite of pm-updatecode is likely to take another three to six months from start-to-stable.

It's up to Moshe to decide, of course; however, I think we should do 3.0-stable before DrupalCon, so if update-core is not ready for inclusion in a stable release, then it should be backed out.

Okay, clean repository, checked in Drupal 6.15, ran drush up drupal:

Looks like it did exactly what it should. It did seem to take quite a bit longer than my previous attempts before the patch however. I'd say it takes about 40 seconds for the whole process, previously it took about 5 seconds. Maybe the download from d.o was just slow though.

I agree with the sentiments here though. While this works I'm not sure it's the right thing to be doing. I doubt this would work if you had .bzr or .git files in the root of your Drupal installation, requiring yet more exceptions to factor in.

committed #8. lets discuss how to proceed in #759906: Update core without moving core files out of the way..

Automatically closed -- issue fixed for 2 weeks with no activity.