Allow upgrading core with the update manager

Sounds pretty critical, that by default plugin manager can't upgrade its biggest use case?

Right, sorry. Jacob and I started a conversation about this in IRC but neither finished, nor wrote up the partial results here. Here are some of the main tricky issues:

A) authorize.php runs in a reduced bootstrap, so broken contrib can't harm it, but it's still bootstrapping and running parts of core: (Form API, Batch API, system.module, user.module, maintenance theme system, language system, plus the FileTransfer and Updater classes). Updating those things via authorize.php while it's running is slightly scary and hard to get right.

B) We can't necessarily assume there's another directory higher up in the tree than the webroot, so we can't exactly move the current core out of the way, extract the new one in place, make sure it's working, and then switch over. Basically, recovery from a botched upgrade and rolling back is hard -- this is probably going to have to be destructive (simply overwrite your existing version with new files). If we auto-generate a backup of your existing files (our Archiver classes would need to support create, not just extract), and we write that to your webroot somewhere, we'd have to write it to your private/files folder to avoid the security risk of someone being able to download the thing, extracting your settings.php file and getting your DB credentials (not to mention all your files).

C) Related to (B), but slightly different -- although you're not supposed to hack core, people sometimes do. It's not clear how to detect these cases and what to do about it when we hit a locally modified file.

D) Since we can't just move your sites folder out of the way and then destroy everything else, it's not obvious what to do when core removes files between versions. It's easy to go through the extracted new version of core and move all those files into place, but what about the files that aren't in the extracted copy that were part of an earlier version of core? A few options we discussed:

1. Core starts shipping with a manifest.txt file of some kind, listing every file included in the archive. This might be useful for #509010: Build a Package signing system for d.o. so that people can securely and simply download and verify packages anyway (since the manifest.txt can contain file checksums, and could be digitally signed). However, we don't yet have any infrastructure on d.o for such a file -- we'd have to change the packaging script to generate it, figure out where/how to present that info for each core release outside of the tarball (e.g. if someone wanted a copy and they checked out from CVS), etc. Only once that file exists can we put in the logic to use it, which would involve comparing the new manifest.txt we downloaded to the one in the current installation, and any files listed in the old manifest that are missing in the new manifest would be removed.
2. When upgrading core, instead of using a whole new tarball release file, download patches and just apply them. This would require implementing patch in PHP (apparently something in PEAR exists that does that, I haven't looked), and again, lots of changes to d.o infrastructure. We'd have to auto-generate a patch file for every official release, attach the patches to the release nodes, include data about the patches in the release history XML files. If your site is missing a few releases, it'd have to know to download the patch for each release you're missing and apply all of them, etc.
3. Ignore the problem entirely. ;) Upgrading core from 7.x to 8.x is completely outside the scope of this tool. So, if we're only talking about going from 7.x to 7.x+n, I don't think core is ever going to remove files in a point release along a stable branch. Sure, obviously, tons of files are added and removed between M.x and N.x, but never (?) between X.m and X.n.

E) It makes solving #606190: Fix handling of database schema updates in update manager workflow even more urgent. Even though schema updates from point releases of core are rare along a stable series, they can happen, and it's possible (though unlikely) that a schema change which changes some of the lower-level bootstrapping code might break authorize.php -- we update the code, but the schema hasn't changed, so now the code trying to generate the landing page to report about your update is broken with a fatal SQL error in one of the queries. :( We'd have to study how update.php handles this problem a little more closely, and then basically replicate that in authorize.php.

...

There are probably a lot of other smaller details, but this should already give people enough to chew on...

FYI: #22336: Move all core Drupal files under a /core folder to improve usability and upgrades would simplify things quite a bit in here...

Run for your lives!

Okay, getting very close to a prototype here.

One problem we've got to solve though:

In update.manager.php around :425

This is kinda hacky to begin with, but when you DL drupal core, it unpacks into drupal-7.x-dev or whatever so it doesn't match up, unlike modules and themes. I'm thinking we should probably refactor this so that we are just grabbing the first directory out of our tarball and calling that the project directory. We then have to make sure we can track this throughout the project so we don't have to guess it based on the name of the thing we are updating later.

Hey Derek,

Sorry I'm getting this so late. I've got a million kids running around the house, work is just not going to happen.

Here is a patch I sent to someone who was having issues with copy-contents vs copy directory. The upgrade core patch also addresses this, so I thought it might solve his problem. I haven't messed with this at all in over 2 weeks, so it might not even apply, but much of the useful bits will.

Last time I ran it, it worked, although warning: you may blitz your install.

Best,
Jacob

me too

The last submitted patch, upgrade_core-606592-42.patch, failed testing.

After talking to webchick moving this to Drupal 8, simply to much work left to be done

1. I don't think this needs to wait until http://drupal.org/node/22336 is fixed. It's just an excuse, and this could be fixed w/out that.
2. Subscribe
3. I think simply extracting Archive.tar.gz/drupal-7.0 -> DRUPAL_ROOT would be perfectly fine.
4. I think we should make sure all files are is_writable before extracting the archive.
5. I'm unassinging this, as I don't think Jacob Singh is going to do any work on this issue.

But that's just my opinion.

Subscribing.

Uploading a partial patch for this.

Excellent, thanks for getting this patch started!

A) There are some code style bugs, especially in the CoreUpdater class.

B) There's no sense keeping all the "manual" updates plumbing if there's no longer anything that needs manual updating. ;)

C) This shouldn't be here:

Also, this patch doesn't address any of the points I raised in comment #4...

Anyway, this is a totally legit start, but "needs work" is definitely the appropriate status here.

Thanks!
-Derek

+ $archive_errors = array(); //update_manager_archive_verify($project, $local_cache, $extract_directory);

Yes, that was kind of a hack (update_manager_archive_verify() was saying that it didn't have a .info file in it).

I'll try to work on this more today.

Sub

In fact, we could use get_included_files();, then copy those files from the archive first.

you can leave out these blank lines, they are not needed.

There are a couple of them in the code

Powered by Dreditor.