Clean-up the upgrade path: UPGRADE.txt

Mark #291495: UPGRADE.txt needs some rephrasing as a duplicate.

I think it would also be useful to distinguish clearly between minor and major upgrades, and to document them separately. Here I am considering a minor upgrade to be one from, for example, 6.5->6.6, and a major one to be 6.x->7.x. Specific areas that could be clarified in this way include:

1) is it really necessary to disable all contributed modules for a minor upgrade?

2) if you are only upgrading Drupal core, is it necessary to rerun update.php to handle custom tables and contributed modules, as is suggested by step 12 in 6.x UPGRADE.txt?

If people are supposed to upgrade promptly (and often), doing so needs to be as easy as possible. Looking at UPGRADE.txt makes it look like a much bigger production that it probably needs to be for minor upgrades. If the two types were identified as such, then the release notes for a given upgrade could indicate it that upgrade was minor or major and point to the relevant section in UPGRADE.txt or UPGRADE-minor.txt/UPGRADE-major.txt, depending on how the two types were organized.

1) is it really necessary to disable all contributed modules for a minor upgrade?

No, although very occasionally we issue security updates which by necessity break APIs, so there's occasions where you might want to do that.

2) if you are only upgrading Drupal core, is it necessary to rerun update.php to handle custom tables and contributed modules, as is suggested by step 12 in 6.x UPGRADE.txt?

No, I also think this is out of date since we run upgrades for disabled modules now too.

@5: AFAIK minor upgrades don't ever change the behaviour of the current UI, except to fix a problem. 403/404 pages first stopped displaying the sidebars in 5.0 - if that's what you were referring to.

But yes, distinguishing between major and minor upgrades per #4 might simplify things a bit (the main difference of course being that you generally don't have to worry about contrib modules/themes when doing a minor upgrade).

A thread related to this issue:

The current UPGRADE.txt lacks instructions for minor updates (6.x -> 6.y)

I received some email today from a nearly irate Drupal user who had this to say, slightly edited... This was for 6.x but the file is nearly the same and the same confusion applies. Just another illustration of why this file needs to be updated...

"The upgrade.txt file sent out with the 6.16 talks about checking your addon/custom modules and verifying their compatibility. While this is technically accurate, it seems to me that this is really for users that go from 5.x to 6.x. When we go sub-release like we did this month from 6.14 to 6.16 that's a bit more shady. When you try to check your modules per the instructions, its not clear anywhere how to do so and the upgrade document exacerbates the problem. I suspect this is even more important due to the soon to arrive 7.x release.

Further - the instructions say to log in and leave yourself logged in. This is not possible in some cases since you're removing the core files. But the instructions don't really clearly tell you what or how you're dealing with this and why."

The 14 steps of UPGRADE.txt seem clearly written for upgrades between major versions such as 5.x -> 6.y, or 6.x -> 7.y. The guidelines for updates between minor versions like 6.x -> 6.y, or 7.x -> 7.y, are surprisingly missing from UPGRADE.txt, but given for example by Dries' company Acquia, and people like jt_jones, VeryMisunderstood, etc.

From those still unofficial guidelines, five steps of UPGRADE.txt are unnecessary for updates between minor versions:

4. If using a custom or contributed theme, switch to a core theme such as Bartik or Garland.
5. Disable all custom and contributed modules.
11. Ensure that the versions of all custom and contributed modules match the new Drupal version to which you have updated.
12. Re-enable custom and contributed modules.
13. Return the site to its original theme.

The importance of clarifying the frequent minor update procedure can be seen in users' comments like this:

So you don't disable all modules? You are skipping a lot of steps in UPGRADE.txt, such as disabling all custom and contributed modules.

I upgraded 6.6 -> 6.8 recently, and this was the most painful part... especially since disabling all modules requires multiple passes due to dependency issues, and you need to keep track of exactly which modules you need to re-enable after (and my site uses a LOT of modules).

(From the thread: Is a different approach in upgrading possible?)

For example, the following is the useful clarification by VeryMisunderstood (VM now):

I also state that you don't have to disable any modules because you are only moving up a minor version. You only have to disable modules when you are moving major versions. ie Drupal 5.x to Drupal 6.x because drupal 5.x modules will not work with drupal 6.x thus disabling them is a best practice as they will have to be upgraded too.

Going from 6.8 -> 6.9 is as simple as
put site in offline mode
back up your DB and files
delete all files and folders except the sites folder
upload all 6.9 drupal files and folders from 6.9 except sites folder
run update.php

The only files you will need to replace in drupal, is your robots.txt and your .htaccess. You only have to replace them in the new install if you've customized them.

Another issue was recently opened about this same topic. I just marked it as a duplicate:
#672028: UPGRADE.txt is confusing on mandatory steps for minor releases

I'm taking a pass at rewriting the file.

Here's a patch.

I did have one question... catch mentioned above in #6 that upgrades are run for disabled modules. If this is true, then I have a question:

If you are upgrading from major to major version (6.x to 7.x), do you need to grab 7.x-compatible versions of your contrib and custom modules before you run update.php in step #10? I'm assuming here that all the modules have been disabled first, but I'm basically wondering if you need to get the 7.x-compatible files (step #11) before you run the update (step #10).

Anyway, assuming the answer is no, a review of this patch would be appreciated.

Tagging this and also escalating it to major to get some more attention. As it doesn't break anything critical wouldn't be proper, but this is an embarrassing issue that has been floating around all to long without getting the love it deserves, so major!

I had a look at the patch and although it's a good step in the right direction I think we should take the full step and make it 2 sections "HOW TO UPGRADE FROM ONE DRUPAL VERSION TO ANOTHER" and "HOW TO UPDATE DRUPAL FROM ONE MINOR VERSION TO ANOTHER MINOR VERSION" (not necessarily in tis wording). The division as such would make things clearer but even more important it will also allow for a clearer to understand language.

I will take a turn on it and see what I can come up with, so... Let's fix this once and for all!

I decided to change the title because when looking around I found several issues highliting or directly concerning UPGRADE.txt in some way and this is just dividing and unproductive. What it's all about is to have a clear, non confusing, and for anyone understandable instruction for how to upgrade Drupal from one major version to another, and from one minor version to another minor version, and basically nothing else. So one issue that solves this once and for all, until the actual upgrade/update path or procedure changes should be enough. I'm prepared to give it a try and first step must be to direct the flow to focus on this issue alone, then I know there are lots of Drupalists out there willing to assist.

I would also like to give an example of why I think the current patch needs work:

-UPGRADING
+HOW TO UPGRADE FROM ONE DRUPAL VERSION TO ANOTHER
...
+ compatibility (#11), if you are upgrading to a new major version (e.g. 6.x
+ to 7.x).
+ * If you are making a major version upgrade, you have first upgraded to the
...

It's my experience that when trying to explain something, too many words often tend to confuse instead, especially when you say the same thing that already been said but with other words.

Just as one of the possibilities, a simple -maybe temporary- fix could be to add a brief note like this or similar:

Note for minor updates: The steps 4, 5, 11, 12 and 13 -on switching to a core theme, and disabling modules- are unnecessary for updates between minor versions, such as 6.x->6.y, or 7.x->7.y.

@juan_g: While your input is welcommed I don't belive the path of temporary fixes is the way to go and probably (at least partly) is to blame for the confusion the current document often causes. A much better path imho is to separate the document in 2 sections to avoid this type of interchangeble instructions that jumps between the two scenarios. Possibly it should be divided in to 2 documents UPGRADE.txt and UPDATE.txt but I think that's of lesser importance right now.

I have started to write a patch to hopefully be up for review within 24h or so but I'm also trying to look around to get a better picture of what's confusing people, steps 8-9 for sure but also such a thing as step 3, shouldn't such thing as "Place the site in Offline mode..." be the very first step, before backing up database as otherwise there will be a "time gap" possibly resulting in your backup is outdated? As depending on the time it takes before reaching step 3 something important may have changed due to user actions.

The more input from drupalists the better but lets make this doc as clear as chrystal.

Also tagging this "Needs Documentation" in respect of an issue I closed as a duplicate.

I'm bumping this to critical, as it blocks #887288: Attempting to use update.php without having configured a database in settings.php results in incorrect/misleading error message - since there are currently zero instructions on how to get your Drupal 6 database connection into Drupal 7, apart from the implicit suggestion that you'll copy the settings.php file over (which is usually wrong because you've then just reverted all the changes to that file made since D7 opened up for development).

Yes, separating into two sections is necessary. I think that by far is the best is if we ask people to start fresh with a new core version and only copy settings.php over from Drupal 6. For minor versions, overwrite the contents of your misc includes modules themes directory but not sites.

I don't even like asking people to copy settings.php between D6 and D7 - here's a diff between the two - http://drupalcode.org/viewvc/drupal/drupal/sites/default/default.setting...

There are new ini_set in there which will be lost if the D6 file is copied, and a load of new variable documentation which I guess can still be found in default.settings.php but isn't really ideal.

Just having my morning coffee and have pondered on this a bit overnight and here are some ideas:

1. Definately 2 sections and at least for now keeping them in the same file as it makes it easier to keep language in sync.

2. At the beginning, make it clear that the document assume "Don't hack core" is respected.

3. Regarding Backup, database for sure but when it comes to Drupal, why take a backup of your old core files? Assuming 2 is followed the backup already exists on d.o. and I think the current wording here basically origine from that "it always been like this". In fact, if you clean out everything that is core, except for sites (and custom added folders) you should basically be able just to expand the new tarball (thinking Linux) in your old Drupal root. The document could of course suggest taking a backup just because it's a good thing to do anyway but in fact it's optional as for practical/technical reasons. So basically taking a more intuitive approach here that actually is in harmony with how Drupal works.

4. Another thing I have reflected over and which also comes up in some of the issues and forum posts that find the current doc confusing, is the amount of irrelevant information. I think the doc should cut the crap of trying to explaining everything like why you need to be logged in as user ID 1 or have the "Administer software updates" permission as isn't that what we have the in depth documentation on d.o. for? At least I think this doc should be more of a clean path of what you have/need to do, possibly also including optional steps, to conduct a succesfull upgrade/update.

Also I agree to Critical although I didn't want to make that call but done properly this issue can probably spare d.o quite a few screams for help.

Feedback welcommed and I'm working on rewriting the doc as I speak and should have a first draft up within a few hours.

Can we assume that Drupal always is installed in the web root aka Document Root on Apache? INSTALL.txt at least doesn't give any alternative so assume it's safe to follow suit.

I agree that two sections are needed, but strongly disagree with splitting into two files (UPGRADE.TXT and UPDATE.TXT were suggested). I think I would have to read both anyway to work out which was which!

I suggest that the minor updates section should come first, because this is hopefully what will be done most often and should also be much shorter.

I sort of agree with yettn's point 4 above that some of the detail should be removed and handled by a reference to the online handbook, but I also think this document needs to be a fairly comprehensive and standalone. Many software installation guides now have a "quick start" section which gives a checklist of steps for the more experienced user and then goes into more detail in subsequent sections. I think this would be a good pattern to follow.

I also agree that this is critical. I suspect that, now the beta is out, lots of people will try and upgrade their D6 site to D7 - so the upgrade process will be their first real experience of D7. So documenting it as well as and straight forwardly as possible is important for the acceptance and uptake of D7.

In response to #26, no I don't think we can assume it will go in the document root. My D6 sites are all on shared hosting and all have gone in subdirectories off the docroot - e.g. <root>/Drupal6.

I would add a note that points out that this upgrade will not convert existing CCK fields to the new D7 fields. This has to be done separately. Since D7 now includes fields in core I suppose it can be concluded that they will automatically be updated too.

I have almost a draft done but need to step away from the computer for a few hours, so bare with me and I will submit it later tonight. There are also some ?? that need to be straiten out, like with the old D6 modules, are they still supposed to be there or not? Reading various issues gives mixed results but I don't have time to follow up on that now. I did various test upgrades before with and without modules with various result. However, I haven't tested with the latest dev where I think at least one of the problems I experienced have been fixed.

Anyhow, I will test ride my doc tonight as well.

@cnolle: I find your comment a bit contradictory as it stands now, please develop it further.

Ok I was able to successfully upgrade drupal-6.19 to latest drupal-7.x-dev following my document leaving old D6 sites directory in place, however I must say I'm not satisfied with the result at this stage if I compare with a clean install of drupal-7.x-dev but I'm too tired to elaborate on that now and it should be dealt with elsewere anyhow as it has nothing to do with the document.

It also have to wait with the patch until tomorrow as I'm too tired to complete it now so better done with a fresh head tomorrow.

Separate sections for minor and major version upgrades sounds great, even if there's a bit of repetition they're completely different processes and should be treated as such. Doing minor upgrades first makes sense too.

On backing up core files - if you do something stupid, it's much easier to copy a directory back than it is to go find the release of core on Drupal.org you were using prior to upgrade etc., so I think that's worth leaving in.

Looking forward to patch.

As yettyn pleased in #934948, I move the according UPGRADE.txt issue to this thread:

In trying to follow the instructions in UPGRADE.txt #6 - remove all "old files" - I first looked for any hint how old a file must be to be removed. No information found.
* Does it mean that all D6 files have to be removed? I first deleted all D6 files and then unpacked D7 (#7 and #8 plus changed ownership).
* In a second try I removed nothing but unpacked D7 over the existing files. Which files to remove now?

Regards

@schildi: This is still a draft but here is that point taken from the current state of my rewrite.

6. Remove all files and directories from the Drupal installation directory,
EXCEPT for the sites/ directory and any custom created directory and file outside of it. As an example,
if you still have a files/ directory in drupal root, orgine from an older
setup, leave it in place.

[Authors note:] Here is the question of old D6 contrib modules "to be, or not to be?"
in sites/all/modules and sites/all/themes, looking around reading various issues don't
give a clear answer. [/Authors note:]

As you can see the judge is still out on that step... I have it currently as leaving the old D6 files in place even if I know that 'may' be wrong. However, I did a successful upgrade with them in place and I also know patches have been done to D7 core to make it possible. The crusial thing is of course the database BUT leaving the old modules there can help you to see which modules you have to upgrade bringing your site back to how it was before.

I'm still working on refining the document to be as comprehencive and no nonsense as I think it should be, and almost there and the patch is coming. I don't expect it to be the final document though as one head only have 2 eyes and never sees enough ;-)

From reading through this thread, and from upgrading drupal sites in the past, it seems like people are looking for the following info, and that this has a place in the cleaned up docs:

1. how to get db settings over from old settings.php files
2. how to get any additional overrides from settings.php files into new settings.php doc
3. how to convert old cck fields into new core fields - does this happen automatically a part of the upgrade, or is there a separate process for this?
4. question about disabled modules raised by jhodgdon in #16 - I'm assuming that the best practice here would be to make sure that all disabled modules are also uninstalled *before* attempting the upgrade. While this is a general best practice that doesn't necessarily apply only to upgrades, if, as catch says in #6, "we run upgrades for disabled modules now too" people could get bitten because it is not intuitive that you should get code for a module that is disabled but not fully uninstalled
5. copy over any changes in the .htaccess file

For minor core upgrades, it would actually be useful to note (as part of the security announcement) if the upgrade contains a change to the settings.php or the .htaccess file - this is less something that is directly relevant to this specific issue, but a changed settings.php file requires some additional steps relative to an upgrade where settings.php is unchanged.

And finally, at the risk of splintering the docs even further, we really have three different upgrade scenarios, listed in order of complexity:

a. Upgrades between versions (ie, 6 to 7)
b. Core upgrades within a version (7.0 to 7.1)
c. Contrib upgrades within a version (module foo 7.x-1.1 to 7.x-1.2)

Arguably, there are even variations within module upgrades within a version, when the module changes from a 1.x to a 2.x branch. These changes can introduce additional dependencies, etc - however, imo, these issues do not need to be addressed in the upgrade.txt that ships with core, but in module documentation, but I'd rather make the choice to exclude detail in the interest of brevity than overlook something.

Is there a simple d6 install profile/site that can be used for testing and documenting the upgrade process to D7?

#34, point 3: At the moment the upgrade process creates node types from D6 CCK content types and moves the title and descriptions, but the D6 CCK fields are not moved and you end up with a load of "content_" tables in your upgraded database that are not used by D7.

As I understand it, this awaits the development of the D7 CCK module. The CCK project currently says

The D7 version of the contrib CCK package currently contains:

• D6 -> D7 data migration code (partially working)

So it looks like it will be a post-upgrade step. Once your D6 site is updated and running in D7, install the new CCK module and its update process will migrate your D6 CCK fields. But this question really needs an answer from the CCK maintainers.

I would certainly agree that the point should be covered by this document.

@ #35, #34 point 3: As this is something taking place after the upgrade and as CCK (at least partly) have moved in to core, couldn't this varant a document of its own distributed with core and UPGRADE.txt could just reference to it? That way it wouldn't delay the completion of this document.

I had to go out for a long walk as it might be one of the last sunny days for a while but is basically prood reading the doc now and will soon put it up.

As for #34 point 1, the upgrade process do update settings.php appending the db connect info in new format. It's not for this issue I guess but I don't understand why it doesn't extract that info, make a backup copy of settings.php, overwrite it with a copy of new default.settings.php and put the dd info in there instead. Anything specific upgrade process cannot or should not deal with (#34 point 2) can then be amended manually by the user afterwards and possibly also be reflected in this document.

RE: "As this is something taking place after the upgrade and as CCK (at least partly) have moved in to core, couldn't this varant a document of its own distributed with core and UPGRADE.txt could just reference to it?"

Yeah, I just spent some time digging into the CCK issue queue, and was about to say exactly that.

The reality is, though, migrating to D7 for most sites will involve upgrading CCK fields to d7 core fields, so both of these will need to get done.

+1 for the reference, and I'll probably start on the separate CCK upgrade docs (after checking that it isn't already in progress somewhere).

bonobo wrote:

4. question about disabled modules raised by jhodgdon in #16 - I'm assuming that the best practice here would be to make sure that all disabled modules are also uninstalled *before* attempting the upgrade. While this is a general best practice that doesn't necessarily apply only to upgrades, if, as catch says in #6, "we run upgrades for disabled modules now too" people could get bitten because it is not intuitive that you should get code for a module that is disabled but not fully uninstalled

No, for upgrades between major versions, like 6.x->7.y, modules are usually just disabled, not uninstalled, unless we want to remove all related tables from the database, which is not a frequent case.

For example, from the book "Using Drupal":

Uninstalling a module will delete all data associated with that module,
possibly including content on your website. Be very careful when using
this option, and be sure to back up your database first. Note that uninstalling
a module does not remove it from the filesystem; you still have
to do this manually.

@#38, I was thinking just the same but forgot to include it in my previous post. The question remains though if it's advicable to leave the old modules and themes there? Does it make any differance? It does have the benefit that D7 list the modules showing dependencies BUT it lists all modules in the modules directory and doesn't make any diffenciation about it being installed or not. So I think it would be advicable to remove all modules that not are/been installed, in fact I think I will add that to my draft.

Also the modules that has moved into core, I think we need to compile a list including in the document and they should probably be removed from sites/all/modules as well? cck, token and admin_role comes quickly to mind but I think there are more right?

btw, how do I add a patch here, just attach it as normal or? I plan to post both the patch and the full doc (as attachment) because it's easier to read upfront while the patch may be prefered by others.

RE #38

for upgrades between major versions, like 6.x->7.y, modules are usually just disabled, not uninstalled, unless we want to remove all related tables from the database, which is not a frequent case.

To clarify, I was *not* referring to modules that are actually in use. Obviously, we wouldn't want to uninstall those.

I was referring to cruft modules - modules that are installed, evaluated, and then disabled without being uninstalled. A version upgrade is a good time to remove that cruft.

From catch's comment in #6 (which is nearly 2 years old) and jhdgdon's unanswered question in #16, we should get this sorted out, as it has implications for the order in which the steps of the upgrade should be completed.

If, as catch says in #6, "we run upgrades for disabled modules now too" a site that does *not* remove cruft could be required to carry that cruft from D6 to D7. For all the obvious reasons, this makes removing cruft a good step to do *before* beginning the actual codebase upgrade.

Would be good to have a handbook page for the CCK upgrade, even if it's a stub for now, just so that it can be linked to.

Contrib upgrade in general, I'd be wary about putting anything in UPGRADE.txt since there are so many scenarios, but again a handbook page could be good to reference.

The issue with disabled modules is that if you want to ever enable them, you need to keep a copy of the module's files in your install so that updates will be run eventually, however I don't think this level of detail needs to be covered in UPGRADE.txt. edit: cross-posted with bonobo - the cruft removal aspect to this is possibly more important.

One more thing, can't believe I never spotted this until now:

 1.  Back up your Drupal database and site root directory. Be especially sure
     to back up your "sites" directory which contains your configuration file,
     added modules and themes, and your site's uploaded files. If other files
     have modifications, such as .htaccess or robots.txt, back those up as well.

... 
3.  3.  Place the site in "Offline" mode, to let the database updates run without
     interruption and avoid displaying errors to end users of the site. 

I haven't done a major Drupal version upgrade like this since I was on 4.6 and shared hosting, this is actually really, really bad advice.

Translation:

1. Take your site offline for an open-ended task.
2. Run something that has a high likelihood to destroy your database or leave it half-upgraded in an unrecoverable state if the slightest little thing goes wrong.
3. Rely on your backup recovery plan working when that inevitably happens.
Ouch ouch ouch ouch.

When doing a major version upgrade, these are roughly the steps I take, this counts for personal sites and paid work.

* Make a full copy of the live site including all files and database, and put this somewhere that's not web accessible.
* Restore the database backup to a /new/ database, let's call it example_com_7.
* Set up a Drupal 7 install for the upgrade, upgrade.example.com or upgrade.localhost
* Point the Drupal 7 settings.php to the example_com_7 database
* Test the upgrade on core.
* Add in contrib modules one or a few at a time, test on them.
* Go through any additional configuration steps and document them.
* Do the whole process again at least once to make sure the documentation you just wrote doesn't have massive holes in it (or convert it into hook_update_N() and/or a drush script).

When you're ready to do the final upgrade:
* Put the live site into maintenance mode to stop anyone posting new content, or use http://drupal.org/project/maintenance_helper so that the site can stay up, but with no content changes.
* Do a new database and files backup.
* Do the final upgrade one more time with the new backups, verify everything is OK, then move the Drupal 6 files out of the way, copy the Drupal 7 ones in their place.
* Take the site out of maintenance mode.
* Your live database is now example_com_7, the old one is there at example_com_6 ready to use with your files backup if you need to check something or roll back. After a week or month, you can just archive the backup you took and drop that db to free up space.

This allows you to do the major version upgrade with only a few minutes down time (or theoretically zero if you use maintenance helper and switch vhosts), and you keep your live site intact until the exact moment you switch to the upgraded one, and have a hot copy of it for afterwards.

Every shared host I've ever had has allowed me to have sites running in subdomains, and more than one database, so I don't see any reason we couldn't suggest something closer to this in UPGRADE.txt too.

Sorry more stuff...

We now require updating to the latest release of Drupal 6 before updating to Drupal 7, to avoid supporting 20 possible different upgrade paths #760014: Stop supporting direct updates from Drupal < 6.17, in practice this means whichever version Drupal 6 is out when Drupal 7 gets to RC, and upwards. This needs to be mentioned somewhere.

Often contrib modules only support upgrading from a particular version (CCK 5-6 did this iirc), we should possibly mention checking contrib module upgrade docs before trying to do core, if that ends up in a handbook page that's linked somewhere this is fine.

Created a separate ticket in the CCK issue queue for the CCK upgrade docs, and assigned it to me: #941694: Create a handbook page for upgrading from D6 CCK to D7.

If there are issues that will be useful/relevant, please add them there, and we can leave this thread for the core docs.

Figured it out how to add the patch by reading a bit further down ;-)

@ #40, was about to up the patch but will add the cruft part as well

@ #42, I have already covered for that.

About modules and a major upgrade 6.x->7.y, when we replace the core files, I think probably the safest procedure is to -at the same time- also replace all module files with the new D7 versions, as suggested in the online handbook, and then run update.php.

#39
please take also into account that D6 modules which are obsolete for some reason can't be disabled or uninstalled in D7. You have to set up some kind of plan which modules to switch off/replace BEFORE upgrading to D7.

EDIT:
And when you have about 60-90 modules installed you probably don't want to check manually for the D7 availability each time you do another try to upgrade your site. So it would be really helpful to
* to have a list of module to disable/uninstall prior to upgrade
* have an automated way to check for availability of D7 modules

@ #45, I will recall I read in some issue this approach causing errors but can't find it now. Anyhow, I haven't taken that approach in the document I now attach, both as patch

So here it is, both as patch and as a clean txt doc, probably in need of more work but this is at least a start.

RE:

And when you have about 60-90 modules installed you probably don't want to check manually for the D7 availability each time you do another try to upgrade your site. So it would be really helpful to
* to have a list of module to disable/uninstall prior to upgrade
* have an automated way to check for availability of D7 modules

While this is true for some folks, this is also a feature request. This thread is the contents of the upgrade.txt file -

The upgrade path for contrib will probably need to get worked out on a case by base basis within the contrib space.

It's also a feature request that's been filled by a contrib module - http://drupal.org/project/upgrade_status

While we very rarely link to contrib modules in core, it'd be good to have a generic contrib module handbook page which we can link to from UPGRADE.txt, then link to upgrade_status from there.

looks like a crosspost.

- Update 4) and Upgrade 6) are missing any files that may be present in profiles/
- I would rather recommend to unpack new drupal tarball, then copy sites/ and profiles/[profile] from old installation directory to new installation directory rather than removing anything from the old installation directory. This keeps the old installation directory completely intact.
- FIrst title says 'PRESUMPTIONS' - shouldn't that be 'ASSUMPTIONS' ? or introduction?

@ #48
I see. From my point of view the documentation should help to examine if - for a particular site - the boundary conditions are met. When there are tools / methods to check these conditions, then it should be mentioned in the documentation - regardless of modules being core or contib.

#52 - Yes missed out on profiles/ and Presumtions was a left over I had forgotten, but Introduction fits quite well - thanks!

Regarding unpacking the tarball I understand what you mean and I do that myself but not everybody have shell/ftp access above their web root so I think such "tricks" better foits in the online handbook.

new version of document attached and I used the D6 trick now to off load the test system.

#53 - While I agree the question is if this is _the_ documentation? I think it's more for the handbook while this document should cover the most common/obvious scenarios and do it in a comprehensive way, but maybe not all and everything. You work on it maybe once for each major release (unless something major) while the handbook can be prefected each day as it passes. JMHO

Feedback is always welcomed though and useful stuff that's just to copy and paste ;-)

yettyn, I took the liberty of cleaning up a couple typos throughout the doc you posted.

Oh howly Mathilda! I see it was shining through I'm not a native English writer! But I found 2 places where you slipped a blurp on the keyboard ;-) plus I realigned the document to stay within a character width of 80, except for 2 instances where it felt acceptable with 81 due to an ending . and /

Thanks for that review and new docs attached, the patch again with -D6 to prevent it from upsetting the test queue.

Subscribing - this is a great issue, because that file is indeed badly in need of work!

One quick thing came to mind while skimming it:

1.  Log in as a user with the permission "Administer software updates"
    enabled. This should be a user belonging to the ADMINISTRATOR ROLE, 
	preferably user ID 1 (also known as the site maintenance account). 
	IMPORTANT! Do not close your browser until the final step is complete.

(A) I don't think we need to say "preferably" for user 1. It should really work just as fine for another admin user - at least that's the idea.

(B) The administrator role is an optional feature in D7 and could confuse people here. I think saying the name of the permission ("administer software updates") should be enough.

(C) You repeat this message verbatim in the "6.x to 7.x" section below, but in the case of someone updating from 6.x, the "administer software updates" permissions won't exist yet, right? :) So in that case, we should just tell them to log in as user 1.

#58 - Yes good points I think no one can argue with so I have updated the doc accordingly. Also expanded a bit on the next to last step for both scenarios giving some directions in case there are problems.

Some general line edits/suggestions intended to keep things shorter and link out to more comprehensive documentation as needed:

In the introduction:

* The "Don't hack core" principle is respected. For background on this principle, please see http://drupal.org/node/144376

Note: keep it short - we can give info on the .htaccess/settings.php/robots.txt later, in context.

The following section can be edited:

This document assumes Drupal is installed in the web root, on an Apache
web server also known as the Document Root. If you have installed Drupal
in a subdirectory of your web root, you need to adjust accordingly when
the document mentions web root, Document Root or Drupal Installation
directory. The same are true for paths, like sites/default/ always starts
from the directory Drupal is installed in to.

We can break this down into smaller, more digestible sections:

* Drupal is installed in the web root on an Apache web server. "Web root" is also known as "Document root." If you have installed Drupal in a subdirectory of your web root, you need to adjust accordingly when the document mentions web root, Document Root or Drupal Installation directory.
* All paths (for example, sites/default) are relative the the directory where Drupal is installed

Update Instructions

In step 1 of the Update instructions, it reads: "IMPORTANT! Do not close your browser until the final step is complete." This warning should be moved to Step 7, because Step 7 is where the upgrade actually starts, and where the browser really can't be closed.

In step 4, people are instructed to move files out of their install. This step needs a warning added, something like: "IMPORTANT: do not move files out of your install until you have made a full backup of your current install. For more information on backing up your codebase, see http://drupal.org/node/22281"

Edit on step 6: If there are any modifications in your settings.php, .htaccess, or robots.txt, copy them from your site backup into the new codebase.

Edit on step 8: Change "Assuming all went well with the database update" to "After the site upgrade has completed"

I'll try and go through the "Upgrade" section later today -

OT: is there a testing profile for D6 that people are using to test the D6 to D7 upgrade? If so, can someone point me to it?

bonobo wrote:

is there a testing profile for D6 that people are using to test the D6 to D7 upgrade? If so, can someone point me to it?

A good one is the Drupal.org upgrade path test, used for the last beta blocker, but I think few people have access to it.

Maybe the Devel module can help, generating content for a test site.

So... I've been away for 6 weeks, and am just coming back to this discussion. Just looking at the patch/file in #59, here are some things that need to be fixed. I can fix them, but I'm assuming that yettyn wants to do the update, since that's who the issue is currently assigned to.

a) We don't use tabs in Drupal files, ever, that I'm aware of. Please format with a 2-space indentation instead of tabs.

b) Needs general proofreeding for punctuation, grammar, capitalization consistency, etc. Again, I'd be glad to edit the file, just let me know when you're done with it by unassigning it.

c) I think the first section would be clearer if you put "upgrade" and "update" in quotes. When I first looked at it, I didn't notice the difference between the words at first glance, and you are defining the terms later. Also, are we using these terms consistently on drupal.org in the Handbook? If not, I think sticking to "major upgrade" and "minor upgrade" would be clearer than arbitrarily deciding in this file only that it should be "upgrade/update".

d) Take this out, it is childish to put it in the file:
* You understand the difference between Upgrade and Update as stated above

OK, I stopped reading after the first few lines. The top of this file sounds like it's assuming the reader is stupid or malicious or something. Can we possibly just keep to the facts, rather than using a tone that implies you're assuming the reader is trying to screw things up?

Oh also: Please when you add a patch, name it *.patch, not *_D6.patch. It's not a D6 patch, is it?

#64 -

Oh also: Please when you add a patch, name it *.patch, not *_D6.patch. It's not a D6 patch, is it?

I did so just to off load the test frame work as it doesn't make sens to test a text file.

#63 -

OK, I stopped reading after the first few lines. The top of this file sounds like it's assuming the reader is stupid or malicious or something. Can we possibly just keep to the facts, rather than using a tone that implies you're assuming the reader is trying to screw things up?

Maybe you need to consider that not everybody are as experienced as you dealing with drupal and upgrading in general, just take awalk around the forums and issue queue as well. Sure keeping to the facts I can agree on that but a part of the fact is also that people do screw up and this file is supposed to help them doing less so.

I don't mean that what I have written need to be final in any way and as a non-native english writer I don't expect it, but I cannot help you been away for 6 weeks... no offence ment an non taken but I have seriously been trying to improve this document that been in a bad state and have been complained at for a long time. I have also been away by the way, hope your tripp was more enjoyable then mine but it's good to be back anyhow!

Anyway, it suits me fine if you like to come in take it over, than I can focus more on doing some real testing on doing D6->D7 upgrades and make comments on my findings.

Sorry! I wasn't trying to scare you away, and I definitely appreciate your work on the file! I'm also happy to edit text written by non-native speakers, and appreciate the effort it takes to write in a language not your own, even if you're pretty fluent. :)

I just also know that Dries/Webchick will be looking at the tone of the file, the writing style, etc. before they commit it. I will give it a quick rewrite and edit, and let's go from there.

jhodgdon wrote:

Also, are we using these terms consistently on drupal.org in the Handbook? If not, I think sticking to "major upgrade" and "minor upgrade" would be clearer than arbitrarily deciding in this file only that it should be "upgrade/update".

About the terminology of "major upgrades" like D6->D7, and the more frequent "minor updates" for security and bug fixes, this seems to be, not a general, but a growing naming tendency, for example in the new Drupal 7 Upgrade Guide (work in progress), which is different from the old Upgrading from previous versions.

There are different opinions about this, that are being discussed in the documentation issue "Upgrade" vs. "Update" language.

#66, it's ok and you have some valid points. This should be a team work and maybe I have got too used to fighting on my own ;-) but it sure feels good not needing to. I'm happy to see you giving it a second breath and it feels good I was able to kick off this important issue. Meanwhile I will do some upgrade testing... No wait, I will go to the movies as it's always healthy to take a break ;-)

There's still some room for making the language simpler. For instance, instead of:

This document outlines the following 2 scenarios:

* To Upgrade your Drupal site from version 6.x to 7.x
* To Update your Dupal site from one minor version to another of Drupal 7, e.g. from 7.9 to 7.10

we can say

This document describes how to:
* Upgrade your Drupal site from version 6.x to 7.x
* Update your Drupal site from one minor version to another of Drupal 7, e.g. from 7.9 to 7.10

For updating between minor versions, mention that it's OK to skip versions (you can update directly from 7.1 to 7.5). When we get to several minor 7.x versions, I think it's OK to go directly to the latest one (ie. from 6.19 to 7.5).

Under upgrade from 6.x to 7.x, point to list of contrib modules which are moved into core in 7.x at http://drupal.org/node/895314.

Good idea on #69. I am editing this document now...

I think that the last proposed version of this file had some excellent ideas in it. I cleaned it up, fixing some grammar, style, punctuation, factual, and formatting issues. Here's a new version for consideration... thoughts?

2. Prepare for the upgrade by ensuring that:

This should include "Make a LOCAL backup of your db and root directory".

Any modules that you do not plan to enable in your Drupal 7 site should also
be uninstalled after they are disabled -- do this by clicking on the
Uninstall tab on the Modules page. (Uninstalling removes database tables
and other information related to the modules, to keep your database compact.)

You maybe should not mention the "disable them first" just as an side note. This should be made more clear.

12. Check the configuration file sites/default/settings.php to make sure it
has the correct information for the connection to your database, in the
correct format for Drupal 7. You may need to consult the
sites/default/default.settings.php to find the correct format.

Based on the amount of changes between D6 and D7 to that file. Shouldn't this be like "apply your setting to the newly created settings.php based on the D6 information"?

14. Update and re-enable your non-core modules one by one, following this
procedure:
- Check your notes to see if any special upgrade instructions apply
- Fully remove the old module directory inside sites/all/modules/
- Download, unpack and move the new module directory to sites/all/modules/
- Re-run update.php and check for any message output

Following this procedure for every module may be time consuming if you
have many modules to re-enable, so it is possible to do more than one module
at the same time, especially if they are unrelated or don't have dependencies.
However, this also increases the likelihood something will go wrong.

You might want to add in which order you should reenable the modules. Which module should you enable first and which last?

12. Check the configuration file sites/default/settings.php to make sure it
  has the correct information for the connection to your database, in the
  correct format for Drupal 7. You may need to consult the
  sites/default/default.settings.php to find the correct format.

This shouldn't be necessary. If you have a D6 settings file, update.php automatically adds the database connection info in Drupal 7 format (and prompts you to make the file writable so it can do that).

In the "Minor Version Updates" section

In point 7, change "Assuming all went well with the database update" to "After the site update has completed"

Possibly, in this section, in the case of an upgrade that did not go as planned, we should include language instructing people to note any error messages they saw, and reminding them that they should restore their site to last stable version before attempting to update again (ie, don't attempt to update on top of a failed update).

In Major Version Upgrades

In section 2, some language to the effect of:

* If you are not sure of the status of any contributed modules and/or themes, DO NOT proceed with the upgrade.

My rationale here is to inject a note of caution into the process, and to make it very clear that that checking compatibility should not be glossed over.

In the module compatibility section, link to the handbook page that details the D6 modules that have been moved into core: http://drupal.org/node/895314

Between steps 6 and 7, we are directing the user to swtich from the web UI to the command line. This context shift should be highlighted.

Also in step 7, will the end user need to adjust permissions to access default.settings.php?

Step 11 - Recommended edit: "Re-apply any custom modifications to files such as .htaccess, settings.php, or robots.txt."

I'm not clear why we would recommend waiting for this, especially as some of these modifications might be needed for their site to run properly - or are we recommending that they wait as a means of eliminating the chance of human error fouling the upgrade process? In any case, if we are recommending that they wait, we should probably add an additional step either before they re-enable modules and themes, or before they pull the site out of maintenance mode, where they check for/add in any modifications to .htaccess, settings.php, or robots.txt.

Step 12 - Recommended edit: "Check the configuration file sites/default/settings.php to make sure it contains the correct username, password, and database name to connect to your database, in the correct format for Drupal 7. You may need to consult the new default.settings.php file downloaded with Drupal 7 to find the correct format." - and to be thorough, we should provide a link to some concrete examples in the handbook of what the "correct format" looks like.

In both sections:

This will create a new directory drupal-x.y/ containing all Drupal files
and directories. Move the contents of that directory into your Drupal
installation directory:

mv drupal-x.y/* drupal-x.y/.htaccess /path/to/your/installation

Just realize the mv command won't work here as sites/ directory isn't empty and even mv --force doesn't work.

Wonder if it wouldn't be easier to backup old site, weap it out and add new files and then copy in what's needed from the backup. If given clear instructions how to do it, it's probably the cleanest way. The above also assumes shell access and the excistance of a directory above the web root, question here I think is what's the standard with hosting companies, cpanel, fantasico etc.?

Another thing, so far I have never succeded to run new D7 update.php on upgrade without getting a Access denied page. Wouldn't it be better to face the fact (or someone prove me wrong) that it's a necessary step to set $update_free_access = TRUE; and bake this into the instructions as one of the necessary upgrade steps?

But maybe it's just me and my server, so if someone really have suceeded to run step 13 without changing $update_free_access = FALSE; please step forward and confirm this.

I think at the end of Major Upgrade part, possibly as final step, a suggestion to visit Getting started with Drupal 7 administration could add some value to the document. Something like:

To Get started with Drupal 7 administration, visit http://drupal.org/getting-started/7/admin

I just had one or 2 little nitpicky suggestions:

1. Line 207-208: Also remove modules whose functionality has been moved into the core of Drupal 7. Is there a page detailing these modules? I'd be happy to create one if there isnt.
2. Line 219: lines 219-243 could do with some kind of introduction that says
   what exactly we are doing (namely moving the core files to the site folder), like:

   10. Download the latest Drupal 7 release from http://drupal.org/ to a
   directory outside of your web root. The files are in .tar.gz format and
   can be extracted using most compression tools.

   Then you will need to move the contents of the file to your Drupal site
   directory using the command line, FTP or through your hosting panel
   interface:

   * Command line: (put lines 220-236 here)
   * FTP/SFTP: (lines 238-240 here - btw just a question: why might it be preferable to use FTP Windows and not, say, on a Mac?)
   * If you are using a hosting panel like Cpanel, you may need to follow their
   specific instructions for how to put your files on your site.

3. Improve references to the 'sites' directory. Line 21 states 'if you have modified
   core files outside of the sites directory' and line 48 states ...EXCEPT for the sites/ directory and any custom files.... I think line 48 is better, but perhaps best would be to put it in single quotes - the 'sites' directory. Why? Because on line 21 people (unaware of the work that has gone into this doc) might assume we meant the site's directory and we just made a typo.
4. Line 131-142:
   

   * You have done an inventory of all your installed non-core modules and
   themes, checking which have a Drupal 7-compatible version and which
   do not.

   * You know what needs to be done to update your non-core modules:

   - For contributed modules, visit http://drupal.org/project/modules and
   change 'Filter by compatibility:' to 7.x, then click the Search button
   to check for modules matching your version of Drupal.

   - For custom modules, review http://drupal.org/update/modules to
   ensure that your custom module is compatible with the current version.

   The first 2 subpoints of the second point might actually better belong to the first point, and the same with the theme points below. When I read the 'You have done an inventory...' point, I was asking myself, where's the page to do that? I found my answers in the next point, but that was under the update header. I'm probably making too much of this - all im saying is the flow is a little broken here.

@nirbhasa in #78, re:

Line 207-208: Also remove modules whose functionality has been moved into the core of Drupal 7. Is there a page detailing these modules? I'd be happy to create one if there isnt.

It's started here: http://drupal.org/node/895314 - it could use an overhaul/additional detail, imo.

bonobo wrote:

It's started here: http://drupal.org/node/895314 - it could use an overhaul/additional detail, imo.

For additional details (but not on drupal.org): More than 50 Drupal modules moved into Drupal 7

I haven't dove into this issue yet, but just wanted to raise as a general thing, please be careful with how much Drupal 7-specific stuff you introduce into this document (i.e. I get nervous about references to that "50 modules moved into Drupal 7" blog post). This document should ideally be able to be read once by people and then the principles applied version to version.

It also makes maintenance easier if I don't have to commit 50 follow-up patches for "Oh, and this is how you ugprade Imgae module, and this is how you upgrade Admin Role module, and...). I am fine if we want to create a handbook section on "Drupal 7 specific upgrade tips" or whatever (afaik one already exists somewhere and was created in Copenhagen) and link to it from UPGRADE.txt, but I don't want huge paragraphs of text in here that relate only to D7.

Carry on! :)

webchick (Drupal 7 maintainer) wrote:

This document should ideally be able to be read once by people and then the principles applied version to version. (...) I am fine if we want to create a handbook section on "Drupal 7 specific upgrade tips" or whatever (afaik one already exists somewhere and was created in Copenhagen) and link to it from UPGRADE.txt, but I don't want huge paragraphs of text in here that relate only to D7.

Yes, that handbook section in the works is Drupal 7 Upgrade Guide, where more detailed info can be added. (There is also a previous page, Upgrading from 6 to 7 tips). The old upgrade guide is at http://drupal.org/upgrade ("Upgrading from previous versions").

I think both old and new upgrade guides could be linked from the new UPGRADE.txt file. The old version includes several links to drupal.org sections in the different steps, and a final paragraph:

For more information on upgrading, visit
the Drupal handbook at http://drupal.org/upgrade 

Related documentation issue: Review and test the upgrade guide for Drupal 7.

webchick: I am fine if we want to create a handbook section...and link to it from UPGRADE.txt - i think thats what I/we meant too, apologies for the vague wording :)

From taking a look at that list, it talks about an upgrade process. I think its important that new user doesnt think 'OMG, I have to look at each of these 50 modules and see how to upgrade them!'. For most of those modules, the 'upgrading' is simply consists of removing the D6 modules from the 'sites' directory, and maybe in a few cases visiting the configuration page once the overall upgrade is done.

Perhaps I can change the title of that page http://drupal.org/node/895314 to reflect that - if there are some modules that do require additional steps, I can list beside the module what to do (if short) or a link to an upgrade page (if long). But it should mainly be a simple list of modules to remove from your D6 installation. Then we can move it into the handbook as specified in #82

(edited to add clarification)

I cleaned up this page http://drupal.org/node/895314 - suggestions welcome.

Just a clarification of #83: It is true that most modules can just be disabled, but some of the more well known ones require data migration. However, at this point in the upgrade step (line 207-208), the important thing is just to get all the will-be-in-core modules out of the sites folder.

A sentence on data migration of contrib modules (and a link to another help page) might belong at the end of UPGRADE.txt

Regarding #76 (by me), I think it must be this,

The following updates returned messages
user module
Update #7000
User passwords rehashed to improve security

resulting in that the session are lost and you are logged out. So either should this be reflected in the upgrade documentation or get filed as a bug - although I'm not sure if it's even possible to rehash the password for the currently logged in user, without losing the session, at least not in a safe way, but maybe some of the php gurus here can answer that question?

I have tried now in various ways to upgrade and find it impossible to stay logged in and even if $update_free_access = TRUE; is set or not, when time comes to visit administration pages you always get an Access denied page and have to go visit hxxp://www.example.com/user and log in.

Again, I call for someone to step forwards and claim he can stay logged in during the upgrade process all the way to it's time to visit hxxp://www.example.com/admin ?

@yettyn: I would open a separate issue for not being able to stay logged in during the upgrade progress - while this may end up being something we have to document in UPGRADE.txt, it'd be good to get it confirmed as a bug, and fixed if possible.

Lots of good comments since the last proposed version in #71... looks like we need a new version?

I think so...are you happy/free enough to do it? I could give it a go, but it would be tomorrow before I got around to it.

I will assign to myself and do it if/when I have time. Or someone else can do the same when they have time. :)

OK, I'm making another pass at this one.

RE comment #72 above:
2. Prepare for the upgrade by ensuring that:
"This should include "Make a LOCAL backup of your db and root directory"."

That is in the Introduction section, since it applies to both types of upgrades.
--- more from #72:
Any modules that you do not plan to enable in your Drupal 7 site should also
be uninstalled after they are disabled -- do this by clicking on the
Uninstall tab on the Modules page. (Uninstalling removes database tables
and other information related to the modules, to keep your database compact.)

"You maybe should not mention the "disable them first" just as an side note. This should be made more clear."

Ummm. It's just after the paragraph that says to disable all non-core modules -- how much clearer does that have to be?
--- More from #72:
The comment on step #12 is addressed in comment #73 above.
-- More from #72:
And a question was asked about what order to enable modules. I don't have an answer to that at all. Presumably a site admin can figure out an order that makes sense to them?
-- Comment #73
I changed this to say you should make sure settings.php is writeable (even though the update script will prompt for it).
-- Comment #74
I took all of these suggestions, except for the one that conflicted with #73.
-- Comment #75
I changed these to cp, which I think will work fine?
-- #76
Leaving as-is for now, as per comments 85-86
-- #77 - good, added
-- #78 - all good points, I made some edits accordingly.

Phew!

Here's a new version... hopefully a bit better. Please review/criticize!

I get nervous about references to that "50 modules moved into Drupal 7" blog post

We're close to 100 in the meantime. Just for the record. Upgrade Status will be updated shortly.

--
Here's my initial review. Note that I have tried to remain as ignorant as possible as to what's been going on in this issue, so that I can give it a totally fresh set of eyes. Apologies if any of this was already mentioned above.

Also note that my comments may sound harsh due to their shortness. Just take the message, please, nothing else. I really welcome this effort, as it took me hours to figure out the "proper/intended" upgrade path myself.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  * Update your Dupal site from one minor 7.x version to another
+    minor 7.x version; for example, from 7.9 to 7.10 (first section below).
...
+MINOR VERSION UPDATES
+---------------------
+(Major version upgrades are described in the section below.)

UPGRADE should not describe how to update from a point release to another. If we want to document that, then we must add a separate UPDATE.txt. However, UPGRADE.txt should keep clean and mean and focused on upgrading to a new major version. It's bloated enough already.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+1. Upgrade to the latest available version of Drupal 6.x, following the
+  instructions in the UPGRADE.txt file in your Drupal 6.x installation. If you
+  are currently running Drupal 5.x, you will need to upgrade to 6.x, and if you
+  are not yet running the latest 6.x version, you will need to update to the
+  latest 6.x version.

1) Wrong indentation.

2) This paragraph can be shortened.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  * Your system meets or exceeds minimum requirements for Drupal 7 as
+    specified at http://drupal.org/requirements

update.php already tells you.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  * You have done an inventory of all your installed non-core modules and
+    themes, checking which have a Drupal 7-compatible version and which
+    do not.

Not required.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+    - For contributed modules, visit http://drupal.org/project/modules and
+      change 'Filter by compatibility:' to 7.x, then click the Search button
+      to check for modules matching your version of Drupal.
+
+    - For custom modules, review http://drupal.org/update/modules to
+      ensure that your custom module is compatible with the current version.
+
+    - Check each module's project page for any special procedures to follow
+      when upgrading to Drupal version 7, and read any UPGRADE.txt files found
+      in the Drupal 6 or 7 releases of the module. You may find that your module
+      has been included in the core of Drupal 7, has been discontinued, or has
+      not yet been upgraded for Drupal 7.
+
+    -  If there is a problem or a conflict with this document, consult one of
+       the support options listed on http://drupal.org/support

All of this does not belong into this document. If we want to write handbook pages, then we can link to them at the beginning, but UPGRADE.txt is not and should not be a handbook page.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  * You know what needs to be done to update your non-core themes:
+
+    - For contributed themes, visit http://drupal.org/project/themes and
+      change 'Filter by compatibility:' to 7.x, then click the Search button
+      to check for themes matching your version of Drupal.
+
+    - For custom themes, review http://drupal.org/update/theme and
+      http://drupal.org/theme-guide to ensure that your custom theme
+      is compatible with the current version.
+
+    - Check each theme's project page for any special procedures to follow
+      when upgrading to Drupal version 7, and read any UPGRADE.txt files found
+      in the Drupal 6 or 7 releases of the theme. You may find that your theme
+      has been discontinued, or has not yet been upgraded for Drupal 7.
+
+    -  If there is a problem or a conflict with this document, consult one of
+       the support options listed on http://drupal.org/support

Same here. Upgrading a theme from D6 to D7 is a nightmare on its own (I can tell you), but anything except the bullet point "Change your default theme to Garland (and disable your maintenance theme override)" has nothing to do with this text file.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  * At this point you may judge whether an upgrade of your site to Drupal 7.x is
+    feasible. If not, you might consider contributing to the modules or themes
+    that are not yet ready for Drupal 7.

Remove.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+3. Log in as user ID 1 (also known as the site maintenance account).
+  IMPORTANT! Do not close your browser until the final step is complete.

That appended warning is based on what, exactly? Remove.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+4. Navigate to Administer->Site configuration->Site maintenance. Check the
+  "Off-line" radio button and click "Save configuration".

AFAIK, update.php already ensures the maintenance mode. But even if that may not be the case for the major upgrade: The system menu pointers in this text need love.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+5. Switch to the core theme Garland, if you are using a different theme. The
+   easiest way to do this is to navigate to Administer->Site building->Themes,
+   and click the "Reset to defaults" button at the bottom.

...the first, really important upgrade task.

Resetting to defaults, however, *deletes* any possibly associated variables, so this advice is wrong.

Cut all the rest after the first comma.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+7. Remove the sites/default/default.settings.php file, so that it can be
+  overwritten by the Drupal 7 version.

A file that is removed can, by definition, not be overwritten.

Care must be taken prior to deleting this file. There should at least be a backup.

Furthermore, I wonder whether D7 isn't able to overwrite the file as is on some/certain configurations. At least, I think there's some code that tries to *rewrite* the database settings.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+9. Remove any modules from sites/all/modules and other sites/*/modules
+  directories that you don't plan to enable in Drupal 7. Also remove modules
+  whose functionality has been moved into the core of Drupal 7. You can leave
+  the Drupal 6 versions of other modules in place, even though they are
+  incompatible with Drupal 7. The benefit of leaving them is that Drupal 7 will
+  list them and show their dependencies, even though they can't be enabled. This
+  may help you in the process of upgrading them in a non-conflicting order.
+
+  Follow the same process with custom and contributed themes.

This can be shortened.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  web browser, unpack the files with a program like winrar or 7-zip and then

If we really want to state individual software vendors here, then we should capitalize/write them properly. (However, we should remove those mentions)

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  If you are using a hosting panel like Cpanel, you may need to follow their
+  specific instructions for how to put your files on your site.

Remove.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+11. If you need to re-apply any custom modifications to files such as .htaccess,
+  this may be a good time for it. However, unless you are an expert and
+  really know what you are doing, we suggest you wait until your new site
+  is up running and confirmed to run properly.

Completely remove.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+14. Update and re-enable your non-core modules one by one, following this
+  procedure:

"one by one" is wrong, remove.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  Following this procedure for every module may be time consuming if you
+  have many modules to re-enable, so it is possible to do more than one module
+  at the same time, especially if they are unrelated or don't have dependencies.
+  However, this also increases the likelihood something will go wrong.
...
+15. If applicable, return the site to its original theme following the same
+  procedure as in #14, putting the theme in sites/all/themes/.
...
+16. Assuming all went well with the database updates, check your site to verify
+  that everything is working as expected.
+
+  If there is a problem, consult one of the support options listed on
+  http://drupal.org/support

Completely remove.

Replace with a one pre-final step "Check your site's status report."

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+For more in depth information on upgrading, visit
 the Drupal handbook at http://drupal.org/upgrade

This has to be at the top, not at the bottom. Anyone, who can't make any sense of the text file should resort to the handbook page(s).

Powered by Dreditor.

I think that was a cross post... but some of the comment still apply to the latest version - sun, can you confirm which version you were reviewing?

I'm doing a handbook page to deal with the upgrading of custom modules and themes, using some of the text that sun suggested excising from the main UPGRADE.txt.

+++ UPGRADE.txt	15 Oct 2010 19:53:24 -0000
@@ -1,114 +1,300 @@
+  web browser, unpack the files with a program like winrar or 7-zip and then

If we really want to state individual software vendors here, then we should capitalize/write them properly. (However, we should remove those mentions)

I can understand the reluctance to mention vendors as a rule, but it should be recognised that inexperienced users might not be aware what software to use. In addition the phrase 'unpack' will be one many users might not have come across ( Where extract might be better)
EG.
+ web browser, unpack / extract the files with an archive tool such as one of the rar or zip family of programs and then

I accept that my wording here isn't great either, while its more complex that before it approaches something thats more understandable ( and generic) for users.

It may even be better just as
+ web browser, unpack / extract the files and then

I agre that this is could be improved:

+  Following this procedure for every module may be time consuming if you
+  have many modules to re-enable, so it is possible to do more than one module
+  at the same time, especially if they are unrelated or don't have dependencies.
+  However, this also increases the likelihood something will go wrong.

However Should there not be some warning that turning on a bunch of modules at the same time makes it harder to spot where an error comes from even if it doesn't actually create more errors. My point here being that we should recommend turning modules on after an update one by one, or in small groups to help troubleshooting. You will get users who will get errors, and if they have just updated/re-enabled even three or four modules and got an error, who could say where it came from?

+  re-enabling modules one at a time or in small unrelated groups
+  will make is easier to spot any issues that might occur

I created a page for upgrading non-core modules and themes (http://drupal.org/node/948216) and moved the contents of point 2 there. This latest patch replaces that text (added -D6 to escape the testbot). I didn't address the rest of sun's points, as I felt there might be better placed people than me to do it :)

Also moved contents of points 14-16 there, but i left that bit of UPGRADE.txt file as is for the moment, as I was unsure of the best approach to recommend when updating modules - one by one, a few at a time or all at once? I would lean towards all at once (and perhaps ask them to make a db backup after they run update.php for core).

Why? The most likely reason that something goes wrong during upgrade, is that the upgrade path for a particular module is broken - in this case, we can look at the messages upgrade.php spits out, disable the offending module and run update.php again. In this case, the process is still quicker than updating one by one :)

Perhaps another reason might be that modules don't play together well, but i don't think this is as likely, seeing as they did play together in D6. In any case, this issue is more likely to affect experienced site admins with 100+ modules on their D6 site, then newbies who are intently studying UPGRADE.txt point by point as they upgrade their site.

(edit: i mistakenly patched against version in #91, rather than original UPGRADE.txt.)

I don't think I agree with sun about removing all that stuff from the upgrade.txt file, actually. I'm sure some of it can be removed, but I think there should be enough information in the file for someone to be able to do an upgrade/update. Otherwise, why have the file at all? It could just be one line and say "go to this page on d.o", but that's not the philosophy we've adopted with the install.txt file. It should have a similar level of detail to install.txt probably?

@jhodgdon: yeah, the review on the patch before yours.

@verbosity:

You will get users who will get errors, and if they have just updated/re-enabled even three or four modules and got an error, who could say where it came from?

It would be wrong to suggest an overly defensive procedure, just because there is a possibility of bugs. That possibility exists at any time, not only during upgrading.

@nirbhasa:

all at once (and perhaps ask them to make a db backup after they run update.php for core)

Exactly. Doing backups at the right point in time is most important in the process. That is the level of defense everyone should follow. If, and only if an operation fails, you go back to the last backup milestone and re-attempt the operation differently. Documenting the edge-case of errors during the upgrade belongs into the handbook page though.

hmpf... UTF-8.

Reading through the patch in #100 - some quick initial thoughts here, with more to come after more thought and a more thorough read through later in the day

RE:

UPGRADE should not describe how to update from a point release to another. If we want to document that, then we must add a separate UPDATE.txt

While I'm always a fan of brevity, not including the steps to perform a minor update in this file seems like a net loss. It reduces the usefulness of this documentation. It should be added back in.

Including this information in an additional file is *not* an acceptable alternative, as the thread on the "difference" between update and upgrade helps illustrate. While we (the people in this thread) might come close to understanding and difference between these terms, having two files with a nearly identical name, with very similar instructions, both requiring the use of update.php, seems like a bad usability choice.

For these reasons, and for the reason that this doc should cover the range of scenarios people will expect to find here, we should return that info to the doc.

I didn't test the instructions but they are *very very clear and good*! :D
Thumbs up for everyone editing this document!

From the version attached to #100:

If you uninstalled any modules, remove them from the sites/all/modules and other sites/*/modules directories. Leave other modules in place, even though they are incompatible with Drupal 7.x.

We should make it very clear here that we are referring to cruft, unused modules, and not the modules that were disabled as part of step 6.

On point 15, the process of creating the new sites/all/modules directory needs to clarified.

- Download and extract the D7 versions of your contributed modules
- Collect the new versions of your contributed modules into the sites/all/modules directory
- Some modules might require third-party libraries; check the module project pages for these dependencies

#100: drupal.upgrade.100.patch queued for re-testing.

We definitely need to have both minor and major upgrades in this document. That has been thoroughly hashed out above. So let's go back to the version on #97 and review that one.

It is time to move minor updates out of the upgrade file. +1 to #100 as cleaner starting point to make this document shorter and cleaner

@steinmb - aside from it makes for a shorter file, what's your rationale for this change?

The majority of people reading this file will be looking for instructions for a minor update. Because we are currently focused on the D7 launch, we are more more focused on the D6 to D7 upgrade process, but we should not allow our current focus to shape (and decrease) the overall usability of the upgrade.txt file.

Just throwing my 2 cents into this. I think we should drive #100 towards a committable state (which I think it pretty much already is), close this issue, and then open new targeted issues for further polishing (e.g., #103), and open a new "major" (as in issue priority) issue for documenting a "minor update".

In that new issue, we can continue the discussion of whether "update"/"upgrade" is the correct terminology for distinguishing an increase in minor vs. major version, since #941896: "Upgrade" vs. "Update" language is still in "active" status, http://drupal.org/node/338208 contains no mention of this terminology standard, and http://drupal.org/upgrade does not follow it, so I can only conclude that that decision is not yet settled. Once that decision is settled, we may decide we want an UPDATE.txt file, or we may decide it makes sense to put it all in UPGRADE.txt. In either case, we all know it's needed somewhere, and that it is a release blocker for 7.1, but it is not a release blocker for 7.0.

Meanwhile, this issue was made critical in #22, because the current, completely wrong information in HEAD's UPGRADE.txt file, is blocking our ability to solve another critical issue: #887288: Attempting to use update.php without having configured a database in settings.php results in incorrect/misleading error message. So what do you all think? I know you've all worked hard getting it this far, and would love to get it perfect, but would it be possible to call #100 good enough for now, and consider the remaining work that's needed not critical to getting 7.0 out the door?

I still don't understand why a couple of people want to move the minor upgrade stuff out of this file. Has that decision been made or endorsed by webchick/dries, and do we really want to have yet another .txt file floating around at the top level of the Drupal distribution?

The patch in #100 is very close, but not there yet.

The following changes get us closer/possibly all the way there:

1. add the update text for updates within the 7.x branch - the language needs some cleanup, but that would be pretty minor work

The update/upgrade issue is (as I see it) a side issue, and should not affect this at all. The current text makes the process clear, and that's what matters.

The other wordsmithing issues I raise (in #103, above) can either get rolled in here, discarded altogether, or addressed via another issue.

The single blocker I see here is reverting the change that removed the updates within 7.x from upgrade.txt

In either case, we all know it's needed somewhere, and that it is a release blocker for 7.1, but it is not a release blocker for 7.0.

I think that a release blocker for 7.1 is also a release blocker for 7.0. (Because for all we know, three days after 7.0 is released there could be a security issue which necessitates 7.1 being released immediately - we hope that won't happen, but it could!)

I don't care if the issues are split up or not, but both are important. And it definitely makes sense to have this all in one file (at least certainly for the time being; it would be a major change to do otherwise).

Regarding #887288: Attempting to use update.php without having configured a database in settings.php results in incorrect/misleading error message, maybe we've already settled this enough to reopen that? The relevant part of the upgrade instructions for that, I think, is this part:

12. Make your settings.php file writeable, so that the update process can
   convert it to the format of Drupal 7.x. settings.php is usually located in

     sites/default/settings.php

Rereading #22, it seems we might also want to tell people to compare default.settings.php to their (old) settings.php and manually copy over some of the other new core settings (if so desired) - or maybe there are even some we should be copying automatically?

But, that doesn't affect #887288: Attempting to use update.php without having configured a database in settings.php results in incorrect/misleading error message anymore - that issue is just about what to do for people who have followed some other procedure than the one here, wound up with a settings.php file that has neither the D6 $db_url nor the D7 $databases in it, and what kind of error message to show them.

I prefer to add the minor update into #100 (preferably shortened down similar to what sun did with the major upgrade).

Some more suggestions:

1. In the introduction, write Don't hack core. instead of The "Don't hack core" principle is respected.

2. In step 6 for major upgrade, remove the last sentence, and add the following new paragraph:

If you have installed any of the modules listed on http://drupal.org/node/895314, there may be additional steps you need to follow. (More information on that page.)

Reason: There may be special steps you need to follow to avoid losing data or having the upgrade crash and burn. (I think I had one crash when not removing Update status completely going from D5 to D6.)

3. I really really really agree with #41. If a user has only done minor updates before, he/she might not realise that a major upgrade is another beast entirely.
So, start the major upgrade section with something like:

IMPORTANT: Do major upgrades on a test copy of your site, taking notes as you go, before upgrading your live site.

I think that a release blocker for 7.1 is also a release blocker for 7.0.

Good point. So if we do what I suggest in #110, and commit something along the lines of #100 (without the minor updates section), we should leave this issue open, and critical. But, we can at least then un-postpone #887288: Attempting to use update.php without having configured a database in settings.php results in incorrect/misleading error message, which is at least some progress.

But, that doesn't affect #887288 anymore - that issue is just about what to do for people who have followed some other procedure than the one here.

Right, but we can't tell people, "you did something wrong, go read UPGRADE.txt", if our UPGRADE.txt file is wrong, which it currently is in HEAD. That's why I want a correct one committed to HEAD, even if it's not perfect yet.

The update/upgrade issue is (as I see it) a side issue, and should not affect this at all. The current text makes the process clear, and that's what matters.

It seems to me the update/upgrade issue is pretty key here. #97 is written with the assumption that decision has been made. If you add the "Minor Updates" section back in to #100, and use "update" within its body, and have it say "(Major version upgrades are described in the section below.)", and then have "Major Upgrades" use "upgrade" within its body, and say "(Minor version updates are described in the section above.)", you're establishing a pattern that we haven't agreed to yet, and that our handbook pages aren't consistent with. #100, as it is, currently sidesteps that issue. It uses "upgrade" to refer to pretty much everything, which is consistent with the name of the file.

I still don't understand why a couple of people want to move the minor upgrade stuff out of this file.

A couple people might want to do this, but just to be clear, that is not my position. My position is that we commit something that doesn't deal with minor updates, unblock the other issue, and then continue deciding how we want to deal with documenting minor updates. I'm open to it being in the same file or a different file.

Has that decision been made or endorsed by webchick/dries, and do we really want to have yet another .txt file floating around at the top level of the Drupal distribution?

And it definitely makes sense to have this all in one file (at least certainly for the time being; it would be a major change to do otherwise).

A major change (requiring a signal from webchick/Dries at this stage of the discussion) relative to what? D6's and HEAD's UPGRADE.txt file doesn't distinguish between major and minor upgrades (except a tiny mention in step 11). D7 is where we decided that the two processes are different enough in order to warrant separate explanations. We have no precedent for whether to make the separate explanations in the same file or in different files.

Hm. I think I'm going to side with effulgentsia on this.

We have two problems here:

1. UPGRADE.txt is quite simply wrong.
2. UPGRADE.txt confuses people and could be much better written.

I think we need to fix #1 here (tagging as beta blocker) and #2 in a separate, major, issue.

So the question in front of us is, do we:

1) Set #100 to RTBC and commit it, despite missing information about minor updates, which are totally irrelevant to everyone using beta2, and then keep this issue open, critical, but not beta blocking, to work out how to incorporate information about minor updates, or

2) Add the "Minor Updates" section from #97 back into #100 and commit that, and accept that then HEAD/beta2 will contain update/upgrade terminology that has not yet been officially agreed to by the Drupal community, and consider it a "major" issue to follow-up with getting that cleaned up?

#1 to option 2.

EDIT: this should read +1 to option 2.

Oy.

IIRC, webchick and jhodgdon also voted for #117.2. Seems majority opinion is clearly that way. I'll re-roll and post shortly.

Added the Minor Updates section from #97, but changed wording to be consistent with #100 where possible.

Typo in opening section:

Update your Dupal

Should be "Drupal"

Aside from that, it looks good.

+1 to commit.

Thanks. Typo fixed, and given #121, RTBC.

Committed to CVS HEAD. Thanks.

Thanks, all! For those wanting to keep perfecting, please do so on #949102: Polish UPGRADE.txt. That way, new people can join the discussion without having to wade through all the comments on this issue.