The order in which updates for the hosting module currently run causes an issue when upgrading from a 0.3 setup (or any pre-alpha9 version it seems) to 0.4-beta2 (or probably to any version post-alpha8). This order was changed in 97b6111 (can't find a corresponding d.o. issue for this commit).

The commit above changed the order in which updates are run to:

  • hosting_update_6002()
    • "Manually" calls hosting_update_6008()
  • hosting_update_6005()
    • Installs hosting_server module, which includes setting up the {hosting_server} table
  • hosting_update_6008(), which fails if hosting_update_6005 did not run since it queries the {hosting_server} table.

The end result is that hosting_update_6008() fails with:

Executing hosting_update_6000 [17.81 sec, 16.36 MB]                  [success]
Executing hosting_update_6001 [17.81 sec, 16.36 MB]                  [success]
Executing hosting_update_6002 [17.81 sec, 16.36 MB]                  [success]
WD php: Table 'consolelocalhost.hosting_server' doesn't exist        [error]
query: SELECT n.nid, title FROM node n LEFT JOIN hosting_server s ON
n.nid=s.nid WHERE n.status = 1 AND n.type='server' in
~/WWW/hostmaster-0.4-beta2/profiles/hostmaster/modules/hosting/hosting.install
on line 236. [17.81 sec, 16.36 MB]

Comments

mr.baileys’s picture

mr.baileys
Primary language Dutch
Location 🇧🇪 (Ghent)
commented

After digging deeper, it seems that this will only affect upgrades from versions < 0.4-alpha6 (since that was when the hosting_server table was introduced). As suggested by anarcat in IRC, a temporary solution could be to upgrade to 0.4-alpha8 first.

memtkmcc’s picture

memtkmcc commented
Priority: Normal » Major

The upgrade from 0.4-alpha8 also fails: http://drupalbin.com/17788

We should really fix the upgrade path, so it at least doesn't fail because of wrong order (or similar fail) of db updates.

[EDIT] There were also other issues, related to changes in the config files/dirs structure, but maybe we should at least strongly recommend incremental upgrades, like alpha8 -> alpha15 -> beta2 -> rc1.

anarcat’s picture

anarcat commented
Assigned: Unassigned » anarcat

I'll take a look at this now.

anarcat’s picture

anarcat commented
Status: Active » Needs work

Part of the problems can be resolved by running this command before the upgrade:

drush sqlc "update system SET weight = 0 WHERE type='module' and name='hosting';"

This is a long-lasting bug in hostmaster that I will try to fix, but for now that's a good workaround. The next problem is with ports:

Executing hosting_update_6009 [1.9 sec, 9.98 MB]                                                                                                                                                                                    [success]
WD php: Unknown column 'port' in 'field list'                                                                                                                                                                                       [error]
query: SELECT port, restart_cmd, available FROM hosting_service WHERE vid = 2 AND type = 'apache' in  on line . [1.9 sec, 9.98 MB]
WD php: Unknown column 'port' in 'field list'                                                                                                                                                                                       [error]
query: SELECT port, restart_cmd, available FROM hosting_service WHERE vid = 2 AND type = 'mysql' in  on line . [1.9 sec, 9.98 MB]
WD php: Table 'aegirtniorg_0.hosting_ip_addresses' doesn't exist                                                                                                                                                                    [error]
query: SELECT ip_address FROM hosting_ip_addresses WHERE vid=2 in /srv/aegir/hostmaster-0.4-stable/profiles/hostmaster/modules/hosting/server/hosting.ip.inc on line 72. [1.9 sec, 9.98 MB]
[...]
Unknown column &#039;port&#039; in &#039;field list&#039;                                                                                                                                                                           [error]
query: SELECT port, restart_cmd, available FROM hosting_service WHERE vid = 2 AND type = &#039;apache&#039; in  on line . [1.91 sec, 10.04 MB]
Unknown column &#039;port&#039; in &#039;field list&#039;                                                                                                                                                                           [error]
query: SELECT port, restart_cmd, available FROM hosting_service WHERE vid = 2 AND type = &#039;mysql&#039; in  on line . [1.91 sec, 10.04 MB]
Table &#039;aegirtniorg_0.hosting_ip_addresses&#039; doesn&#039;t exist                                                                                                                                                             [error]
query: SELECT ip_address FROM hosting_ip_addresses WHERE vid=2 in /srv/aegir/hostmaster-0.4-stable/profiles/hostmaster/modules/hosting/server/hosting.ip.inc on line 72. [1.91 sec, 10.04 MB]
anarcat’s picture

anarcat commented
Status: Needs work » Needs review

I have committed a fix to hosting master (ra2dba3af6f4e, merged into 0.4 too) that goes around most of the stuff detailed here. The only problem remaining is the weight of the module. I have a fix waiting in head (f64487706638) that needs regression testing before merging in. Apparently, the weight of the module was introduced in commit ffe3533e, back in 2008 (!!) to "hide unsupported features" (?!). The commit in the frontend fixes that, but it will not fix the upgrade path because well, updatedb needs to run first. :P So I have introduced another hack, in the backend this time, that should fix it (aa1e08c5d73a, on master too). That fix should be tested then merged into 0.4 if it works, then *reverted* from master (because it's not necessary in post 0.4!).

For those less crafty with git, just try the following:

* run this before the upgrade:

drush sqlc "update system SET weight = 0 WHERE type='module' and name='hosting';"

* upgrade to the 0.4 branch or master

That should fix the upgrade path.

stella’s picture

stella commented

I had to upgrade from 0.4alpha7 to 0.4rc1. Changing the weight of the module fixed the majority of the errors. I then tried using latest head from git but couldn't get it to work. I eventually got it upgraded to alpha8 but couldn't upgrade to alpha15 - 3 errors were returned, but no messages explaining why.

My sites were offline, so the quickest solution for me was to install a clean rc1 and import my platforms and sites from there, which worked perfectly.

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs review » Needs work

Lots of errors still occur upgrading from 0.3.

Too many hook_updates are dependent on schema changes, so far identified in hosting_task, hosting_platform and hosting_package, that are taking a long time to trace down. My theory is that the weight was set high so that those other modules could complete their hook_updates first.

The first snag occurs at hosting_update_6002() because we rely on an API call of hosting_add_task() which fails at the 'delta' column missing in hosting_task table.

I worked around that and then hit similar issues with the 'makefile' column in hosting_platform and some i.version_code or something in hosting_package.

My first output, I'm stopping for now: http://pastie.org/private/rrupeaxhw50d6mj43y4xq

anarcat’s picture

anarcat commented
Priority: Major » Critical
Issue tags: +Upgrade path

It looks like the 6002 update tries to verify all platforms and sites - that seems to be a bad idea, or at least we should do this once, at the end of the update.

I am more and more feeling that a lot of this stuff can go into hostmaster-migrate directly, now that we have it. Something it could do, for example, is exactly those verify...

There are a few updates I feel we should totally drop now:

hosting:
6002 - re-verify at least one platform and all sites

hosting_platform:
6000 - Verify at least one platform This is to make sure ~configincludesglobal.inc is properly setup

hosting_site:
6004 - Re-verify all sites This is to make sure the open_basedir protection is in place

I have turned all those into a noop (in fact the latter two were already noops) - if we want to verify all sites, maybe that can be done after the whole frontend upgrade path is done?

The way things stand, anyways, is that sites, platforms and servers need to be verified *after* the whole updatedb thing, I believe... And this is not what the current upgrade path does *anyways*. I believe that mass verification should be a policy decision made by the operator during the upgrade, so that we have control over that... Anyways, the dispatcher is not running during hostmaster-migrate (it's been paused in hostmaster-pause) so we can't depend on those tasks being ran at all...

Let's see what happens if we don't run those tasks, shall we? Please try the master branch again.

Anonymous’s picture

Anonymous (not verified) commented

Great!

I had a successful upgrade from 0.3 with the master branch.

The only error I saw was this:

Executing hosting_web_server_update_6002                                                                                                                      [success]
DROP TABLE {hosting_web_server}                                                                                                                               [success]
WD php: Invalid argument supplied for foreach() in /var/aegir/hostmaster-HEAD/profiles/hostmaster/modules/hosting/task/hosting_task.module on line 12.        [error]
'all' cache was cleared                                                                                                                                       [success]
Invalid argument supplied for foreach() in /var/aegir/hostmaster-HEAD/profiles/hostmaster/modules/hosting/task/hosting_task.module on line 12.                [warning]
Finished performing updates.                                                                                                                                  [ok]
Failed to revoke user privileges                                                                                                                              [warning]

But nonetheless it seemed to work... looking into it more

anarcat’s picture

anarcat commented
Status: Needs work » Needs review

That's a rather weird warning, considering it comes from there:

    foreach (hosting_available_tasks($type) as $task => $info) {

The thing is, I think, that $type = 'server' and at that point, there's no server defined so the hook returns nothing. I have committed a patch to master (a8dfee3) that tries to fix this, can you test the upgrade again?

Anonymous’s picture

Anonymous (not verified) commented
Status: Needs review » Reviewed & tested by the community

I got no more warnings like that on a new test, so let's merge that in.

anarcat’s picture

anarcat commented
Status: Reviewed & tested by the community » Fixed

I couldn't find the commits mentionned in #6 on master, for some reason. I have merged those two commits instead:

http://drupalcode.org/project/hostmaster.git/commitdiff/75191804926c5c29...
http://drupalcode.org/project/hostmaster.git/commitdiff/a8dfee3391cb89f0...
http://drupalcode.org/project/hostmaster.git/commitdiff/520fe7a5c1af4aa8...

we'll just try to release rc2 with this stuff, regardless of those other commits.

Status: Fixed » Closed (fixed)
Issue tags: -Upgrade path

Automatically closed -- issue fixed for 2 weeks with no activity.

Issue tags: +Upgrade path

Restoring issue tags, see #2125755: System messages removed all issue tags during D7 upgrade.

  • Commit af5fba1 on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat: 
    #1068280 - try to fix upgrade path again by avoiding to generate new...
  • Commit 8ed075b on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - don't loop over empty tasks lists
  • Commit 84d956b on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - try to fix upgrade path again by avoiding to generate new...
  • Commit fed9606 on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - don't loop over empty tasks lists

  • Commit af5fba1 on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat: 
    #1068280 - try to fix upgrade path again by avoiding to generate new...
  • Commit 8ed075b on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - don't loop over empty tasks lists
  • Commit 84d956b on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - try to fix upgrade path again by avoiding to generate new...
  • Commit fed9606 on 6.x-2.x, 7.x-3.x, dev-ssl-ip-allocation-refactor, dev-sni, dev-helmo-3.x by anarcat:
    #1068280 - don't loop over empty tasks lists