[meta] Battle plan for Update Manager

I'm not sure about this. Update manager seems to work just fine in the default case, I've verified on localhost and DreamHost and we *haven't* had a huge influx of bugs coming in. I do see a lot of major bugs in that list that are going to hit people on the edges, but don't understand why most of those couldn't be fixed in 7.1, 7.2.

Just fixing the typo in the title!

I don't know if my comments would be appreciated here or not but I feel they should be herd.

D7 is shaping up to be a really great reslease in the usability scope of things. That means drupal will have a larger user base of people that are not developers, theme'rs.. Basically a whole set of users who are really limited in their technical scope. This is wonderful I'm sure we will even be 'poaching' some of wordpresses user base. Most of this user set see this as a huge requirement. Having something as half baked as what's currently available is only going to hurt the project. So what dww has said I really appreciate and back. This project should rather live as a add-on module if it can not be usable by this user set (as this is where it is targeted at).

My 2 cents and views.

Can someone please explain to me what, specifically, is "half baked" from a non-technical user POV about Update Manager?

Larry was going through this list earlier and noted that we have a pretty big UX WTF where if your server doesn't support any of the available backends, it will look as though it does but then give you a weird non-sensical error. (I believe that's #686060: Explain that the Update manager only works if you have FTP or SSH access to your server.) So, sure. We should fix that. But that doesn't render the entire system useless, and definitely not enough to warrant ripping out a feature that's been in place for over a year. And is one that those very non-technical users you're talking about winning over desperately need (this is why we made a feature freeze exception for it in the first place).

I'm just a user, or not a code developer and haven't been using drush..

I really like the update/upgrade manager because it's cool and a huge time saver for all or most drupal users. It will help people in the long run to keep it, even if it has some rough edges. Although I haven't really worked it a lot at this point,, Just sayin I think it is a very useful tool and a keeper for usability reasons.

dbeall++
I'm in the same boat, just a user not a code developer. Although my goal is to learn to use Drush eventually, I haven't (partially because the server on which a D6 production site i take care of doesn't have permissions set correctly yet). Anyway, my point is that the update manager is important to me. It's interactive, informative, shows continuity from D5, D6 to D7. It may have some issues with RAM but it makes the experience more transparent and less mysterious (i.e. the D6 "upgrade manager" screen, no offense it just feels a little confusing to me seems it looks like it wants to automate things for you but it doesn't really).

Since upgrading is such a stressful, nail-biting, time-sensitive, and at times a discouraging experience; I'm hoping I won't be of the population who encounters any open bugs in my upgrade period. I thank you for all your hard work and understand the paradox of trying to get an RC out but wanting to make this version the best that it can be. Just giving my perspective.

Well, just skimming dww's list in #0, it looks like there are number of API changes and even more UI changes, so that would be one reason why those issues can't be fixes in 7.1, etc.
Update manager is in a very unique position in that it is
A: integral to Drupal 7's ability to rock for non-technical users
B: of little interest to most developers
so it is very important to get it right (A) and it might not be that harmful to break the APIs/UIs (B). That, of course, is up to the core maintainers to decide.
I think it is obvious, though, from dww's list that if we do not want to break APIs/UIs we can go 3 routes, all of which are a really really bad idea for obvious reasons:
1. Remove update manager
2. Ship with a basically working, but very fragile Update manager
3. Delay the Drupal 7 release for a long time to get Update manager up to speed.

I really wish I had the experience and knowledge to work on fixing the Update Manager because I think it's a very important part of D7. However, I don't have the skills, so I just want to say, if you do have the abilities, please help! And thank you to everyone working on D7 for all your hard work.

Its a case of WTF is wrong when one cant connect.

In the case that I have attached screen shots, is the issue that this is and 'edge case' where my FTP is not into the drupal root and maybe requires an initial root option in the FTP details of which there is no place for. Kinda of a scratch ur head moment for a user, a message saying hey the credentials you supplied failed to connect to the FTP server failed because..

In the case I have provided im still not sure if its because of my non drupal root, FTP initial path is the cause or its something else. All the details are correct and I have a FTP server running which you can see via the transmit log in.

So I walk away kick the screen not knowing what's wrong my FTP server setup or something I have done wrong on the form?

Its just a case of letting the user know hey FTP access only works in these cases... As from the bug list there are some 'edge cases' that are not supported. Knowing these upfront is key, to this being usable?

@webchick: #605318: Add some garbage collection to the update manager is really critical. I guess if there is a D7 release before this issue get's fixed we will see daily ~5-10 new cases only for this issue. I cannot speak for the other issues, but people will start using DEV versions in the first days/weeks/months and they cannot upgraded after they have been installed/downloaded without manually killing update-cache folder. It took me a good time to figure out that stale DEV versions are the reason for the failed update...

@jucallme #10:
You are getting the error message "Cannot connect to FTP Server, check settings" because quite literally you cannot connect to the FTP server, not because the FTP root is not /.

The relevant section of code is includes/filetransfer/ftp.inc, namely

  public function connect() {
    $this->connection = ftp_connect($this->hostname, $this->port);

    if (!$this->connection) {
      throw new FileTransferException("Cannot connect to FTP Server, check settings");
    }

I'm wondering why you are getting the connection error... what happens if you were to set the server to 127.0.0.1 instead of localhost?

For the more experienced in this area:
Is it possible to add an 'path to drupal root' in the advanced ftp settings, or does this violate the 'jail' functionality?

@bfroehle

I have the following setup.
1 ubuntu box running web and ftp server
its got a few apache virtual servers for a bunch of dev domains

I use either localhost, 127.0.0.1 or one of the dev domains that are set to 127.0.0.1 in that ubuntu's etc/hosts file or even the ip address of the box
and i get same issue over and over.

I'm starting to think I'm now defiantly some sort of edge case.. Something must be screwed in my case. I have the same result on a dev D7 and the current beta 3 that i have been updating since beta 1.. version I run and test against.

+1 for removing this from the "critical" list.

dww's concerns are certainly valid and (thankfully) well-documented. But I, too, have been using the update manager as an end user and not come across the problems he ennumerates. On the other hand, the value of the update manager is *enormous*.

From a cost/benefit point of view, the update manager should stay in its current form, although the fixes should be a high priority for 7.1.

I am a new user, who is therefore completely unqualified to comment on whether the problems are critical or not, but reading this thread seems to have confirmed the concerns I expressed in http://drupal.org/node/955642, namely that experienced Drupal users have been able to ignore these problems so it has never got the attention it deserves. However, from the perspective of a new user, if my experience is reproduced by thousands of others who try out Drupal for the first time, it will kill their interest in the project stone dead. It's just too frightening if, every time I have to update a module, I get half-explained error messages with no log messages and no way to clean up. Since I was fairly committed to using Drupal 7, I learned to use drush so it's not a huge issue for me now, but it will be for many of the new users out there.

In the referenced ticket, I give plenty of details of the problems I encountered. I think my setup is likely to be typical of many, and I followed all instructions to the letter. If I did something stupid, I am happy to be enlightened, but if I didn't, then this definitely needs to be fixed before Drupal 7 is released.

Isn't Update Manager redundant with Plugin Manager that was supposed to make it into core?
#395472: Plugin Manager in Core: Part 1 (backend)

I am using Plugin Manager in D6 all the time, and it massively simplifies the whole module update and install experience. Instead of fixing this, how about getting that into D7?

I believe "plugin manager in core" mutated into the update manager. What's lacking is more robust error handling and a built-in module/theme browser. The latter can be provided by contrib. The former, yes we should work on. The issues mentioned in #0 are a good start for those.

Looking through the issues above, I agree that update manager needs some attention but I disagree that it's so critical that we need to pull update manager back out. Let's triage the outstanding issues and fix the ones that can't be fixed after we hit string freeze / hard API freeze. The rest we can fix as time permits, just like any other Drupal bug.

I guess we need to redefine some older rules for critical vesus major. From the bugs listed by dww I'm sure we will find many others, too that give many many users a feeling that drupal has been released as unready product. Maybe the wording must be someting like - we release if the major issues count drop to nearly 0 or less then 25 or if none of the issues gives users a feeling of an unfinished product. Or otherwise - do not use Drupal before minor version .5

Nope. We're not changing any rules here. There was over 12 months to clean up this chunk of code. dww didn't get any help. Unless this changes really fast, it's looking like this aspect of the code is going to be pretty rough until 7.2 or so.

Drupal will always be an unready product. More, show me any software that is finished. I link a lot http://cyrve.com/node/23 these days

We need folks to Get Real, Get Dirty, and Give up (on perfection). We are in the 'least of all evils' stage. Easy resolutions are unavailable. All software ships with bugs, and sometimes serious ones.

I'm willing to have a look at the code and see what I can do, expect me in IRC in the next 24 hours ( while I'm a coder I need help with patches, and I dont have a whole lot of exp with d7 ).

While I probably wont be of much use I'm happy to give as much time and effort as I can to this issue ( and the issues its related to ), I'm sure if even just a couple of others offered the same, most of the issues would be resolved rather quickly.

In addition I feel that this should
A: be in core
B: be as stable as possible
C: be as usable as possible

I think chx's point is valid here, its probably not finished - but its value is far greater than the cost of it sucking.

I think its generally to late to fix the major UX bugs, they have been identified long ago - and since no developer actually uses it, it will unlikely to be fixed given the technical difficulties we face solving these issues ( I hope contrib will solve this far better). So lets be realistic escalate 1 or 2 issues that are truly critical to fix, and close this issue.

This issues isn't critical, it's only good for visibility..
I thinks that Drupal 7 can be released perfectly without any fixes from this issue.

The update manager isn't prefect, but what we have now still is a great value for Drupal 7.

We should stop wasting time & energy (IOW close this issue) and focus on the issues listed in #0 instead. Get them shorted by importance and hit them hard one-by-one starting with the most important and moving to the next one(s). I cannot help with coding, but I sure as hell can beta test + help other newcomers like myself to also start testing and providing feedback.

IOW, I think this should definitely ship in D7 in whatever state it is, but till D7 final is out lets squash as many of these bugs as we can people! Go-go-go!

I agree with chx on this one. The Update Manager has some pretty serious problems, to be sure, but it is necessary for Drupal to advance in its usability for a general audience.

I would hope that the more serious bugs can be resolved in point releases if they can't be in the next 2 weeks.

Here. Let's do this.

I'm going to keep this as critical for now, though, because we do definitely need folks to focus on these over the next couple of weeks.

#605318: Add some garbage collection to the update manager seems like one of the most serious of these, but I haven't seen the others yet. It seems like without a fix for #605318: Add some garbage collection to the update manager, we can't have an Update Manager that downloads dev releases.

@bfroehle #12 and @jucallme #10, #13.

I've also had this error and played around with the code to see if I could figure out what's going on. It seems that ftp_connect is always trying to connect to localhost:21, instead of using the values entered in authorize.php.

As a test, I edited .../ftp.inc and put in and ip address string in ftp_connect in place of $hostname, and it connected fine.

But then it ran into another error, a permissions error this time. Apparently it was trying to create/delete/move (don't remember which) the ".." directory, which of course it can't do.

Other issues I've noticed include:
- not recognizing that my system has SSH capabilities and only offers me ftp.
- no way to cancel/return to the drupal config. page without retyping the address in the browser
- doesn't handle multi-site very well. i.e. always installs modules into /sites/all/modules instead of offering user a choice of which site to install in.

So yes, I agree as and end user this is definitely buggy and not ready for release.

@dww

#850846: Update will happily downgrade modules unless you are careful is missing from your list.

I've read alot posts and thought this one was as good as any to make a comment regarding the principle fixing "critical" bugs the right way vs descoping them, which is what I've seen from high level perspective just to get D7 shipped.

I want to begin by saying that I'm not a real Drupal contributer, not a module developer, not a core maintainer, etc. My skills are small modules, adding hooks and form alters. I'm learning, and D7 is adding levels of complexity I'm not ready for. That, coupled with descoped "critical" bugs will make things I do that much more difficult, and I believe, hurt D7's user adoption for the Drupal novices such as myself.

Regarding what @chx said:

Drupal will always be an unready product. More, show me any software that is finished. I link a lot http://cyrve.com/node/23 these days

Here's my view of where Drupal is trending:

• D5 : Windows 98
• D6 : Windows XP
• D7 : Windows Vista
• D8 : Windows 7

Launching Vista was a mistake for MSFT, it took them 2 and a half years to fix it with W7. And, to this day, there are still more computers using XP than Vista and W7 combined.

Yes, Drupal can be shipped with bugs, but consider how that affects user adoption and our standing as we compete with Joomla and Wordpress. My 2 cents.

Oops, accidentally untagged it.

@dww

Agree with you 100%. Another reason I can give why it is very important this works properly is that the Update Manager is going to be highlighted in reviews and it will be compared to how it works in especially WordPress. WP's update manager works very well and even allows you to update their core. Then if Drupal is not even able to update contributed modules correct we risk losing a lot of creditability that has been so hard won over the years.

Readers of these reviews and the new audience it is aimed at are not developers. They are users that expect it to work without surprises. In many cases they will not be able to fix any problems themselves, give up on Drupal and move to WP. Simple as that.

@jucallme #10, #13 and @ADrupalUser #28:

Can you please create separate issues for your problems so they don't get lost in the general discussion happening here.

In the case that I have attached screen shots, is the issue that this is and 'edge case' where my FTP is not into the drupal root and maybe requires an initial root option in the FTP details of which there is no place for.

I haven't seen an issue for this. (Although this might fall into the wont-fix category)

I've also had this error and played around with the code to see if I could figure out what's going on. It seems that ftp_connect is always trying to connect to localhost:21, instead of using the values entered in authorize.php.

As a test, I edited .../ftp.inc and put in and ip address string in ftp_connect in place of $hostname, and it connected fine.

I didn't see an issue for this. Please retest with the latest -dev or -beta3 and create a new issue if it is still a problem.

But then it ran into another error, a permissions error this time. Apparently it was trying to create/delete/move (don't remember which) the ".." directory, which of course it can't do.

This is #935036: File exists in FileTransferFTPExtension->createDirectoryJailed().

- not recognizing that my system has SSH capabilities and only offers me ftp.

I didn't see an issue for this. Drupal should directly test for the availability of the necessary PHP functions to implement the various file transfer methods --- so it's probably a problem with your PHP installation. What we should do is work on a page that lists all available file transfer methods and describe why certain ones aren't available, with links to documentation for how to add them.

- no way to cancel/return to the drupal config. page without retyping the address in the browser

I've seen several issues about the 'dead-end' nature of Update manager, so you can probably find an existing issue for this.

- doesn't handle multi-site very well. i.e. always installs modules into /sites/all/modules instead of offering user a choice of which site to install in.

I didn't see an issue for this.

@dww: You may be right that the current downsides outweigh the benefits, especially by comparison to WordPress's Update Manager. I haven't tried the Update Manager on many different server configurations.

I hope that the issues get fixed; ultimately, it will be the core committers' decision on whether the Update Manager is ready for inclusion into core.

I'm not taking update manager out of core at this point, unless Dries overrules me. We have two weeks to whip it into shape. Let's go.

Thanks dww for all your hard work. If you could offer direction on what pieces of code us non-ninja programmer and testers can help out with please let us know. I would love to see this issue resolved and solid in the final release. How can I help best?

I'll have a quick look over those two in the next hours.

+1 for keeping this in, downgrading the issue from critical, and giving a more descriptive error message when the process fails, that include possible fixes for an end user to try (mentioned in the forum topic linked below).

At least I'm assuming that the errors I talked about in http://drupal.org/node/970542 are related to this issue, since although it was the "install module" link, the sticking point was, at first a permissions issue, and then an (S)FTP issue. But I got around both. (I hadn't filed it as a bug because I assumed I was doing something wrong.)

I'm adding one more #977408: Subdirectory in "update-extraction" is not deleted before extraction of other versions that can also lead to security issues.

And another one #977414: Allow updating to a -dev or new major branch with Update manager that makes update manager a bit useless.

I've closed these duplicate.

Another patch for consideration
#961060: If a module is Not Supported, Drupal's warning links to available updates page, which says everything is OK
Small patch fixing a UI WTF for a semi-rare case.

This issue was quite effective at rallying the troops on Update module. Thank you so much to everyone who helped out!

However, RC1 is now imminent, so it's time to push this back to merely a "major" issue so we can clear out the critical queue again.

subscribing, and thank you everyone who pushed this forward!

I'm not sure where I should post this, but after updating Views module, I find myself on authorize.php page, with nowhere to go:

Update was completed successfully. Your site has been taken out of maintenance mode.
views

* Installed views successfully

Next steps

* Your modules have been downloaded and updated.
* Run database updates

The only option is to "Run database updates", which has already been done by this stage. Very confusing UI. Is this something already reported elsewhere? Or already fixed? I'm using RC2.

Oh! I'm sorry. I got the idea because of the website being put into 'offline' mode. I'll go find something else to update so that I can read the test more carefully next time.

Luckily Views module updates very regularly, so didn't have to wait long.

OK - the wording is "Perform updates with site in maintenance mode (strongly recommended)" which I took to mean database updates, not module updates, as I have never used maintenance mode to install a new module.

It's a shame that update.php doesn't have the same nice option of automatic maintenance mode. It just has the suggestion to "Put your site into maintenance mode" but no offer to do it for me like the last screen. I'm taken to admin/config/development/maintenance which is a dead end, and have to manually navigate back to update.php to continue.

Once done, I am given choices of Front page or Admin, but no reminder to go back online before I go.

@dww

I agree with Scott here. The key to the confusion is the "Update was completed successfully. Your site has been taken out of maintenance mode." part.

It raises many questions such as:
If the database hasn't been updated, how can it then be a completed success?
Why is the site taken out of maintenance mode before the database has been updated?

At least for me it has always been my understanding that putting a site in maintenance mode is prevent it from running code that is not complete (i.e. its temporarily deleted due to that a module is being updated) and to be able to update the database with the needed changes before the new code is used.

The database update should run automatically and then the success page should summarise all the steps taken, the action performed and information about the results for each module updated. Best would be if these reports where outputted on the screen as they happen, not just after everything is done.

Can we mark this as closed?