I finally had a chance to chat with Dries about update_status in core and the state of things. Mostly he's happy (see http://drupal.org/node/153968#comment-266276 for some of the news).

However, he's very opposed to the change I made over at http://drupal.org/node/146564 to deal with the fact that a slow d.o would sometimes hang update_status's cron/fetch runs. In talking about it with him some more, and brainstorming and poking around with alternatives afterwards, I think I've got a much better solution...

The client should always fetch from here:

http://updates.drupal.org/release-history/[project-name]/[core-version]

By default, project_release.module will have its existing menu callback registered there, to serve the appropriate .xml file based on the path args. However, if we decide this is too slow, we can just create an actual "release-history" directory off the updates.d.o webroot, stick in a .htaccess to point to a thin php wrapper script, and serve the .xml files without a full Drupal bootstrap that way. This gives us much more flexibility in the long run. It'll also means we can go back to the original plans for saving usage data (http://drupal.org/node/128827) without having to scrape apache logs, which is very nice. The code in the client to optionally append the anonymous site key to the URL args is still there, and will work no matter what we do.

The inflexiblity of the current updates.d.o/files/release-history/... approach has bothered me ever since I committed it, even though it was a big performance win. Going to this new approach will be much better for 5.x-2.0 and for the update.module in D6 core. We'll have plenty of flexiblity at the server infrastructure side to handle the load however we see fit, and still have a relatively easy task of saving usage statistics, etc.

I'm not bothering to post patches, but I wanted to write up the plan and motivation in an issue before I committed, so we have a record of it.

Cheers,
-Derek

Comments

dww’s picture

Status: Active » Fixed

http://drupal.org/cvs?commit=71718
(new menu path from project_release, now on d.o.)

http://drupal.org/cvs?commit=71719
(D5 update_status.module fetches from the flexible path.)

http://drupal.org/cvs?commit=71720
(D6 update.module in my sandbox fetches from the flexible path.)

merlinofchaos’s picture

I don't get it? Why the opposition to compiling stats from weblogs? Weblogs provide very powerful stat mechanisms. It's not difficult write awstats modules that compile these stats...so I feel like this opposition comes for the sake of opposing weblogs than any real deficiency in using them.

dww’s picture

a) Because if we go this route, we are free to choose if we want to just use weblogs + awstats, or a thin php wrapper to record and serve, or a full blown drupal menu path. By hard coding it as before, we were stuck and would have no way to change our minds ever again if we didn't like the weblogs + awstats approach.

b) Because Dries said the scraping would be a pain in the ass, and he'd rather see us spend our time on other things...

(a) is compelling to me. (b) not so much, but then again, time is the scarce resource among us...

hass’s picture

subscribe

BioALIEN’s picture

Subscribing also. Thanks for the clear update.

dww’s picture

I was stuck in an airport this afternoon for a little while, so I wrote a thin PHP wrapper to serve the history files:

http://drupal.org/cvs?commit=71890

Installed on d.o (and therefore, updates.d.o, what all the modern clients and the core patch uses to fetch from. That should help a lot.

Anonymous’s picture

Status: Fixed » Closed (fixed)