As per http://drupal.org/node/128827 -- we desperately want to start collecting stats at d.o about actual installed module usage. That issue is about the server-side code in the project_release.module (or a new project_usage.module, most likely) to collect, process, and store these stats. This issue is about client-side code to make this possible. As far as I can tell based on the discussions of the server code, all we need is:

  1. Some kind of setting UI to opt-out or in. This could either be site-wide, or another checkbox on the settings page in front of each project (with some check/uncheck all jQuery goodness). Doing it per project is kind of nice, since a) you might be happy to report you're some things but not others, and for projects not hosted at d.o. OTOH, it's more complicated, and might clutter the settings page a bit.
  2. Pending the admin setting, if we're going to allow the stats to be collected, when we generate the URL to fetch the XML history for a specific project, we add a sitekey query to the URL. The sitekey value would be something like md5hash($site_name . variable_get('drupal_private_key', '')). See #128827 for background on this.

I think that's all we need. If the server code sees this sitekey in the URL's query args, it can record the fact that this (anonymous) site is using a particular Drupal core version (5.x, 6.x, etc) of a given project. After that, it's up to drewish's SoC stuff for project quality metrics to digest and display the data in useful ways...

But, I'm nearly positive that these changes to the client are all we're ever going to need to support this functionality. Therefore, I believe we could work on this now, before the 5.x-2.0-beta2 release (and eventual 5.x-2.0) so that the clients in the wild will already support it, and then we can phase in the server side stuff whenever it's ready. The existing server will just silently ignore the extra query in the URL, so there's no harm there...

The only minor gotcha is the case of a custom URL specified in a .info file that points to a site without clean URLs. So, we just have to parse the URL and check for ? and if it's there, append the query with &, otherwise ?...

CommentFileSizeAuthor
#8 update_status_usage_stats.patch.txt6.42 KBdww

Comments

nancydru’s picture

nancydru
she/her/hers
Primary language English
Location Boston
commented

If you don't mind, here's another perspective: I will not use update_status on a production site because I largely follow the "if it aint' broke, don't fix it" philosophy and I don't want to waste my monthly bandwidth checking for updates that I won't install several times a day. I only check for new stuff on my test sites, which are almost all on my PC. So such a stats collector is not going to be seeing my production sites, and is only going to be able to see my test sites when I do things manually (such as trigger Cron). I doubt I'm alone in this method, so I'm not sure such statistics are going to be very accurate.

dww’s picture

dww
we/he/they
commented

a) This will be in core. So, you won't have a choice once D6 comes out (except to opt-out of the statistics gathering).

b) update_status is important for quickly warning you about security updates. You should run it, even on your production servers.

c) That said, there should be a way to put update_status in a mode to only mark it as an error when there are security updates. See http://drupal.org/node/153757

d) The statistics won't be 100% perfect, but they'll be vast improvement over what we have now (which is only sketchy data, and only if you enable the drupal.module, which also implies distributed authentication, etc).

nancydru’s picture

nancydru
she/her/hers
Primary language English
Location Boston
commented

A) I understand it will be in core. Unless it is a required module, I will disable it on the production sites. I don't have the income to support the bandwidth it will use.

B) Unless it emails me, I will find out much more quickly on my test sites. I simply do not go to the production sites and check for things like this unless I have some indication there is a problem - my clients don't pay for me to do so. Nor are they likely to check it either.

C) covered in A & B.

D) Well, I have to admit that I'd like to know who is using my modules. But I have no desire to cause them the expense to report it to me every day.

dww’s picture

dww
we/he/they
commented

At Dries's request, this isn't going to be a separate module at all. It's just going to be part of system.module, and therefore, required.

I'm not sure what you mean by the "expense" of the bandwidth. Are you honestly concerned about once a day fetching at most a few kilobytes of .xml files? If so, you need a new hosting plan. ;)

alpritt’s picture

alpritt commented

"I simply do not go to the production sites and check for things like this unless I have some indication there is a problem - my clients don't pay for me to do so. Nor are they likely to check it either."

Which is why it should be on. Security bugs might not reveal themselves as a problem until your site is compromised. Update Status is a good indication that there is a problem (in many cases, the only indication if you don't go looking).

The fact that you say you would disable it, kind of convinces me that it should be required. I cannot see that bandwidth is a real issue, but some may consider it to be, shut it off and then neglect to maintain their sites. Just like many people do with Windows auto-update, for example, for no good reason.

If this CAN be turned off, I would make it hard to do so.

+1 for part of system.module

Re: B) Can it not email you using the new logging functionality?

dww’s picture

dww
we/he/they
commented

Re: email, see http://drupal.org/node/124715. And no, it's not logging anything using watchdog(), so the new logging interface (which is only in D6, anyway) won't help you.

That said, this *really* isn't the right place for this debate. Let's leave this poor issue alone to talk about the usage stuff. ;) Any more comments on if it should be required or optional once it moves into core belong over at http://drupal.org/node/153968.

Thanks,
-Derek

dww’s picture

dww
we/he/they
commented
Assigned: Unassigned » dww

i'm actively working on a patch now. stay tuned.

dww’s picture

dww
we/he/they
commented
Status: Active » Needs review
StatusFileSize
new update_status_usage_stats.patch.txt6.42 KB

I had to slightly refactor the refactoring of update_status_get_projects(), update_status_process_project_info() and update_status_refresh() from http://drupal.org/node/152387 since I forgot we'd want to send the exact version of the installed module in the query args, which requires invoking hook_version_alter() to be accurate in the cvs_deploy case. Luckily, this turned out to be much simpler than I feared, and even cleaned up one annoying hack from before, which is that we were saving the filename into our projects array when we only needed that for hook_version_alter().

So, the attached patch adds:
1) New setting on the settings tab
2) hook_version_alter() invoked in update_status_get_projects(), not update_status_process_project_info()
3) Logic to check the setting when generating the fetch URL

Tested and works nicely. Of course, there's no server code to deal with this yet, but it's silently ignored for now if you include the sitekey + version info...

dww’s picture

dww
we/he/they
commented
Status: Needs review » Fixed

Committed to HEAD (http://drupal.org/cvs?commit=71437). If anyone objects, feel free to re-open.

Anonymous’s picture

(not verified) commented
Status: Fixed » Closed (fixed)