Jump to:
| Project: | Download Count |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Issue Summary
Sorry I can't be more specific, but I can only describe the general problem I'm having as a series of points:
1. A couple weeks ago I first noticed errors showing up in Drupal that concerned DL Count, but I failed to record them. I recall that they concerned some missing tables for statistics. I simply turned the module off because I was in a rush, and planned to bring up the errors again later and record them, but they have not reappeared.
2. The other day I turned the module back on, and my admin informed me that I was maxing out the CPU on our server with SQL calls. He provided a log showing a call occurring continuously that takes 50 seconds to execute. Log excerpt attached.
3. I received some interesting error messages, but accidentally deleted them all because I had turned on a module that allows log-clearing and it has a really terrible design decision (replacing the log filter's Reset button with Clear). I have since turned it back off, for all the good that does me now. One such error concerned the page "individual_file_count" being missing.
4. I also receive the error: "warning: file_get_contents(sites/default/modules/download_count/jquery.sparkline.min.js): failed to open stream: No such file or directory in [...]/includes/locale.inc on line 1714." each time I turn the module on.
Sorry if these errors are not all the result of one issue, but I really have no way of knowing what is causing what. Any suggestions at all?
(Using Drupal 6.20 and latest dev version of download_count, although these issues started around the time I upgraded to an earlier 2.x-dev version.)
| Attachment | Size |
|---|---|
| sql log excerpt.txt | 900 bytes |
Comments
#1
1. Can't really do anything here unless it happens again and you can post the error messages.
2. that query comes from the download_count_files default view. But the
WHERE dc.fid = files_download_count.fid) > 'S'is odd-- there should be no "S" there (it should be zero). did you alter that view at all?3. I don't create any reference to "individual_file_count" and a file search doesn't show that string anywhere in my code. I do provide "individual_file_downloads" which is handled by the default view mentioned in #2. if that's what you meant, then it could also be related to #2.
4. I think I committed the fix for this.
If you can, it would be good if you disabled the modules, uninstalled them, downloaded a fresh dev (once the fix appears in the tarball), and try it again.
Thanks for testing and reporting back your findings! There's only so many ways i can test this module myself and almost infinite ways to use it...
#2
1. Yeah, I suppose not. I only mentioned it in case it rang a bell for you and helped explain the below issues.
2. I don't believe I've modified anything (not sure what I would have to change for that to happen), but my admin tells me that 'S' is a placeholder value used by mySQL in place of the actual number(s) that were used in multiple instances of that SQL call. In other words, it mashed multiple "duplicate" calls into one line in the log by replacing the 0 with S. Anyway, just to be clarify, it's when we click on the Individual File Downloads page that we're seeing the server spike at 100% CPU for almost a minute while we wait for the page to be rendered. Sometimes it appears immediately, but subsequent pages (2, 3...) take that long, so I guess there's some caching being done.
3. Yes, sorry, I meant "individual_file_downloads". I now see many instances of the "page missing" warning in my log, referencing that page name -- from the time period that I just had the module turned *off*. Could that simply be from web crawlers that are asking for a page they saw once before? But the error vanishes while the module is on, so I'm not too concerned about this one.
4. I could not get the error to appear with the newest dev build, so I think it has indeed been fixed.
So, in hindsight it looks like issue #4 might be unconnected to whatever is causing the server to overload. I hope the d/l count database isn't farked... I'm also seeing massive entry duplication where a single download is recorded 8 or 9 times, so I don't know what that's about (I should probably make that a separate report, but I felt like I should mention it here in case it sounds significant).
#3
I'm no mysql guru, but if it were a placeholder, i would think it would have to be %s. afaik, 'S' is going to be seen as a string that query but like I said I'm no expert. You can simply disable that view and try to make one of your own and see if that also produces a similar server load.
As for the duplication-- if someone clicks the download link 8 times then you'll get 8 entries. All the module does is simply intercept the url and record when it has been clicked it doesn't actually do any flood control (on my list for a future version) or download verification of any kind. Is this in the download_count or download_count_statistics table?
#4
Sorry, for some reason I was unaware that the module creates views that I can disable (although disabling both "Individual File Downloads" and "File Downloads" also makes the main "Download Count" page go away, which I don't understand). The Indiv. page is disabled now, so I'll let things run for a while to see what happens. Btw, I checked the view settings for the page before disabling it, and it was indeed using "> 0" (my admin assures me that 'S' is a stand-in token for logging purposes). I had already asked the admin to run the SQL call himself (with "> 0"), and it did take about 50 seconds to return. So although I don't know what was causing that, if taking the Indiv. page away makes the CPU stop spiking, I'll be content.
#5
Maybe this title is a little better, I don't know. By the way, I succeeded a couple times in reproducing the high CPU load when building a custom view. One thing I did to cause the load, after building essentially the same view as the Indiv. page, was to turn on the sorting that the Indiv. page uses. Without any sorting, the page retrieved in milliseconds. Turning it on led to the 50 second SQL calls. This was with no filter on, although adding the "> 0" filter made no difference. Obviously this is an issue related particularly to my setup, so this may not help you. Just curious, what would be a normal size for the download_count table? Mine is 15,000 records and 1.2MB.
#6
Thanks for tracking that down-- indeed helpful. Now I can concentrate on finding some info on tuning that sort.
Not sure what a 'normal' size would be-- it all depends on what type of site you have and your user base. My main sites are on intranets and tend to be on the small side traffic wise (my biggest one is only 1000 records). I'll be interested to hear what size others have myself.
#7
"Not sure what a 'normal' size would be-- it all depends on what type of site you have and your user base." Right, I would figure that some site admins get far more downloads than I do, so I can't imagine that it takes that long to sort for everyone, but it might be helpful to find out how many records other people are working with in their dc table.
#8
I'm running into performance issues too. The select sql query is taking 4 seconds, and the pager count query is also taking 4 seconds. I have a download count field, as well as download count sort criteria. I also have a large number of download count entries - 4500. I'm still using 2.0-beta1.