Posted by fgm on October 29, 2008 at 8:48am
| Project: | Theme developer |
| Version: | 7.x-1.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | reviewed & tested by the community |
Issue Summary
During its operation, devel_themer creates files in /tmp, but then never removes them, causing /tmp to fill over time, possibly leading to server crash.
These files are generated in devel_themer_exit(). There's a comment about how "Writing to temp dir means [...] cleanup is free.", but this is not really the case, as /tmp is not necessarily cleaned every few hours. It would probably be useful to add a cleanup step, probably as devel_themer_cron() to avoid putting yet more code in normal page runs. If you think this is a good idea, I can write it.
Comments
#1
this module is intended to be used by a single person of an elevated role for a short duration. you would never turn it on for all users since the pages break and show checkboxes and such.
this is an extremely unlikely error condition. did you actually experience it or are you speculating?
#2
No, this is not speculation. I found out about the problem the hard way while developing a theme because the server constantly crashed on me and
dfpointed me to the problem.A simple workaround for the problem is to set drupal to use a tmpdir outside the default
/tmp, but then of course, there is no automatic cleanup since other directories are not normally emptied periodically, and someone/somecode has to care about it. For now, I've set up a non-drupal cron to clean these files, but I suppose not all users have the same latitude.#3
#4
I had the same issue /tmp was allocated 100MB - which filled up very quickly. I believe some form of clean-up should be part of Devel...
for me I only found out about the issue after getting this error:
user warning: Got error 122 from storage engine query: SELECT DISTINCT b.* FROM blocks b LEFT JOIN blocks_roles r ON b.module = r.module AND b.delta = r.delta WHERE b.theme = 'fourseasons' AND b.status = 1 AND (r.rid IN (2) OR r.rid IS NULL) ORDER BY b.region, b.weight, b.module in /home/sitback/apache/vhosts/tixnetwork/web/modules/block/block.module on line 407.
Then ran df -h in console and saw I had used 100% of my disk quota:
Filesystem Size Used Avail Use% Mounted on
/dev/vzfs 100MB 100MB 0 100% /tmp
If you still disagree that's fine... but I just thought I would bring up the issue once more.
Cheers mate
- Chris
#5
This is definitely a problem in the real world. This module needs to clean up after itself. grrr.
fgm, if you would develop a patch for this, it would be much appreciated.
#6
OK, I would take a patch to clean this during cron.
#7
OK, here is one possible implementation.
#8
Patch variant: done using drupal API instead of plain PHP.
#9
Committed after a qick rename to devel_themer_cron().
#10
I just received this message from my ISP; pretty surprised by it really.
We found the module "devel" used by your drupal generates too many temporary files, puts them into the system folder /tmp and didn't remove the files late. It make /tmp out of free space and stops some services. So we had to block the module /devel in order to resolve the problem.
--
--
Kind regards,
So yes, it's definitely a problem in the real world. If the patch works then perhaps I'll refer them to this page and explain that this has now been resolved.
#11
Automatically closed -- issue fixed for 2 weeks with no activity.
#12
Just want to say thanks to all those who helped get this issue fixed, as well as moshe for committing it. I've been doing a lot of development on one Drupal site in particular, and this bug caused the disk on the VPS to continue to fill up, until eventually the disk was completely full, with a whole 2GB of devel_themer_* files left over in /tmp.
So, thanks again! :)
#13
Hi, I experienced this problem recently with:
We are just getting started with Drupal and have a working installation on our development server. Drupal had been working properly, though we were starting to see slower page loads (or longer delays), and one day the following warnings appeared:
/admin/content/comment (Comments)
user warning: Incorrect key file for table '/tmp/#xxx_xxxx_x.MYI'; try to repair it query: SELECT c.subject, c.nid, c.cid, c.comment, c.timestamp, c.status, c.name, c.homepage, u.name AS registered_name, u.uid, n.title as node_title FROM comments c INNER JOIN users u ON u.uid = c.uid INNER JOIN node n ON n.nid = c.nid WHERE c.status = 0 ORDER BY timestamp DESC LIMIT 0, 50 in /home/xxxxxxxxx/public_html/modules/comment/comment.admin.inc on line 60./admin/reports/dblog (Recent log entries)
user warning: Incorrect key file for table '/tmp/#xxx_xxxx_x.MYI'; try to repair it query: SELECT w.wid, w.uid, w.severity, w.type, w.timestamp, w.message, w.variables, w.link, u.name FROM watchdog w INNER JOIN users u ON w.uid = u.uid ORDER BY w.wid DESC LIMIT 0, 50 in /home/xxxxxxxxx/public_html/modules/dblog/dblog.admin.inc on line 71.Core and modules are current as of today, Monday, August 10, 2009. cron run manually results in no warnings or errors.
http://drupal.org/node/335529
looked similar to my problem, so I logged in as root and edited /etc/my.cnf as follows:
Before
[mysqld]skip-bdb
skip-innodb
log-slow-queries
set-variable = max_connections=200
set-variable = table_cache=256
set-variable = max_delayed_threads=20
set-variable = max_tmp_tables=32
set-variable = query_cache_type=1
set-variable = query_cache_size=4M
set-variable = thread_cache_size=4
After
[mysqld]skip-bdb
skip-innodb
log-slow-queries
set-variable = max_connections=512
set-variable = table_cache=256
set-variable = max_delayed_threads=20
set-variable = max_tmp_tables=32
set-variable = query_cache_type=1
set-variable = query_cache_size=16M
set-variable = thread_cache_size=4
(changing max_connections and query_cache_size) but when I revisited the pages above, both warnings remained, so I restored my.cnf to its original state.
I successfully repaired all tables with phpMyAdmin as suggested in
http://drupal.org/node/49851
but when I revisited the pages above, both warnings remained.
I looked in /tmp (should probably have done this earlier) and found (partial listing):
total 100Mdrwxrwxrwt 2 root root 2.0K Jun 24 15:29 .ICE-unix/
drwxrwxrwt 2 root root 2.0K Jun 24 15:29 .font-unix/
-rw------- 1 cpanel cpanel 0 Jun 24 15:29 .ftpquota
-rw-rw-r-- 1 nobody nobody 93 Aug 4 14:36 .htaccess
-rw-r--r-- 1 nobody nobody 1.2M Aug 5 10:45 devel_themer_11729584a79c53e6d3aa
-rw-r--r-- 1 nobody nobody 275K Aug 5 09:22 devel_themer_83911994a79b1b58576c
-rw-r--r-- 1 nobody nobody 196K Aug 4 14:32 devel_themer_118266764a78a8f20bbbf
-rw-r--r-- 1 nobody nobody 163K Aug 5 08:31 devel_themer_249041484a79a5b9eb65b
-rw-r--r-- 1 nobody nobody 503K Aug 4 14:41 devel_themer_255094154a78ab10da586
-rw-r--r-- 1 nobody nobody 314K Aug 4 14:37 devel_themer_355892544a78aa0fe6190
-rw-r--r-- 1 nobody nobody 577K Aug 4 17:06 devel_themer_356817214a78cd09857a6
-rw-r--r-- 1 nobody nobody 162K Aug 5 08:20 devel_themer_410103084a79a34ca5304
-rw-r--r-- 1 nobody nobody 742K Aug 5 09:02 devel_themer_476252344a79ad034dcef
-rw-r--r-- 1 nobody nobody 107K Aug 4 17:05 devel_themer_542774014a78ccb75fe07
-rw-r--r-- 1 nobody nobody 90K Aug 4 14:20 devel_themer_637473644a78a60e1ccff
-rw-r--r-- 1 nobody nobody 64K Aug 4 15:03 devel_themer_649372804a78b03b9eba0
-rw-r--r-- 1 nobody nobody 86K Aug 5 09:16 devel_themer_673455084a79b06aaeb2a
-rw-r--r-- 1 nobody nobody 714K Aug 4 15:44 devel_themer_708947484a78b9e339f7c
-rw-r--r-- 1 nobody nobody 97K Aug 4 16:43 devel_themer_730306514a78c79e7d123
-rw-r--r-- 1 nobody nobody 108K Aug 4 17:05 devel_themer_811487624a78ccacde882
-rw-r--r-- 1 nobody nobody 365K Aug 4 14:40 devel_themer_814653554a78aaeae5f51
-rw-r--r-- 1 nobody nobody 122K Aug 4 16:47 devel_themer_815891324a78c87405004
-rw-r--r-- 1 nobody nobody 185K Aug 5 08:48 devel_themer_845756334a79a9e8462a7
-rw-r--r-- 1 nobody nobody 634K Aug 4 15:30 devel_themer_899653714a78b683b67d8
-rw-r--r-- 1 nobody nobody 329K Aug 4 16:58 devel_themer_948502674a78cb2df3777
-rw-r--r-- 1 nobody nobody 555K Aug 4 15:01 devel_themer_1163258804a78afa454bab
-rw-r--r-- 1 nobody nobody 0 Aug 5 11:20 devel_themer_1225689024a79cd7eac0a2
-rw-r--r-- 1 nobody nobody 98K Aug 5 10:53 devel_themer_1238082324a79c72a06269
-rw-r--r-- 1 nobody nobody 145K Aug 4 16:58 devel_themer_1284649344a78cb1b185cc
-rw-r--r-- 1 nobody nobody 388K Aug 5 09:20 devel_themer_1329694574a79b142a381f
-rw-r--r-- 1 nobody nobody 369K Aug 4 15:48 devel_themer_1337281164a78bace44709
-rw-r--r-- 1 nobody nobody 366K Aug 4 14:41 devel_themer_1381708604a78ab1a05649
-rw-r--r-- 1 nobody nobody 614K Aug 5 10:47 devel_themer_1399904954a79c59c5767b
-rw-r--r-- 1 nobody nobody 127K Aug 4 16:51 devel_themer_1517640504a78c96c6045c
-rw-r--r-- 1 nobody nobody 141K Aug 4 16:57 devel_themer_2010505794a78cafc6f741
-rw-r--r-- 1 nobody nobody 0 Aug 5 11:20 devel_themer_2060511884a79cd6316e1a
-rw-r--r-- 1 nobody nobody 338K Aug 5 09:03 devel_themer_2165834894a79ad34d1f6e
-rw-r--r-- 1 nobody nobody 370K Aug 4 14:59 devel_themer_2219599194a78af2de0765
-rw-r--r-- 1 nobody nobody 186K Aug 5 10:33 devel_themer_2234913964a79c2504ec94
-rw-r--r-- 1 nobody nobody 94K Aug 5 09:16 devel_themer_2344251814a79b0702bc31
-rw-r--r-- 1 nobody nobody 0 Aug 5 11:20 devel_themer_2368441784a79cd87a9fd9
-rw-r--r-- 1 nobody nobody 555K Aug 4 15:02 devel_themer_2397446364a78b00d2a7e7
-rw-r--r-- 1 nobody nobody 0 Aug 5 11:21 devel_themer_2421305944a79cd8e8d6ef
-rw-r--r-- 1 nobody nobody 1.2M Aug 5 10:44 devel_themer_2518023734a79c5089b791
-rw-r--r-- 1 nobody nobody 275K Aug 5 09:23 devel_themer_2560055184a79b1f772e8a
-rw-r--r-- 1 nobody nobody 448K Aug 4 14:34 devel_themer_2703681894a78a96574483
-rw-r--r-- 1 nobody nobody 109K Aug 5 08:28 devel_themer_2772740734a79a52bb820e
Based on information I found in
https://support.mayfirst.org/ticket/2159
and at
http://www.debian-administration.org/users/Steve/weblog/176
I executed
find /tmp -name 'devel_themer_*' -and -ctime +3 -and -type f -print | xargs rmas root and /tmp went from 100% used to 1% used. I refresh /admin/content/comment and /admin/reports/dblog -- the warnings are gone! Problem solved (for now)!
I turned off all the Devel modules to prevent the problem from recurring. However, I still have a few questions:
Thank you!
#14
I've been experiencing this same issue as well. The /tmp folder is filling up with themer files. But I'm using 6.x-1.16 (should I start a separate thread?). It's happening in two cases, separate installs on separate boxes. My strategy has been to simply delete the tmp files, about 50MB or so worth perhaps, but haven't tried any patches yet. Is there a production-safe stable patch or solution yet by any chance? Thanks for any help.
#15
Temporary fix, but if you can edit your crontab file (crontab -e from shell, or via cpanel/similar), add the following line:
45 * * * * /bin/rm -f /tmp/devel_themer_*Will clear out devel's cache files every 45 minutes.
#16
I checked the patch file (from earlier in this thread) and my version of devel already has that code in it. I ran cron manually, but the devel files in /tmp were not deleted. Perhaps it's because I am running multiple sites from one copy of Drupal. The devel files are in /tmp one level below the root, but perhaps the devel_themer_cleanup function (run by devel_themer_cron) is looking in mysiteroot/sites/mytestsite.com/tmp.
Or maybe the devel_themer_cron is not being run. Is there something I need to do to get it to run when I run cron for mytestsite.com?
#17
Nudge re: my previous post... any thoughts???
#18
I have also a flooded tmp dir.
latest dev version.
#19
@garethsprice #15: Actually, this will clean them every 60 minutes, 45 minutes after the hour, but that's normal use. I'd even suggest something more like
0-55/15 * * * * /usr/bin/wget -q -O - http://(your site)/cron.php > /dev/null
Which will clean every 10 minutes using devel's hook_cron implementation.
#20
Here is a quick n dirty patch to resolve this in the current version
#21
#22
gouaille:I have applied your patch but I don't think it's doing anything. Each time I refresh the page, I get devel_themer files in my windows->Temp folder that fill up the folder. Is that normal? On D7 here...
#23
Actually that worked on a linux based machine. Didn't test it at all on a windows but you should take a look at the path and the slashes used to find files.
#24
I have tested the last patch,and it is working,but this devel_themer_cleanup() should be called while the module is disabling too.
Here is a patch for devel_themer.install.
#25
Just had to clean nearly 3 gigs of devel_themer files from /tmp dating back 5 months. I have the latest dev release which is Feb 25, 2011.
#26
Ported to latest version
#27
How about this instead?
#28
I've tested your patch and the files created by devel_themer older than DRUPAL_MAXIMUM_TEMP_FILE_AGE are now deleted when cron is run.
It's strange that the file module defines FILE_STATUS_PERMANENT but doesn't define FILE_STATUS_TEMPORARY. Should this be reported as a bug?
#29
Kenneth: Did you test the patch in #26 or #27? The patch in #27 removes the cron hook so I think you tested #26.
#30
I've tested #27.
+ $file->status = 0;+ file_save($file);
setting $file->status = 0 will make this file a temporary file. Everytime cron runs, it will delete temporary files older than DRUPAL_MAXIMUM_TEMP_FILE_AGE = 21600s = 6h. So it's not necessary to implement a cron hook to delete temporary files. Drupal will take care of this automatically.
#31
I was suffering the same and by looking at the code I've found that file_save_data is hardcoded to make files permanent, so no "free cleanup" by Drupal will be done.
I solved it by setting the file object status to 0 and resaving and everything works. It makes it a bit slower I suppose, but not a problem in a devel module, IMO.
I was going to post an issue, but seems that you got it fixed before. This is what happens if you don't search the issue queue first :S
Anyway, posting the patch just to confirm that #27 & #30 are the correct solutions for me too. As kenneth says, there's no need to cron hook any longer. I also suppose that cleanup related code can be safely removed.
My 2c.