cache_form is never cleared on some sites

I am still seeing cache_form getting huge in D7, and my cron runs fine!

A search shows up tons of issues: http://drupal.org/search/site/cache_form%20table

PS: hmm this may have been fixed here:#227228: cache_clear_all and cache_get fail to clear caches when Minimum cache lifetime is on
Still, I am not sure why my cache_form is so big.

Just wanted to let you know that first I thouht that my cache_form is also not cleared. And it is not really cleared when it should in 5 hours after creation. You can see that in the expire table in phpMyAdmin.

Actually if you don't understand unix timestand, i.e. time in the following format - 1355326036
you can convert it at http://www.unixtimestamp.com to see human readable time.

So as I said it's not cleared exactly at this time even if your cron configured correctly. Then I stopped truncating this table manually and tried to wait a little bit longer.

In fact the next day after that the cache_form was completely cleared automatically.

I suppose it somehow depends on the minimum and maximum cache lifetime set at the Performance page in your site. My minimum was 12 hours and maximum 1 day. So it might take 1 day for the cache to be cleared instead of 5 hours.

Or it may also depend on the timezone set on you Drupal site which could be different from the standard Unix Timezone. I'm not totally sure about that. These are just my thoughts.

Anyway the cash is constantly cleared but with some delay. It doesn't worry my anymore ;)

My cache_form get filled up after using search when you are logged in and adding up to 1.5 MB on every unique search.

Hope it will help to save some nerves for those who as I was desperate to find any answer for several days why the cache_form is not getting cleared automatically ;)

Our cache_form table is now HUGE. More that 5 Gb (not Mb)!

Our cache min and max are 15 min in the performance page.

Using Drupal 7.16.

Just to confirm, cache_form not cleared with cron.php or UI. I am in process of completing a 6.x to 7.x upgrade and preview stopped working. Was finally going to export the database back to dev server since I couldn't find the error after much squirreling around, believing it was a server migration/upgrade issue. Had to kill mysqldump as the exported database over 20G and still growing! That's in the space of 72 hrs. from the upgrade database, D6 of 1.6G.

Had to kill the mysqldump process and used myphpadmin to empty cache_form. This is a critical bug.

#2 ILLIA will report back after some time for future behavior. Good description, thx.

Update: 12 hrs. later it appears now cache_form is being cleared out after an empty and reset of cache settings via the UI. Looks like an initialization problem to me.

Possible Mollom issue and not core? I looked into the Mollom source and it caches all protected forms, "at once" and I don't see a really good flush. I haven't done formal debugging on this to see since the caching of forms, I don't see an easy fix to use Mollom with javascript/ajax/jquery forms.

This is not related to Mollom. For us it did happen when cron was not able to complete its tasks.

Probably true. The massive, never ending growing form cache was upon initialization, but I had no cron time out, it ran successfully, no errors. Mollom seems to be consistent in just loading up the form cache to GB but not the same behavior.

I've got a real problem. cache_form just grows and grows. It's weird because the timestamps of created and expire are within a 6 hr window of being cleared, yet it just grows by 1.2GB every 24 hours. Beyond writing up a separate script to empty the cache_form and put it on a cron, I just cannot find out why this is happening.

My site performance is a disaster because of this. Eventually preview stops working unless i manually empty cache_form via myPHPadmin.

@RobertOak Is cron still running? In our case, cron was blocked and failed to run.

@xmacinfo

Hey, first, thx for the help.

That's the weird thing, I have no error messages on cron.php manually, externally and set up as a cron job via Cpanel (system is VPS, CentOS 6).

How did you figure that out? Do you have any suggestions on how to debug cron.php further?

I sure looks like none of the caches are cleared ivia myPHPadmin and I've changed permissions on cron.php to 777 even yet I get all sorts of other messages, clears out the spam identified by Akismet, on manual, updates the aggregator, yet the scheduled runs, Akismet spam is not being cleared out when scheduled, although aggregator is updating.

I should be seeing all cache tables cleared out on either update.php or cron.php (with key) no matter what the expire timestamp is correct?

I'm also wondering even about funky Drupal "secured permissions" causing me grief, as recommended here:

http://drupal.org/node/244924

The site is in production, but I am about to install the devel module, the performance module, but I cannot turn on PHP warnings on a live site and don't have the resources to run a sandbox on the same server.

Update: the cron in Drupal is using _SERVER['SCRIPT_FILENAME'] and that is picking up the wrong path.

Now the issue is why is this system variable wrong? _SERVER['SCRIPT_FILENAME']

I'm in a VPS and the site is functioning which means system variables are correct except this one?

I set up cron.php as a job to execute and it is, other modules, processes which rely on cron.php are all working. I set cron.php to execute and also looked over other modules to make sure they are registering their forms and didn't see any not. I also set cron.php to run every hour. It's like on cache_form the only thing happening is expire timestamps get increased or something instead of flushed.

Your problem is interesting! Looks like it's not the same error we are having, but still, it is related to cron.

Can you try out this Elysia cron (https://drupal.org/project/elysia_cron) and see if that can help?

Still working on it, but so far the cache_form is large, but I'm not seeing the growth previously. In addition to configuring cron jobs and I ended up using wget and if this doesn't work below, I'll check it out.

=============

For now:

I looked over suspect 3rd party module code. In one, http://drupal.org/project/captcha-free, I scanned the code and saw the form state being set to no cache: $form_state['no_cache'] = TRUE;

Now, it looks like that's just on protected forms, but the real suspect is this module is calling a sub php script, extension .php
That called sub function is then calling Drupal bootstrap, but from outside of Drupal (.php) with this:

define('DRUPAL_ROOT', $_SERVER[(($_SERVER['SUBDOMAIN_DOCUMENT_ROOT']) ? 'SUBDOMAIN_DOCUMENT_ROOT' : 'DOCUMENT_ROOT')]);
include_once DRUPAL_ROOT . '/' . 'includes/bootstrap.inc';
ob_start();
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
ob_end_clean();

and then this: Using header mods with "no-cache" options

// Add headers
// Always modified
drupal_add_http_header('Last-Modified', gmdate("D, d M Y H:i:s") . ' GMT');
// Expires in the past
drupal_add_http_header('Expires', 'Sun, 19 Nov 1978 05:00:00 GMT');
// HTTP/1.1
drupal_add_http_header('Cache-Control', 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0');
// HTTP/1.0
drupal_add_http_header('Pragma', 'no-cache');

================================================
I hope no one is offended by my reposting of the suspect lines of code, just trying to chase down the bugs. (the concept is awesome and really works!)

I did not actually load up a full bore VPS CentOS dev Linux VM, go in there with develop and Eclipse to really step through the code in the debugger and see if that's my problem, so I do not know for sure and hope I don't have to go to that level of dev. extreme to nail this puppy.

I just removed this module after just scanning/reading the code and seeing this php script called.

=======================
I'll update in 48 hrs my cache tables results to see if I finally found the cause.

=============
Update: I still don't know exactly what's going on here since I think I'd have to review D7 arch/API in detail, but I suspect my issue is coming about by setting the base regional time in the CMS to "my time" region which is not the UNIX timestamp ($date) on the server. The server is located 3 hrs away of "my time". I've been watching expires timestamps and when I set the base Drupal regional time to the "UNIX timestamp" on the server the physical time zone the server is in, the expires look right. Set it to "my time" which is my region where I am generating content from, the expires keep getting increased and thus not flushed. Maybe related to this: http://drupal.org/node/534092

Like "something" creating forms is going off of regional time for caching whereas the cache flush is managed by UNIX timestamp, there is an time offset mismatch. i.e. Drupal time + 3 hrs = server time. Still investigating and I am using Views with caching and haven't dug deep enough to know which module, core, or line of code is resetting expires instead of deleting the entires.

I'm letting it just run for awhile and checking cache_form size via myPHPadmin. Regional timezone set to physical server timezone, cache_form not growing, staying at around 800MB. Regional timezone set to PST, which would imply a timestamp offset from UNIX $date, timestamp, cache_form grows about 500MB every 24 hrs.

Have cron as a job, using wget.

Same issue here. Large cache_form table -> 3.8GB.
Clearing cache and running cron does not help.

Update: It seems the problem was multi-fold on our site and the cache-form table is now being flushed, is now emptying.

This was an upgrade from Drupal 6.x which in turn was upgraded from Drupal 5.x. While I heavily truncated tables, removed tables, and removed left over variables from modules, especially in the system table which do not uninstall themselves, it still looks I missed some entries and as well as the database upgrade didn't completely migrate from D6 to D7.

Second, watch out for custom code, coding errors and my bad, I did it. I left a space in template.php as I was adding preprocess functions. Look for PHP errors from cron jobs and a space before functions or in empty lines in PHP code can cause these errors.

Third, the default timezone of your Drupal install versus the physical location of the server and thus the server UNIX timestamp (#date), if they are different there is some module or even core not calculating expires offsets from the system timestamp correctly and instead of flushing the cache, is updating the expires time.

Set the cron job up via the server and disable the Drupal cron to not run.

Sum of what worked:

1. look for suspect 3rd party module code and disable
2. set default Drupal timezone to the timezone the server is in
3. truncate cache_form table via PHPmyAdmin
4. repair entire database via PHPmyAdmin
5. optimize entire database via PHPmyAdmin
6. disable internal Drupal cron and set up a cron job via your server
7. Correct code/issues for any PHP errors cron.php is throwing
8. sync up server time with timeserver (ntp is package, installed in WHM, others already)

Drush, command line also work to truncate table, repair entire database and optimize entire database.

Hopes this helps somebody else.

still active on 7.21 here

I am still seeing this problem on 7.22. Of course we have many modules enabled, so hard to say if this is a core bug or tied to another module.

Maybe this will help, but the site that is experiencing extreme table sizes is a Commerce website. The table is ballooning to a huge size within weeks, and this is a site that maybe has a couple hundred visits a day. Switching to running cron.php via server cron, versus Drupal's internal process to see if that helps.

if you're using memcache, don't forget about the "recommended setting" that put the cache_forms into the DB, it was not clearing for me

Many of specified actions in the comment #1506196-14: cache_form is never cleared on some sites, the module can perform OptimizeDB. You can try to use the module to solve the problem with the table cache_form.

Can people having this issue confirm if they are using the webform module?

I dont have webform. I suspect it is caused by http://drupal.org/project/captcha

I can confirm I have this problem on a site that does not use Webforms. We do use Captcha, ReCaptcha and Mollom, however.

We are experiencing same problem - cache_form grows to 1-2 gb in a day. I suspect that having node_form in large percent of your pages doesn't help.. The problem is annoying when cron tasks are performed, which causes high IO because of large number of cache_form entries.

What are the possibilities that can cause this overhead? For now I saw these possibilites:

1. cache_form table is broken.
2. servers and php's timestamps are not in-sync, which causes expire date to be changed rather than created.
3. badly written custom modules or whitespaces cause this problem.

Let's discuss all possible options as I see this happening only in some portion of D7 sites (including mine, which I administer). This is important scaleability question, so let's share best practices in managing this problem. Cheers!

yes, I am able to reproduce is with webform module. cache_form table grows without an end. Looking for a solution too.

UPDATE: I have to get back on this. A couple of hours later those entries expired. Like Eric_A said, it takes about 6 hours for a row to expire.

I'm having the same issue. At one point I had 80 Gigs in my cache_form table. I emptied the table and two hours later I see 10 Gigs. I disabled memcache and the issue went away.

In my previous comment I stated that disabling Memcache fixed the issue but instead it was just hiding the issue because it wasn't configured correctly. It turns out that the site was rendering a view that had tons of teasers with comment forms on every page.

I'm pretty sure that to solve this problem it will be different for everyone but this is how I solved my issue:

Went into forms.inc and tried to figure out what was going on.

This is where I found out that when I went to mypath I would get tons of logs in my watchdog.

  if (!$form_state['rebuild'] && $form_state['cache'] && empty($form_state['no_cache'])) {
    if( request_uri() == '/mypath') {
       watchdog('form_state',  request_uri());
    }
    form_set_cache($form['#build_id'], $unprocessed_form, $form_state);
  }

Then I believe I just did a looked at the form_id and noticed that there were 15 calls to: comment_node_blog_form

I tracked it down to a view that allows comments in the teaser view.

The code was set up to look for the nids of a certain view by using view_get_view() function:

$view = views_get_view($view_name);
$view->set_display($display_name); // like 'block_1'    
$view->render();

$ids = array();
foreach ($view->result as $row) {
  $ids[] = $row->nid;
}

Since it was being rendered, it would cache each of those forms in the cache_form table.

I fixed it by updating to:

$results = views_get_view_result($view_name, $display_name);
foreach($results as $result) {
  $ids[] = $result->nid;
}

I am having the same issue. Cache_form gets huge. I have webform, Mollom, ubercart quiz etc.
Cron is running now every 30min and emptying the cache_form does not help. Also some other tables are getting huge.

I disabled Mollom but it didn't help. I think it's server related issue, I am going to swich to another host, lets see what happens.
I am getting 400 rows to cache_form about every 10 minutes. I have 200 visitors in a day but they are viewing 20 pages

In my case it was some custom code that was causing this issue. Perhaps its another module that is causing it. Best thing I can recommend is going into form.inc and debugging it to see what is causing the problem. (Again this is how I solved my problem. It may not be the same case for you)

In our case it was a custom feature that caused a PDOException (Field 'plugin' doesn't have a default value: INSERT INTO {rules_config}...) which in turn prevented further execution on cron.

We have also had issues in the past with the Honeypot module filling up the cache_form table

We also have this problem on one site. cache_form table grows up to about 500 MB and site is then more or less unusable. Site has here mentioned modules captcha and webform. Although another sites having also those modules (or mollom) are fine. Caching of forms and blocks is turned off. Cron is working fine. I'm thinking about trying https://www.drupal.org/project/safe_cache_form_clear but I don't use drush, so I'm not sure...

Try https://github.com/saitanay/force_fct

you can use hook_form_alter and add to all forms an argument
$form_state['no_cache'] = TRUE
and then all forms will not save in cashe_form table(and it will stop grow), but if you want you can add this argument not for all but to some forms
if you use AJAX you can clear old cache in hook_form_alter for example DELETE FROM cache_form WHERE cache_form.created < (UNIX_TIMESTAMP()-300)

is there a module for this??

I am using way too many modules to track down what is causing this :(

I have content types with around 50 fields, with conditionals fields, associated rules, field visibility settings etc..

Project DB Cache Clear has been originated by issues #2382227: Add option to clear cache_form expired records and #2409377: Add option to clear all cache tables using 1 button and by cron. The release is going to be created soon.

Hi, @Nafes

As I understand it, your module will be a duplicate of the module https://www.drupal.org/project/optimizedb

sergeypavlenko, thank you for the timely comment. I will place links to your module. Are you going to make cleaning the cache tables on Cron? If so, there is no need in extra module any more. I see a postponed issue about Cron in your's module issue queue.

Yes, module clear table "cache_form" by Cron.

Very good. I placed links to your module to the above issues and to DB Maintenance page as well.

Thanks!

Thanks for link.

This problem persists to HEAD

In my particular case, the cache_form was filling up because the index on the table was corrupt. This caused the sql DELETE command to fail during the purging of expired rows. Running OPTIMIZE TABLE cache_form; fixed it for me.

Based on reading this issue thread, there are different reasons why cache_form would not clear. To debug your particular issue, it's useful to understand how/when the cache_form get's cleared. Expired items in the cache table are cleared by drupal in the system_cron() function (found in system.module). This gets called when the drupal cron system is run (either via the automated method - after the page is served, or via a cron job on the server, or via drush). You can try running these drush commands to narrow down where your error occurs, checking the cache_form table after each command to see if a flush has occurred, and also checking the php error logs, and drupal error logs:

drush cron
drush ev "system_cron()"

If you have to, add debug error_log() msgs throughout the system_cron() function to get a more detailed analysis of what is going on. Hope this helps.

Hi,

I have a drupal site based in hongkong. But recently i have observed that the server time and my drupal site time are diffrent. Can this be the issue for huge size of cache_form table.

Does anyone has a solution to this problem?

Regards
Saurav Singh

Just emptied a 21+ Gb cache_form table. It fills up to that size or bigger every month. This has been going on for us for well over two years. I've tried to figure out why it wont empty. I've tried optimizing the table, but that hasn't worked. I've tried watching the logs after cron runs, but I haven't been able to identify anything there. =(

FWIW I think implementing a different approach to cache_clear_all is worth investigation here.

The issue is that DELETE FROM cache_form WHERE expire <> 0 AND expire < :timestamp could hit MySQLs lock limit, depending on the configured value for innodb_buffer_pool_size and the size of the records.

See this post for more information.

So a fix would require deleting the expired rows in chunks, instead of in one pass.

The cache_form table on one of our sites grew to 16GB. Clearing the caches didn't help. Using TRUNCATE TABLE cache_form; didn't work for me either. Then, I followed ttkaminski's directions and optimized the cache_form table, which corrected the issue for me.

I can confirm that the "cache" table was 32 GB suddenly for one of our clients. No idea what caused that so far.

Our table cache_form grew to 15GB. I can't understand why this bug isn't solved yet.

Sorry, but this ticket needed to go up from NORMAL priority to MAJOR, to help gain adequate visibility and resource considerations. Project maintainers have the final say on ticket priority, I'm just trying to bump this up some since the 15-30GB of db cruft per site has me worried we're not the only ones overpaying for db sizes on the cloud due to a form cache flush bug.

+1

This is an issue for me too, I'm seeing a 8GB increase a day and is not sustainable.

I see tons, and I mean tons of rows sticking around all day.

I can confirm this is also happening to me. cache_form table grew to 80 GB (not megabytes) overnight! And even though I clear it, it just starts growing again.

+1
Really hoping that I do not have to install additional modules like OptimizeDB or Safe cache_form Clear, just to keep the database from exploding! Just cleared 17GB out of the cache_form table manually.

wow 322mb in my cache_form. what is the official solution to this?

This issue is a bit all over the place... However:

1. The problem described in the issue summary (with the "cache_flush_cache_form" variable) doesn't make sense to me and I wonder if it's outdated. If that variable is 0, then https://api.drupal.org/api/drupal/includes%21cache.inc/function/DrupalDa... will set it to the request time, and the next call to either that function or the garbage collection function (https://api.drupal.org/api/drupal/includes%21cache.inc/function/DrupalDa...) after the cache lifetime has passed should result in a database query to try to clear the expired cache entries. Can anyone actually experience or explain a problem with that variable? If not, the issue summary could be updated to reflect that this isn't the actual cause. Also, that variable looks like it only comes into play when the "Minimum cache lifetime" setting is used, but I get the feeling this problem is more broad than that and is affecting other sites.
2. @larowlan's comment in #46 seems like a more likely explanation for the problems most people here are experiencing. If so, it would be good to get a patch for that. @webservant316's comment in another issue pointed to the existence of https://www.drupal.org/project/safe_cache_form_clear (edit: and other people in this issue linked to it above also), and perhaps the code in that module could be used as inspiration for a core patch.

I'm reading out this code the same way. If this current IS were true, then all caches would be having problems.

From the DrupalCache perspective, {cache_form} is treated like everything else, and pruned by system_cron() along with everything else. The main difference is that drupal_flush_all_caches() doesn't clear it to prevent problems if someone is filling out a form at that exact instant.

I added Needs tests since that is what is really needed to figure out if we have a problem, but cache.test does have good coverage.

For people that are having the problem and know the have cron running, does mysqltuner provide any insight into? Eg, if you subtract the size of your {cache_form}, is everything else optimized? I also lean to #46 being the real problem. I had a site with a runaway {cache_form} from anon node forms, but mysqltuner and a good cron strategy kept things bounded.

Added a merge request that hopefully implements chunking as per #46.
https://git.drupalcode.org/project/drupal/-/merge_requests/3602

The issue summary must still be updated, which means that Needs work is probably a more correct status.