Views needs a lot of memory

Perhaps you need to increase the amount of memory PHP has.

This isn't a very useful bug report, unfortunately; no information is given which can help me figure out what's going on. Since you deleted it I doubt you'll get back to this, either.

I have the same error. When enabling views (views and views gui) I have this :

On the screen : "Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 296314 bytes) in /var/www/drupal62/includes/database.mysqli.inc on line 303"

In the logs (not sure if this is related) : "Invalid argument supplied for foreach() in /var/www/drupal62/sites/all/modules/views/includes/view.inc on line 1172."

This is a new install. CCK is enabled and some core modules, but that's it.

Looking in the status report : PHP memory limit 16M (green)

There is no indication that views should take more memory in the docs. Nevertheless is it suppose to fit in the 16MB?

Well, it runs in 16K just fine for me, but frankly every Drupal installation is different and the amount of memory taken up is going to vary.

If you're bothered by the amount of memory it takes up not being in the docs, I suggest 'beta' software is not for you.

BTW, if you've got the devel.module installed with the theme developer module active, there's a known bug with it and Views.

All that said, 16M is low enough that you should consider increasing the amount of memory and seeing if that helps.

Yep, putting more memory in php.ini does the trick.

>If you're bothered by the amount of memory it takes up not being in the docs, I suggest 'beta' software is not for you.

LOL, correct.

I have the same exact problem in 6.2 but I am not convinced it has to do with any particular module. My PHP limit is set to 32MB. I get the memory exhaustion error only on Drupal's module listing page (admin/build/modules). And it only happens when I turn on x number of modules (depending on the memory required by each module when loading the listing page). Otherwise, every module that is enabled works just fine. Only the admin/build/modules page returns the error; i.e. I can turn a module on directly in the DB and it will run perfectly in the site.

If I go into the DB and start turning off modules one by one, the error eventually goes away. However, it does not seem to matter which modules or in what order they get turned off. Depending on which module I turn off each time, the error returns a different file on memory exhaustion. Eventually, when enough of the right combination are turned off to reduce memory usage below the 32MB limit, the error goes away and the page comes up just fine. Then if I re-enable a single module from that page (instead of from the DB), the error returns (naturally) but not before the module gets set to '1' in the DB. So, that particular module will work but the module listing page will not come back up.

Is it possible that one single module (like views perhaps) is taking up most of the memory and not leaving enough for the other modules to operate?

Of course, I probably don't understand the logic, but my question is why does the module listing page have to load each module just to list them? (Once, the error returned memory exhaustion while loading an INSTALL.txt file!) Why would Drupal need to load an INSTALL file when generating the module listing page? Does this make it a Drupal core issue?

Here are the modules I'm running:

CCK
Aggregator
Contact
Menu
Taxonomy
Update Status
Upload
Calendar
Date
Date API
Date Popup
Date repeat
Date timezone
DHTML menus
IMCE
TinyMCE
Views 6.x-2.0-rc1

Each module takes up some amount of memory, and the amount each individual module takes varies. So your experience makes perfect sense, and it means you probably need to make more memory available.

This problem as been reported several other times:
http://drupal.org/node/267211
http://drupal.org/node/279386

Thanks. I've marked those duplicate in an attempt to combine memory related issues into one. If people see other issues that are obviously memory related, please feel free to mark them dup and link here. There's plenty to read here and on the other links presented above.

@#9:

Thanks for the response. I understand that each module takes a certain amount of memory and perhaps Views needs more than most, but does it make sense that only the admin/build/modules page returns the error when every other page runs smoothly? I guess I am questioning the logic behind loading so much of a modules code just to create the module listing page. Does that particular page really need to know anything more than the name of the module, version information, a short description, dependencies and whether it is active or not? All of that info is in the DB 'system' table. Why does it seem to be rifling through the files for each module as well? Or is it?

This is why I said it seems to be a Drupal issue. Even without Views, load enough modules and the module admin page will crap out long before any other page on the site. But why? It really shouldn't be executing any more modules than any other page view. That tells me the code that generates the list of modules is likely the culprit. No?

The modules page has to rebuild the menu, which takes a significant amount of memory; so yes, the modules page ends up being the most intensive page because of this.

subscribe

Just curious... is this one of the issues with Views that 6.3 addresses? I haven't updated yet but will in the next few days, and I'm hoping this goes away. Ha!

The two issues that 6.3 addresses are related to view-specific theming templates and the breadcrumb on the views edit pages. So far as I know, the memory problems may be endemic and not fixable.

I have updated to 6.3 hoping to fix this issue. I still am experiencing it. Is there a way to limit what needs to be loaded?

At this time -- and probably for all time -- the only solution is to increase your memory limits.

What i do to workaround the memory limit is having a separate subdomain that point to the same site.
The settings.php of the main site has memory limit of 32M, to show the pages to the public.
The settings.php of the subdomain has memory limit of 128M, intended for admin and moderator users.

Naming the subdomain 'panel' or 'admin' makes sense for those users.
In this way, the site admin is not constrained by the public limit when going to the modules page.
The anonymous users keep using a fixed amount of resources.

inforeto, That's a great solution. Can you do a step by step on how you did that?

Justin

making subdomains is detailed on the handbook:
http://drupal.org/getting-started/5/install/multi-site

you just add the new memory limit to the new settings.php
ini_set('memory_limit', '128M');

I thought you were doing it some other way. Someone should create a module that adds this setting to the admin setting. It would be great if it could be set by role.

Back to the original discussion, this happened to me too.
Fresh installation of drupal 6.3, right when i enable the Views module, i get a 404 error page trying to load the module page.
Reviewing the error log, i can see there's a problem with the memory.
(I'm not at home and i can't review logs right now, will eventually post error messages later)

I think the keyword with this issue is "install".

I'm running two other websites, same hosting, pretty much same configuration(*), pretty much same modules installed(**).
They run fine and smooth.

when i'm back home, i will try to install a older version of the module (either 6.x-2.0-beta4 or 6.x-2.0-beta3) and then update to 6.x-2.0-rc1

if it works, then the problem should be related to enabling of views module

ADDITIONAL NOTES:
(*)
first website: PHP 5.1.2 / PHP memory limit 128M
second website: PHP 5.2.4-2ubuntu5.1 / PHP memory limit 128M

(**)
on both running websites, i have CCK/Date/Image/Counter/i18n/Bad Behaviour/FCKEditor/IMCE/Meta tags/Pathauto/Tokens/Sitemap/XMLSitemap/Fivestar/VotingAPI and few other modules, and as i said they run fine and smooth

on the website with the fresh install of drupal, i got the error just having few core modules installed (Blog, Book), CCK (only Content) and Date (Date API and Date timezone).
that makes me think it is not really memory running out as suggested above, but it's something which involves memory occupation and shows up only during the enable process of the views module

TIA,
gc

gc,

>> "shows up only during the enable process of the views module"

That doesn't really pan out for me. I get the same error even after Views is installed, enabled and running. In fact, if I navigate to admin/build/modules on a 'working' site and enable Views, the page will submit but not reload. But when I check the DB, Views status is set to '1'. Even though I can no longer load the build/modules page, I can still bring up the Views admin pages and edit and create Views all day long with no problems!

Or are you saying that the build/modules page is going through the install process of each module every time it loads? If so, then it may be an 'install' issue. However, that would seem to be an issue with the way Drupal core is generating the build/modules page.

BTW, like I said earlier:

"Even without Views, load enough modules and the module admin page will crap out long before any other page on the site. "

And I intend this in the context of having a reasonable memory limit like, say, 32MB- that's what most shared hosting setups allow by default.

If there's an error on the fresh install perhaps there's a problem in the files.
Reinstall the modules to avoid corrupted files, wrong text encoding, etc.
Check for missing permissions or anything related.

For views itself, more troubleshoot should discard common problems.
For example, try disabling the other modules to see if views can be installed or indeed has a problem.

Also put the site offline before working on the modules page, because the database may run into bottlenecks if the site is live.
These may not be of RAM only, but possibly different bottlenecks like cpu, disk io and database locking.
Of course, this may not apply to a fresh install.

Now, one thing that Views do special when being installed, enabled or disabled is to rebuild the views cache.
This may be one source of bottlenecks. I have one site where this happens.

Still, the usual common problem is that the modules page loads all the modules at once to work out the dependencies, install files, etc.
All the site cache is deleted at that point and this includes the views cache.
In this respect 32M is not enough if there's many modules, and also incur on the other bottlenecks.

Not sure if disabling the site cache will work here, because views cache is independent.
The point is to figure out what in the site or the views is causing problems on a fresh install.

I'm running 6.3 core and just downloaded, extracted, and enabled the view and view UI modules. I started with the same out of memory problem (which i then increased from 16M to 32M) and now every page in my Administer section comes back with "You do not have any administrative items." no matter how i attempt to access it.

This message is generated on line 58 of ./modules/system/system.admin.inc and seems to be happening because the $blocks doesn't seem to be populating (even though when i run the db queries manually they return values). I've since deleted the views directory but the error is still occurring. It seems like perhaps the views module changed my DB a bit or something, not sure right now. This looks like it's just the tip of the error but just wanted to post and see if anyone had any tips to try. thanks.

gblush: Your problem sounds like a corrupted menu_router table. Try visiting admin/build/modules. If that doesn't work, you need to do the following:

Edit index.php; after the drupal_bootstrap, add this line:

menu_rebuild();

Then visit your site's front page once.

Then remove that line and restore your index.php to its original state. That will usually reset a corrupted menu router table.

that did the trick. much appreciated.

Do we have a fix for this on a shared hosting site like 1and1.com?

I can't seem to change my php memory to more than 24 MB. I can set it in php.ini and reported on drupal system status as such. However, when I look at top, php shows it uses to 24 MB and goes away. I'm not sure if it really means my PHP memory allocation is less than 24 MB.

Hi folks,

Encountered the same error today after installing the Views module when I click on the Modules
Fatal error: Allowed memory size of 18874368 bytes exhausted (tried to allocate 820769 bytes) in ....main/includes/database.mysqli.inc on line 303

I called my hosting service (bizland.com) and they increased my PHP memory from 18 to 100MB. This seemed to fix the issue.

James

I traced down the problem to _views_include_handlers, I have created a fairly simple install with Views, OG, CCK, flashvideo and a few more without Views integration (I was trying to measure update status memory usage really). The *views.inc files are altogether 232 571 bytes (comment, user, taxonomy, node, og and content are the biggest among them) and the views_module_include('views.inc'); line in itself bumps memory usage by 6MB. In comparison, module_load_all for the same site eats 11MB. The homepage altogether is eating 25MB. I am investigating more.

If you would put your classes in something like classname.views.inc then you could use __autoload or given PHP 5.1.2 you could even use spl_autoload. Use function_exists for the latter maybe? Also, demaning PHP 5.1.2 might not be too hideous -- six days and PHP4 is totally EOL.

After a few weeks of inactivity, i came back on my intentio to install drupal on my main site.

And, again, i stumbled upon this issue.
With only 3 modules installed (CCK, date and views), when i enable Views i got the memory error and all the stuff previously described (http://drupal.org/node/259479#comment-919792)

After hours of frustration, i remembered *what* exactly was different in the two other installations that went good: dang, I was *not* into "Site maintenance".

So, after a couple of tests, i could sum up that:

• - it happened *only* the first times i enabled Views (installation of the module Views)
• - it happened with fresh installation (i.e. no content), while in "Site maintenance" mode
• - "making room" (i.e. temp deleting a couple of other modules from /modules) and turning off the "Site maintenance" *seems* to permanently fix this issue (no more memory errors loading the modules admin page, whatever i enable)
• - after this "fix", it works like a charm, like drupal always did for me

Can anybody, of those who had the same issue, confirm this?
And, eventually, confirm the "turn Site maintenance off" fix?

EDIT:
unfortunately, i have to correct myself: putting the site online seems to have some kind of role in this issue, but does not solve it permanently

gc

A fresh new install should be easier to troubleshoot than one with existing traffic.
Enabling site maintenance might not be of help if the ram limit is still being exceeded.

Requiring PHP5 is not an option. Drupal 6.x requires PHP4, and so does Views. Views is used by 95% (or so) of Drupal installations, so any additional requirements imposed by Views are as good as saying Drupal 6 requires it, and that would lead to a significant backlash. I'm not part of the GoPHP5 movement; I am unwilling to be the one accepting that backlash.

Since PHP5 is not an option, auto class loading is not an option. Also, I'm not convinced that Views is using 5 megs of memory just from loading a bunch of code. There's not that much code there.

I appear to be having this problem on Views 6.x-2.0rc1. I am running PHP5, and was using views and CCK without problem on a very active new development site for 3 weeks with no problems.

This warning started when I clicked twice in the Views UI, screwing up its process. It also initiated a javascript error with ajax, solved by turning off javascript in views. I can't increase the packet size of mysql (currently 16M, I checked with my host). I don't see any errors in the PHP log that tells me what initiates this. I have turned off EVERY module except core, even views, still get the error, emptied the cache, the views cache, everything I can think of. I even went back to an unadulterated garland theme. I used devel to uninstall views and imported two of the views I use (not the one on which all the errors started).

This is the error I get, EVERY TIME I reload a page:

Warning: Got a packet bigger than 'max_allowed_packet' bytes query: UPDATE cache SET data = 'a:390:{s:13:\"theme_default\";s:8:\"newskete\";s:13:\"filter_html_1\";i:1;s:18:\"node_options_forum\";a:1:{i:0;s:6:\"status\";}s:18:\"drupal_private_key\";s:64:\"92435af7f38cafa840efc37059977b734a52797e36a78463c66a842857a3eb12\";s:10:\"menu_masks\";a:28:{i:0;i:127;i:1;i:125;i:2;i:63;i:3;i:62;i:4;i:61;i:5;i:59;i:6;i:58;i:7;i:56;i:8;i:45;i:9;i:44;i:10;i:31;i:11;i:30;i:12;i:29;i:13;i:24;i:14;i:22;i:15;i:21;i:16;i:15;i:17;i:14;i:18;i:12;i:19;i:11;i:20;i:10;i:21;i:7;i:22;i:6;i:23;i:5;i:24;i:4;i:25;i:3;i:26;i:2;i:27;i:1;}s:12:\"install_task\";s:4:\"done\";s:13:\"menu_expanded\";a:0:{}s:9:\"site_name\";s:9:\"New Skete\";s:9:\"site_mail\";s:22:\"email@gmail.com\";s:21:\"date_default_timezone\";i:-14400;s:23:\"user_email_verification\";b:1;s:9:\&q in /home/mgwynm/public_html/dev/drupal6/includes/database.mysqli.inc on line 128

This is a major problem (I can't even get the errors to stop writing to the top of my page (turned of statistics, logging, etc.).

Is there anything I can do? Besides pay more money for larger space? Sigh. This is all non-profit work. No money in non-profit.

Thanks,
Maria

Interesting. That's the Drupal theme registry that's causing max allowed packets to burst. That's intriguing. can you use devel.module to display the theme registry and see what's in it?

Sorry for the delay... took a break.

Here is the theme registry, have no idea what it means:

http://dev.mariagwyn.com/newskete/nsthemereg.html

Thanks,
Maria

I'm getting memory exhaust errors on my site when views2-rc1 is activated. I have 16MB and drupal 6.3.

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 77824 bytes) in /Divorciados/sites/all/modules/views/modules/taxonomy.views_default.inc on line 183

Sometimes is cache.inc, or views.inc in the admin.

Maybe it's related to this? : http://drupal.org/node/218187

Dnicolas: Search for the issue "Views uses a lot of memory"

mariagwyn: Hm. I don't see any reason why that would be causing the error you're seeing. I'm not sure what to suggest. :/

hmmm, well, if something pops into mind, let me know. I have even less idea than you. It appears from this thread that the only solution is to increase the max packet size?

merlinofchaos: No luck with that. I was already using node view and I also tried making a custom module. I just tried deleting/disabling all views, enabling only the views core module and it still uses too much memory.

About core views using much memory: a php accelerator seems to help.
Some server and vps setups i tested with actual traffic were having difficulties with ram and it was solved with eaccelerator.
This may only benefit embedded views in cacheable nodes, the more load off the database improves the results.
For example, exporting a view to a custom module in one case reduced the ram usage by 60%.

How does one export a view to a custom module? Are there instructions for this or does it take lots of drupal/php knowledge?

Same question as mariagwyn, thanks.

Enable the views_export module.

Click on the tools tab in the Views admin area. Go to the bulk export tool. Follow the directions from there. Once you've exported your views into a module, instead of 'Delete' the view should have the option to 'Revert' which will remove it from the database. You'll want to do this, but I recommend performing a database backup first just in case something's wrong with the export. Once you've done the whole process once or twice and are comfortable with it, it's fairly easy.

Thank you merlinofchaos!

A few questions:

1. I assume that once the module is exported, no more changes to it are possible (unless I can tweak the code itself). Yes?
2. Can I run views and the module side-by-side for testing purposes, or is this going to cause conflict (at the least, conflict between shared URLs)?
3. If I royally screw it up (quite possible) can I import the module BACK into views (yes, I will make a DB backup) and start over?

Wow...if this works, it is VERY cool!

m

Once you have your code exported into a module, you can still make changes; any changes you make will go into the database. If there is a version of a view in the database, it will override the version in code. When you've tweaked, all you have to do is re-export your view, stick it in code, and then 'revert' it again to remove the version in the database.

And yes, you can keep the export and re-import it. When using the bulk exporter, though, there is a little extra stuff (which is why taking a db backup is teh best idea). If you examine the code differences between an export and the bulk export, you should be able to identify what is extra so that you can cut & paste just what is needed to do an import.

Wait, so I can export (as opposed to BULK export) into a module? Or does the bulk export provide the module features?

You can do either, actually. The bulk export just adds the extra little bit to make it very easy. If you export just one view, you have to add a little bit around the view (the hook and an assignment) -- it should be relatively easy to see this by comparing a single export with the bulk export side by side.

okay, so I did a Diff compare of an export vs. bulk export. These are the additional lines added in the bulk export:

At top:
function Reflections_Archive_views_default_views() {
At bottom:

  $views[$view->name] = $view;
  return $views;
}

1. If I import the bulk export, I assume these are the lines I need to remove?
2. If I use the export rather than BULK export function, do I need to add these lines to make the module work? And simply place in the module folder?

Pretty much! You just need to turn it into a function and be sure to return the list of views.

Alas, I seem to be having problems with exporting a view to a module. So, here are the steps I have followed:

1. Bulk Export ONE view, called "Reflections Archive". This was a CLONED view, not merely modified.
2. Saved exported data into a file titled: "Reflections_Archive.views_default.inc" as instructed. See it here.
2a. DELETED the view from the views list. I did not have the option to REVERT since this was a cloned view, not a modified view. Perhaps this is my key error?
3. Place file in folder entitled "Reflections_Archive" (tried "reflections_archive" as well). So, /sites/mulitisite.folder/modules/Reflections_Archive/file.inc. I also tried /sites/all/modules/....
4. Emptied Cache (and views cache).

No module, no view. So...

5. Created Reflections_Archive.info file:

; $Id: reflections_archive.info,v 1 2008/08/20 14:16:14 mariagwyn Exp $
name = Reflections archive
description = Exported View - Reflections Archive
package = Views
core = 6.x
dependencies[] = views

This didn't work, so I tried to rename it "reflections_archive.info" as well as "Reflections_Archive.views_default.info".
Still, no module, no view.

What step am I missing or messing up?
Thanks,
Maria

Ok, I recommend you make one module, not just named for the view. Call it local or mymodule or mySITENAME.module or something. You need a .info file and a .module file (the .module can be empty) and then the modulename.views.inc file in the same directory as the module (this should all go into sites/all/modules obviously).

Your .info file looks basically fine, except for the naming; probably the issue is that you need an empty .module file for the module to appear. You must enable the module on your modules page.

If you have further questions, lets use a different issue; we're polluting this one =)

moved: http://drupal.org/node/297889

Can I get some clarification? We're talking about exporting views to save memory. So would it also help to revert any default views you have no intention of using? If so there is no 'revert' option on views that are not enabled. Are views that are not enabled a potential problem, or can they be ignored?

I notice that many modules that offer default views do not have them set up as initially disabled. Would disabling them by default help this issue? If so, we should make that a recommendation to modules that provide default views. If they're not disabled by default, would it help at all if the site admin disables them in the UI?

If we can identify the specific steps that could be done to reduce the memory load in Views, I'll write up some documentation.

Karen,

I am not sure this will help much, but I have ALL non-used views disabled. I really only enabled my custom views. In addition, I am running another site on drupal 6.x, just not using much in the way of views (though it is present), and not having the memory problem. Same server, different URL. I am also in the process of upgrading my server to deal with the max_packet problem, but I am happy to test however you would like to see if we can reduce the memory load. Whatever initiated this error has been causing views to behave very strangely, though it still works. I can no longer use Javascript in views, though only on this site, not others. So, let me know what I can do.

Maria

Another thing to take into consideration is wether the view will be used in combination with other modules.
Views in database have a id number assigned, while the default views in modules can only be called by name.

This is normally not an issue, as most views with performance problems are standalone.
Viewsfield module comes to mind but, for example, category_views.module can't use default views.
In this case ever arise, a new feature request issue can be opened on the respective module.

It is still worth to export the view, for the performance gain.
The exported view will also benefit from using php accelerators.

Disabling a view will not save any memory, I don't think.

inforeto: Views database IDs should never be used by anything except for the views load and save functions. The only thing the database IDs are for is to tie records in multiple tables together; the 'name' is a far more useful and important component and should always always always be what is used to identify a view.

I see, then the issue would be directly with the contrib modules.
In category_views.module it calls views_get_view() by number, which default views lack.

That is definitely a bug in category_views.module, then.

This is fixed about as well as it can be.

FWIW, I know this thread is closed but I just wanted to mention that I recently migrated my sites to a new hosting company and the memory exhaustion errors have stopped. However, the PHP memory limit is set exactly the same as it was at my old host, i.e. 32MB. Not sure if anyone is still interested in this issue, but I can provide other server details if need be.

--Brad

which mysql version are you using? both the old and new server

Old server:

Apache version 2.0.63
PHP version 5.2.6
MySQL version 5.0.51a-community
Architecture x86_64
Operating system Linux

PHP Config:
Resource Limits max_execution_time 30
Resource Limits max_input_time 60
Resource Limits memory_limit 64M (*despite this setting, my Drupal 6/Views install always failed at 32MB. See note below)

New Server:

Apache version 2.2.10 (Unix)
PHP version 5.2.6
MySQL version 5.0.67-community
Architecture i686
Operating system Linux

PHP Config:
Resource Limits max_execution_time 30
Resource Limits max_input_time 60
Resource Limits memory_limit 32M

* Other clients were also having trouble with the Drupal/Views memory exhaustion error on the old server. The following discussion is from a thread from the support forum for that host:

[client] The conclusion I've come to (albeit a very unscientific conclusion) is that the original install was somehow corrupt. I installed originally using Fantastico.

When the memory issue kept coming up I tried all of the things mentioned above and still the problem remained, and intermittent. In searching for a solution, I found no good answers.

I manually updated drupal from within Dreamweaver's FTP client and viola it has worked fine ever since. To be precise, I downloaded the tar ball from drupal, unzipped, uploaded through DW and overwrote everything manually. I started with 6.2 or 6.3 (I don't remember) and I think I'm running 6.5 now. I use the custom form module amongst others and it is all working fine. I do not know whether it was some bug in the initial release, or some conflict with Fantastico, a server malfunction, or even corrupted modules. I just don't know. I wish I did know so I'd know what to look for in the future. Nevertheless, it is working now.

**********************************************************

[tech support] I believe it is 32MB per process. Please bear in mind though, just because this is the physical limit set on the server, it doesn't mean you are allowed to suck up this amount of resources under the Acceptable Use Policy (AUP).

Hope this helps,

--Brad

There's two things that affect ram usage between identical sites.

CPU bursts usually last only a couple seconds but make apache allocate a lot more ram to the httpd processes.
It is a pain in a server with bottlenecks because it begins to swap and open new httpd processes instead of closing them.
The resulting slowdown lasts for quite a long time and affect everything on the box including small sites, static sites, ftp, mail, ssh, etc.

The other thing is bugs that occasionally came with updates.
Last time i saw one was last year with a buggy update of perl in cpanel, which affected quite a lot of people for a while.
Most sites never used perl so it largely went unnoticed, but cpanel silently caused problems in the background.

What hosting company did you have and what did you migrate to?

.