Views + CCK (fields) use a lot of memory

subscribing

Background: http://opensourcecommunity.org/2007/05/30/drupal-memory-bug-report

subscribing because I got to little memory.

subscribing

I have a drupal installation with 39k nodes and I simply couldn't install Views, even with memory_limit = 2048M (it reached a ***2GB*** memory limit)... The webserver used was a dedicated machine with 8GB of RAM.
Hopefully people will get around that (in moments like this, I wonder why PHP didn't adopt from the start a 'everything is a reference' policy like Python... certainly would save lots of memory).

Fatal error: Allowed memory size of -2147483648 bytes exhausted (tried to allocate 6549594 bytes) in /www/foobar/www/sites/all/modules/views/views_cache.inc on line 144

As a side note, this bug was triggered in our development server... the production version of the same site has 500K+ nodes (used to have 2 million nodes). The interesting point also is that I don't even intend to use Views to browse through those 2 million nodes, but only to browse simple stuff (blog pages, stories, events, etc) that sum up to some 400 nodes.

There definitely are memory limits, but I'm curious how you have your fields and views set up. See a post at http://groups.drupal.org/node/702 which identifies some memory limitations if you try to join too many different CCK fields in the same view. In that post, dado found that he could improve performance (maybe reduce memory??) by removing the individual fields from the view, then using node_load() in the theme instead. I'm wondering if that could be at least a partial workaround.

subscribing

I didn't have any views set up! I simply couldn't even get it to be installed... so in my case it's not a question of what fields are used on the views, because I simply had no views configured.

subscribing

If you couldn't even install it, it really isn't CCK (directly) causing the problem. CCK's contribution to the problem is that CCK defines lots of fields, each of which may link in another table, and Views get unwieldy when they have lots of linked tables in them. But if you didn't even get that far this is another problem.

My first guess is that the tables array is being created and cached and it is too large because you have lots of modules enabled. Or maybe it's the size of the default views being cached. Those are just guesses, though.

I'm moving this back to the Views issue queue since I think maybe there is something going on during the Views installation that may need some tweaking. Maybe Earl can address that possibility and move it back to CCK if there is something CCK should be doing differently.

I wasn't sure what version of Views to mark this as, so you'll need to confirm that.

subscribing.

There is no way that the Views installation can use 2 gigs of memory. I've NEVER seen anything that egregious in all of my various configurations.

There has to be something else going on. This will have to be confirmed by actually profiling that install and seeing what is using the memory; since I can't duplicate this error, there is no way I can debug it.

I recently had a complete crash of a site due to new content types created by CCK. Luckily the site was only for demo purposes. I created 8 new content types and everything went well until I created another content type that holds references to other modules and users. After a submit of one of the fields I got the famous WSOD. The site is hosted on a shared server by a hosting company and therefore I don't have access to the PHP.INI file.
I tried to troubleshoot the error by migrating the hosted site into a local test environment.

Following steps were taken in a local environment:
1. I increased the memory to 48M and received a lot of warnings:

Warning: MySQL server has gone away query: INSERT INTO watchdog (...

2. I truncated all cache, session and watchdog related tables, but no success so far.

3. I deleted the last created content type that caused the WSOD (straight form the database), but no success

4. I got rid of all CCK and View modules, the site came back online

Can I conclude that CCK and Views are memory eating modules? This is probably because CCK is creating separate tables for all content types which translate itself in many joins used by views and therefore the generated arrays must be pretty large.

I think that it would be better in my case to create custom modules, but I do like the simplicity of CCK and Views.

I used:
- drupal 5.3
- php 5.2.2
- mysql 5.0.37
- CCK: 5.x-1.6-1
- views : 5.x-1.6-beta5

I have had a similar problem and used the "default views" and the "node load" techniques to counter it.
It requires moving from the views ui to the views api (theming).

The symptoms are the same:
strained server (ram, cpu spikes then swap), slow queries (the multiple cck sql joins), slow php execution time, crashing eaccelerator, watchdog timeouts and other onscreen errors.

The idea is to remove all the fields displayed in views, because it is easy to have too many.
The filters and sorting are usually under control, but if cck nodes are used in these the sql joins persist.
See: http://drupal.org/node/67502

In the themed function you modify the view, using node_load to pull the cck fields.
The data is pulled from drupal cache, if any.
For theming, see: http://drupal.org/node/42597

Then you move the whole view to disk. This is done by exporting the view to a custom module.
See: http://drupal.org/node/137898#comment-224760

Since all the php now runs from disk, a lot of ram is saved and eaccelerator is able to compress the work.

Had a similar problem. Views + CCK = site crash.
Thought some modules were the issue. the admin menu module was definitely causing problems.
Checked ?q=admin/reports/status and host using php 4.4.somethingorother. (this is on a shared host)
Switched php engine to php5 using documentation provided by shared host.
Everything is peachy! Running with views, cck, and other modules no problem with 32mb memory limit.

Hope this helps!

Likely related to #218187: Views cache too large.
Please mark this issue as duplicate if appropriate.

Views 2 was the incremental add causing the error message -- I have erased the evidence previously posted since the merlinofchaos does not want to know about it on this thread.

"please don't take over old issues like this. Views 1 and Views 2 are basically different pieces of software." -- merlinofchaos

The "takeover" has been replaced with electronic file shredding, so for future people, good luck :)

Less is more? Oh yeah.

I did resolve my problem, by _______________ (sorry, more silence)

I have optimized Views' memory usage as well as I can. You've got 2 options:

1) optimize it more if youc an figure out how,
2) don't use it.

Second, how do you know Views is at fault here? Drupal core already needs at least 24 megs by itself. What am I supposed ot do, use negative amounts of memory? And what about the rest of your modules? Do they also use no memory?

Also, please don't take over old issues like this. Views 1 and Views 2 are basically different pieces of software.

No update for more than 30 days.