I have a rather large site. Recently I automatically added thousands of pages.

The next time that notify tried to run it ran out of memory. I've partially solved this through the code added below but what is really needed is for notify to break up the number of nodes it handles at one time into chunks of a limited size.

While limiting the number of nodes it handles at one time will make it scalable, the code below helps on sites like mine where the node are quite large in the first place. In my case the node size has come from many cck fields.

Here is the code I used to greatly reduce the "per node memory cost":

    	$node_tmp = node_load($node->nid, NULL, NULL, false);
    	# So we don't chew up too much memory, copy over only the data we are going to use
      $node->nid		= $node_tmp->nid;
			$node->status = $node_tmp->status;
			$node->title	= $node_tmp->title;
			$node->name		= $node_tmp->name;
			$node->type		= $node_tmp->type; 		# Required by "node_get_types('name', $node)"
      $node->teaser	= $node_tmp->teaser;	# Required by "_notify_content($node, $user)"
      $node->format	= $node_tmp->format;	# Required by "_notify_content($node, $user)"
      $nodes[$node->nid] = $node;

Comments

reg’s picture

I should mention, the "false" on the node_load is a var I pass to suppress caching of the node. I use this in calls from cron routines that handle many nodes at once so as to not use up all my memory just in caching. This is not standard functionality of the node_load routine, I added it.

matt2000’s picture

Status: Active » Closed (duplicate)

Trying to consolidate into #365700: Need to queue mailings across multiple cron runs for scalability.

This is a good idea, but limit's the ability to add CCK integration, which is desirable. Not sure yet how we'll proceed on this.