I added another 80 feeds to my test tracker this evening, and I started to get some errors. Adding new feeds to the tracker is now returning php memory exhaustion errors.

Sometimes, it also returns this monstrosity of an error:

Warning: Got a packet bigger than 'max_allowed_packet' bytes query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:1729481:\"Got a packet bigger than 'max_allowed_packet' bytes\nquery: UPDATE cache SET data = 'a:425:{i:0;O:4:\\"meme\\":4:{s:8:\\"headline\\";O:22:\\"content_drupal_feedapi\\":9:{s:28:\\"\\0content_drupal_feedapi\\0item\\";N;s:19:\\"\\0content_drupal\\0cid\\";s:3:\\"605\\";s:19:\\"\\0content_drupal\\0mid\\";s:1:\\"1\\";s:31:\\"\\0content_drupal\\0interestingness\\";s:8:\\"0.821679\\";s:28:\\"\\0content_drupal\\0content_type\\";s in /home/bradflo2/public_html/news/includes/database.mysql.inc on line 128

Warning: Got a packet bigger than 'max_allowed_packet' bytes query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:2070382:\"Got a packet bigger than 'max_allowed_packet' bytes\nquery: UPDATE sessions SET uid = 1, cache = 0, hostname = '24.148.7.175', session = 'node_overview_filter|a:0:{}dblog_overview_filter|a:0:{}messages|a:2:{s:6:\\"status\\";a:2:{i:0;s:42:\\"The memebrowser settings have been updated\\";i:1;s:57:\\"Content Source Feedapi successfully added 128 new content\\";}s:5:\\"error\\";a:1:{i:0;s:1729562:\\"user warning: Got a packet bigger than 'max_allowed_packet' bytes\\nquery: UPDATE cache SET data in /home/bradflo2/public_html/news/includes/database.mysql.inc on line 128

What's the deal? Is memetracker php memory intensive? How can I cut down on that? And what's up with these crazy errors?

The tracker appears to be working. I just don't like big scary errors like this.

Comments

kyle_mathews’s picture

kyle_mathews commented
Status: Active » Closed (fixed)

Yes, Memetracker is hugely memory intensive. The memory use grows exponentially as you add new feeds. That's something that will need to be optimized as time goes on but for now you should be fine if you increase memory limits for MySQL and PHP on your box.

Here's a document explaining how to increase the max_allowed_packet size on MySQL:
http://dev.mysql.com/doc/refman/5.0/en/packet-too-large.html

You will also want to increase the max_execution_time, max_input_time, and memory_limit. If your box does nothing other than run Drupal, you can set these fairly high without running into trouble. You'll have to experiment however. For educon.memes20.org, I set these values to 420, 60, and 320M and haven't experienced any more troubles. Here's another blog post about the subject:
http://agaricdesign.com/note/increase-php-allowed-memory-size-limits

Anyone who knows more about setting up PHP / MySQL for high loads want to chime in?