Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 37340511 bytes) in /home/jrcholtz/public_html/s

changing the chunk size won't effect the processing of the images. Once the system starts processing images all the listings have already been retrieved from the RETS server so the chunk size has no effect. However if you are referring to the $chunk_size variable in drealtyDaemon::process_images(), then that may have an effect, that variable dictates how many listings images are pull for at a time.

it's set at 25 by default, if you decrease it, that would mean that it would pull the images for say, 10 listings at a time, so if each listing has 20 images, you'd end up downloading to memory 200 images worth of data, depending on images size and quality, that could potentially eat up a lot of resources.

try changing that variable and see if it has an effect, you could potentially change it to 1, however there could be potential problems with the number of connections to your RETS server if it's set too low. Just a heads up.

Additionally, i've never attempted to run drealty on anything other than a dedicated server or a vps, as it is VERY resource intensive, so i make no claims that it will actually function on shared hosting, or come close to it's potential (not to say that it won't).

Lastly, and probably the easiest solution would be to up the memory available to php-cli, i have mine set to between 256mb and 512mb depending on the site/number of listing that need to be processed.

I'll consider making the number of images to process at a given time configurable, this might help, however, due to the massive amounts of image data that is downloaded, not sure that shared hosting or 128mb of memory is really going to be adequate.

I'm going to postpone this issue and give it some thought.

good deal, i'm going to go ahead and close this one out then.

server: Linode 1536 VPS, php_memory:256MB

PHP Fatal error: Allowed memory size of 201326592 bytes exhausted (tried to allocate 8 bytes) in ..../modules/entity/includes/entity.controller.inc on line 214

where do you change the photo chunk variable? directly in the code or is there an admin screen I'm missing?

For now it's in code. In the drealtyDaemon.php file on line 505.

  public function process_images($conid, $resource, $class) {
    $rets = &$this->dc->get_phrets();
    $entity_type = 'drealty_listing';
    $chunk_size = 25;  // <-- change this to something like 10

I suppose i should make this configurable sometime soon.

increasing memory-limit to 512M and reducing chunk size to 10, I am able to get in a few image import runs before the memory error occurs....

wow, i'm typically only using like ~256M of memory during an image import. Do you know off hand how many images are typically being downloaded per listing?

Maybe there is something else going on. Let's try to get to the bottom of this:

What OS is this running on? Linux or Windows? and which distro/version/edition?

What version of PHP? Or can you attach a copy of you php.ini or a better yet a scrape of phpInfo()?

Not really sure what other information would be useful, but that should get us started, maybe i can at least try to reproduce the situation.

Each listing can have up to 25 images.
Server: linode.com - linux, Apache/2.2.16 (Debian), PHP Version 5.3.3-7+squeeze3

yea that phpInfo() was good for the apache2 config, did you try changing the memory_limit for the cli?

/etc/php5/cli/php.ini ?

typically the cli php.ini file is defaulted to:

; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = -1

but it may be set to something else.

maybe try doing something from the command line:

cd ~/
echo "<?php echo phpInfo(); ?>" > info.php
php info.php > phpInfo.txt

then attach that txt file here. worth a shot.

Too confusing. Just wrap it in code tags and Drupal.org will not filter it.
echo "<?php echo phpInfo(); ?>" > info.php

thanks!

Hey now we're cookin'!

my /etc/php5/cli/php.ini memory_limit was set to 192, changing it to: memory_limit = -1 has done wonders. I didn't even know that setting existed - my server was a stackscript install so I hope that wasn't set to 192 for a reason.

Anyway, the import is chugging along and no errors so far, I've only enabled the condo listings (about 1,500 listings - 37,500 photos), now I wonder what will happen when I add the single family category (about 8k listings - 200,000 photos!).

Thank you camidoo for getting me one step closer to a drealty site. Now on to theming the listing.tpl.php to have a map and an image slideshow.

good deal! let us know how it turns out!