Hi,

As stated in:http://lists.drupal.org/pipermail/development/2007-January/021972.html
"needs VERY serious performance improvement
(currently the memory overhead makes dealing with large datasets quite
hard - probably the underlying architecture needs serious improvement
in order to fix this), "
The out of memory error is a known error.

I experience it with trying to export my weblinks. I have over 500 weblinks in my website.
Is somebody working on this out of memory issue please?

greetings,
Martijn

CommentFileSizeAuthor
#2 01_importexportapi_improve_export_memory_usage.patch9.06 KBSunil Mohan

Comments

summit’s picture

summit commented

Hi,

The error I am getting is:

Allowed memory size of 62914560 bytes exhausted (tried to allocate 32 bytes) in /test.com/public_html/modules/importexportapi/engines/importexportapi_db_get.inc on line 443, referer: http://www.test.com/admin/export/weblink/xml

60M is a lot allready!
Can somebody help with this?
Greetings,
Martijn

Sunil Mohan’s picture

Sunil Mohan commented
StatusFileSize
new 01_importexportapi_improve_export_memory_usage.patch9.06 KB

I've tried to make a small improvement in the way data is stored internally. I am hoping this will lead to big improvement in memory usage (although all data is still loaded into the memory before exporting).

I am attaching a patch made against importexportapi 5.x-1.x-dev (should also work for 4.7-1.x-dev). Can someone try this and let me know if there is any improvement?

This patch only works for export to xml from database (only db_get and xml_put are touched for now). If this works, I will finish this.

summit’s picture

summit commented

Hi,
I followed the patch, but I get a blank screen...
Tomorrow I will know why, because my ISP only sends me the errorlog tomorrow.
May be it's an easy mistake, I keep you posted.
Thanks for trying allready!
greetings,
Martijn

summit’s picture

summit commented

I made a mistake. I missed a "$" I am not getting a blank screen anymore.

Greetings,
Martijn

summit’s picture

summit commented

Hi,

With executing I got a blank screen.
The error I get is:

PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /public_html/modules/importexportapi/engines/importexportapi_db_get.inc on line 186, referer: ../admin/export/weblink/xml

I tried setting the executing time above 30 seconds with:
ini_set('max_execution_time', 120) in settings.php
or
max_execution_time = 1800 ; Maximum execution time of each script, in seconds
max_input_time = 600 ; Maximum amount of time each script may spend parsing request data
memory_limit = 100M ; Maximum amount of memory a script may consume (8MB)
in php.ini

without success.

Greetings,
Martijn

summit’s picture

summit commented

Hi,

To export Weblinks worked with the following settings in settings.php

ini_set('memory_limit', '200M');
ini_set('max_execution_time', 150);

Of course 200MB is a lot, but it worked!!

Thanks allready for your help!
greetings,
Martijn

summit’s picture

summit commented

Hi Sunil,

As expected I still get an error with importing in a drupal 5.1 site.
It didn't help to enhance the memory to 200MB.

The error I get is:

Allowed memory size of 209715200 bytes exhausted (tried to allocate 195059 bytes) in /public_html/modules/importexportapi/libraries/minixml/classes/doc.inc.php on line 607, referer: http://www.test.com/admin/build/import/links%2Cvocabulary%2Cterm%2Cweblink/xml

Please help to solve this!

Thanks in advance,
greetings,
Martijn

summit’s picture

summit commented

New error:

Premature end of script headers: index.php, referer: admin/build/import/links%2Cvocabulary%2Cterm%2Cweblink/xml

I get an http error 500 :(

Greetings,
Martijn

summit’s picture

summit commented

Hi,

I think the module overloads the CPU of the hosting server and the process is then killed?
I will wait for the import patch please.
Thanks in advance allready.

Greetings,
Martijn

summit’s picture

summit commented

Hi,

How is the import-side going.
I am very enxious to try it!

greetings,
Martijn

summit’s picture

summit commented

Hi Sunil, or any other who knows what to do,

It's a month later now, please also get the import from xml running please without out of memory error.
I am waiting for this for a website upgrade, which is to complex to do without this great module.

greetings and again thanks in advance!
Martijn

sun’s picture

sun
Primary language German
Location Karlsruhe
commented
Category: bug » support
Status: Active » Closed (won't fix)

To quote Jaza, the original author of this API:

Although the API is already working and is somewhat stable (i.e. it's not particularly volatile), it is still in the alpha stage, and is for developer consumption only.

4.7.x, I guess, won't fix anymore.