Hi,

when I want to export the file the download rate does not get higher than 1kb/s

Do you know what could cause that problem.

The page has about 3200 user and i want to export about 10 profile fields.

regards

dave

Comments

kbahey’s picture

kbahey commented
Version: 6.x-1.0 » 6.x-1.x-dev
Status: Active » Postponed (maintainer needs more info)

This indicates that you server processing power is slow, most likely the performance of MySQL. Is your host a VPS running on Virtuozzo by any chance?

We stopped buffering the data inside the module, because it was taking too much memory for large sites with lots of users, and profile_csv was aborting. Instead we are displaying each line as it is retrieved from the database.

You can try the -dev version, but I think there is no much difference.

dontgotanick’s picture

dontgotanick commented

This fied the problem

index erstellen:

CREATE INDEX IDX_profile_fields ON drupal_profile_fields (fid)
CREATE INDEX IDX_profile_values ON drupal_profile_values (uid,fid)
CREATE INDEX IDX_users ON drupal_users(uid)
CREATE INDEX IDX_users_roles ON drupal_users_roles(uid)

regards dave

kbahey’s picture

kbahey commented
Status: Postponed (maintainer needs more info) » Needs work

Maybe you can create a patch to the README file that comes with this module, adding instructions to do this. This way others will benefit from your experience.

dontgotanick’s picture

dontgotanick commented

How can I do that?

kbahey’s picture

kbahey commented
Status: Needs work » Postponed (maintainer needs more info)

The strange thing is that I see indexes: code here. fid is a the primary key for profile_fields, and profile_values has uid and fid already as indexes. And here you see there is already an index on uid for the users table (it is the primary key), and users_roles has uid as part of the primary key.

If you have an old backup before the change of the database, then if you can load it and check which indexes were missing, but it seems core already has the indexes you need.

Anyway, to answer your question, you need to read this to create patches http://drupal.org/patch

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Status: Postponed (maintainer needs more info) » Needs work

Good research, but I think the 'needs work' status might be better? Would it be worth having the CREATE INDEX commands in an update script (that first checked if profile module was enabled), or are you content with just mentioning it in the README.txt file?

kbahey’s picture

kbahey commented

These tables belong to core, not to profile_csv. Therefore messing with them is inappropriate.

We can add to the README.txt file a note saying check these indexes, you should have an index on blah blah, and here is how to do it if there is none.

dakku’s picture

dakku commented

I am certainly experiencing the same issue. We haver over 300,000 users. Trying to export them all isnt fun :P

Maybe batching and writing the CSV to file system until all records are processed wouldnt be such a bad idea. Once the file it ready, it could be zipped and dispatched..

Any interest in this solution?