I have an xml feed that contains over 1300 items.

I have two problems:
1) Getting all those nodes initially imported. I see you can adjust the batch size with a conf variable, but then I just time out. What is the best method for trying to get a large initial import?
2) Once I get that, will this module handle updating those items? What is going to be the best way to keep all those items maintained once I get them in the system?

It's so much cleaner if you are just grabbing the latest 7 items and updating them. But with such a large set, I'm wondering the best way to handle it. Any help would be greatly appreciated.

Comments

mitchell’s picture

mitchell commented
Component: User interface » Code
Status: Active » Fixed

>> What is the best method for trying to get a large initial import?
AFAIK, increase memory_limit and max_execution_time in php.ini. You can also setup php-apc, php-fpm, and some other performance tunings for php and your db.

>> Once I get that, will this module handle updating those items?
Yes, updating works. Feeds supports updating natively.

>> What is going to be the best way to keep all those items maintained once I get them in the system?
You might have some trouble with this on a slower machine, because each item that is to be sync'd will have to be loaded and checked for changes. This could take a long time and thus time out even if the original import worked fine.

Perhaps you could break your exports into smaller subsets if your source exporting service supports that. Please reopen if you need further help on this.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.