Hey there,

My XML file consists of 100 lines of different entries.

Here is an example of a line of my XML file:

<CBDINSTL BDINR="84" BDINAAM_RELATIE="ROC Midden Ned. lok.Gildevaart" BDISTRAATVEST="HARMONIELAAN" BDIHUISNRVEST="2" BDIPCVEST_NL="3438EB" BDIPCVEST_BUIT="" BDIPLAATSVEST="NIEUWEGEIN" BDITELNRVEST="(030) 607 85 85" HOME_PAGE="www.rocmn.nl" />

I have an importer and it is importing reaaaly slow if I add a location mapper. Without adding a location mapper, with all the other mappers, it imports in a second. But if I try adding a location mapper (for example: Location:City) it will take approx. 5 minutes just to import 100 lines.

Also if I add more than 1 mapper for location (say 5 different ones for street, phone, Postcode etc) it doesn't change the import time. It also takes approx. 5 minutes.

I use XPath Parser module to take location information form the attributes of a node.

Thanks in advance...

CommentFileSizeAuthor
#4 feeds_import.zip5.41 KBSinan Erdem
#2 location_feed.zip1.95 KBSinan Erdem
#2 location_xml.zip5.74 KBSinan Erdem

Comments

elliotttf’s picture

elliotttf commented

Can you export your feed into a feature and give me a well formed sample file to work with? I did some local testing but couldn't confirm the issue you're seeing.

Sinan Erdem’s picture

Sinan Erdem commented
StatusFileSize
new location_xml.zip5.74 KB
new location_feed.zip1.95 KB

Hey Elliot,

Thank you for your interest.

I am attaching the feature as a zip(location_feed). I am also attaching the XML file (location.xml) In this file, I only need to take the information under CBDINSTL nodes. V_A_SCHOOL_CREBO nodes are ignored.

Also after the first import, the last CBDINSTL is not imported. There is no success message given either. But if I run the importer for the second time, it will import the last node and will give a success message.

Cheers,

Sinan

elliotttf’s picture

elliotttf commented

Thank you for the data. I was not able to reproduce your specific issue; however, I do not have a "school" node type so feeds was just happily chugging through the list and throwing out data. Can you add the school node type you are using to the feature and re-upload it?

Sinan Erdem’s picture

Sinan Erdem commented
StatusFileSize
new feeds_import.zip5.41 KB

Ahh, sorry, I forgot. Here is the feature with school content type included.

Thanks...

dcsmith130’s picture

dcsmith130 commented

I am experiencing the same issue. I'll follow this thread. Let me know if I can be of any help. Happy to provide more detail if another testcase is needed.

Sinan Erdem’s picture

Sinan Erdem commented

Can it be related to geocoding? Because I am using Google geocoding in order to get the coordinates from given addresses from the feed.

dcsmith130’s picture

dcsmith130 commented

for what its worth I am not using any geocoding during the feed load (data already contains address, lat/lon)

elliotttf’s picture

elliotttf commented

Sorry for the slow response to this, I've been quite busy at work. I just ran the import with the feature you provided and did not experience any slowness. Is there any more info anyone that's seeing this could give me that might be useful?

dskaw’s picture

dskaw commented

Just curious if anything ever got resolved with feeds being slow?
we are trying to import 100,000+ records and it is VERY SLOW....
it would take several hours to upload.

Chris Charlton’s picture

Chris Charlton
Primary language English
Location Los Angeles
commented
Issue summary: View changes

Have you looked at bumping up the variable 'feeds_process_limit'? https://www.drupal.org/node/1551246