I feel like this is a COMPLETE NOB question but I'm willing to put my pride aside :)
I've got thousands of xml files I need to import to nodes but I can't figure out how to essentially turn Feeds into a bulk import module. The contributed xml parser works like a dream on a single xml file but I don't know how to specify a feed to read multipe files in a directory.
I have installed Feeds Hacks but could not figure out where to confugre or access the actions portion
I have tried the Feeds Directory Fetcher plugin but it will only return one file in a directory
I can create multiple nodes from a single csv, which is great, and update multiple existing nodes using a single csv which also great but my content is in xml
I have tried concatenating multiple xml files into a single file but I coud not get the formatting correct so that feeds parses them to separate nodes.
I have created multiple nodes from feeds from other sites and I suppose I could try and create a feed that is composed of my xml files but I have no idea where or how to do this
Any pointers would be much apprecited

Comments

kapayne’s picture

Status: Active » Closed (works as designed)

Marking as closed - my understanding is that feeds is designed to work with one file at at time. I figured out how to concatenate the thousands of files to be imported into a single file. thanks.

allella’s picture

It's still unclear if/how multiple file paths can be passed into a single mapped file field when the input is a CSV.

I've seen reference to "FeedsEnclosures", which sounds like it would provide for multi-values, but if that's a solution the syntax for use in the CSV is not apparent. Doing 123;"filepath1,filepath2" didn't work.

The feeds_tamper module has an explode plug-in that splits input, like for multiple taxonomy terms. However, it's not working for me at the moment, so I'm not sure if that would even work with multiple files.

Marc Angles’s picture

I've seen reference to "FeedsEnclosures", which sounds like it would provide for multi-values, but if that's a solution the syntax for use in the CSV is not apparent.

yep, same problem, looking for the syntax to use...

tried

* image1,image2
* image1|image2
* image1||image2

etc

architectJpres’s picture

I am not sure what you are trying to do here.

I too have many files which need to be imported into a single data table.

Have you resolved this issue? Until then I am going to have to use some 3rd party software.

From my understanding, if you are trying to import multiple terms into a single field the feeds tamper module and explode plug-in will do the trick.

delboard’s picture

Status: Closed (works as designed) » Active

So what's the csv syntax for separating multiple files in one field? Is there any, or it can be done only through the feeds tamper module.

Marc Angles’s picture

Hi,

I managed to import (assign term to node I import) multiple terms in one taxonomy field, also I managed to put multiples images in one multiple image field.

In my csv file I have one colon for each file or term I want to import in the same field. In the mapping page I just assign each colon to the same field/vocab.

I hope I am clear. enough.

johnpicozzi’s picture

Marc Angles,

Can you be more specific on how you got this to work...

I need to add multiple images to one image field (with multiple images enabled). I also may need to add multiple Taxonomy terms as well. I am going to be using a TAB as a delimiter in my CSV file, I don't know if that makes a difference or not. Also using D7

Any insight into adding Alt and title text per image?

Thanks In Advance,
Jp

Marc Angles’s picture

@johnpicozzi:
Hi, I'm not working on D7 at all for now. But in D6 what I've done is :

* in the mapping, add the imagefield several time (as much as you need to add images in the field.)
* put your images somewhere on the internet
* in your csv, add as much column needed (corresponding to your mapping).
* in each column add the url to an image. If the field is multiple, then, each image should be automatically added to the field.

Hope it helps.

johnpicozzi’s picture

Ok, I see where you are going with this...

I am looking for a way to add multiple images to one image field setup to allow for multiple images. I guess the real question is how do you separate the images in the CSV file.

would it look something like this...

sites/default/files/images/image.jpg, sites/default/files/images/image_2.jpg

Note that my CSV is set to use a TAB as a delimiter not a ,

Thanks,
Jp

Marc Angles’s picture

No it looks like

«http://example.com/sites/default/files/images/image_2.jpg»

And it is ONE image by column.

As I said «in each column add the url to an image»

In the field mapping you can add THE filefield several times, it is up to you to create those columns in the csv file and put AN url in it.

The csv should look like :

column 1 (title), column 2 (my unique filefield), column 3 (my unique filefield),
The title, http://example.com/sites/default/files/images/image_2.jpg, http://example.com/sites/default/files/images/image_3.jpg

Image_2.jpg AND image_3.jpg will end up in the same filefield.

Hope it is clearer :-)

johnpicozzi’s picture

I see!!!!

So you are saying add Image Column 1 and Image Column 2 to the same Filefield on the Mapping screen.

Brilliant!!!

Thanks,
Jp

jchabot86’s picture

Version: 6.x-1.0-beta10 » 7.x-2.0-alpha4
Component: Documentation » Feeds Import

Has anybody gotten this to work with the latest Feeds module for Drupal 7? I believe I am using Feeds 7.x-2.0-alpha4 which is the latest version.

SJourney’s picture

Hey @Kapayne
Do you mind sharing how you concatenated the files?
Thanks.

kapayne’s picture

Hi @SJourney,
To create nodes from multiple xml files, I concatented all of the xml into a single file, did some minor edits and then imported the concatenated file using feeds with an xml parser:
1. Each piece of xml already had a md_context tag and this was the signal to drupal to create a new node each time it read a new md_context tag (this is done in step 6)
2. With all of the xml files in a single folder, I used the copy command in DOS to concatenate them:
dos> copy *.xml Allfiles.xml
This concatenated all xml files in the folder to a single file called "Allfiles.xml"
3. Depending on how your xml is formatted, you may need to removed some tags. In my case, I opened Allfiles.xml in a text editor and using the Replace function, removed all instances of these tags. (There was one instance in each piece of metadata I concatenated):
<?xml version="1.0" encoding="UTF-8"?>
I also removed
<?xml-stylesheet type="text/xsl" href="http://mysite/sites/default/files/imported_metadata/ISO19139.xsl" version="1.0" encoding="UTF-8"?>
You may have other tags you need to remove
4. In addition, add the tag "items" once to the very top and "/items" once to the bottom of the full concatenated file )"items" will need to be enclosed in angle brackets which I had trouble typing in here because it tried to parse it. I saved Allfiles.xml as utf8
5. I created a stand alone feeds importer to generate nodes from the concatenated xml file (I used feeds (6.x.1-beta10) and feeds xpath parser (6.x-1.x-dev) - I also used feeds tamper (6.x-1.x-dev) to change the values of a couple of the tags as they were imported so that's an option if you need it but not necessary).
6. Remember to set the following xpath in the parser:
Context: //MD_Metadata (NOTE the double // here). Other tags won't need the double slashes (for example, a guid tag can be set by a simple xpath: fileIdentifier/gco:CharacterString)

I hope that helps!
best, kpayne

SJourney’s picture

kapayne++
Thanks!

Marc Angles’s picture

Version: 7.x-2.0-alpha4 » 6.x-1.0-beta10

this was not a D7 issue. this is annoying since a lot of people are still working on D6, and I suppose D7 versions can be really different from D6 one.

Please fill a bug under D7 for your problem. thx

windmaomao’s picture

Version: 6.x-1.0-beta10 » 7.x-2.0-alpha7

I tested the D7 version.

It wasn't working initially, but i found out where the issue is in the end .

1) turn on feeds_tamper and apply explode plugin on the multiple value fields
2) please make sure to have separator to be "," instead of ", " (there's an extra space after , ) when you export from either CVS or services views. This is important, because it'll break file enclosure file handling.

Multiple files import actually is working fine. this applies to D7

There's another small bug in file.inc of feeds, where if exception is threw, no other files will be processed. so i added continue after that to take care of that.

    catch (Exception $e) {
      watchdog_exception('Feeds', $e, nl2br(check_plain($e)));
      continue;
    }
BoySherman’s picture

#17, Step 2 solved the problem I was having: couldn't import more than 1 item (it would only duplicate the first). Cheers @windmaomao !

jackhutton’s picture

Component: Feeds Import » Code

thanks windmaomao & godzuke #17 ...step 2 worked for me.
feeds tamper..
rewrote the field w the url separated by a comma with no space..
thanks for these notes.

liquidcms’s picture

Issue summary: View changes

ughhh!! yup, #17 solved this... no space with the , to separate.. would think this is a bug and the import code should trim

tterranigma’s picture

Status: Active » Fixed

There were two different issues being discussed here, one on how to import multiple xml files at once and one on how to import multiple values into a single file field. Both have found an answer. Morever, Feeds includes an option in the File Fetcher plugin to 'Supply path to file or directory directly '. If you check this, then you can enter the path to the directory in your site's file structure where you have uploaded your to-be-imported files and Feeds will import them at once.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Byte_Monster’s picture

If you want to upload and parce more than one files the plupload module might help. I am using it to upload multiple html files and create nodes out of them.

mornel’s picture

gmangones’s picture

Perfect, #17 work fine.