Creating / editing Importers

Last updated on
24 May 2018

Drupal 7 will no longer be supported after January 5, 2025. Learn more and find resources for Drupal 7 sites

How to begin creating/editing an Importer

On the feeds (admin/structure/feeds) page, you can create/edit an Importer:

  1. After clicking "Add importer" and entering a name and description of a new Importer, either
    • For a default enabled Importer, click the "Override" link
    • For an importer that you have created, click the "Edit link".
  2. After you have finished creating/editing an importer, you can import a feed by clicking "Import" in the Admin menu, or by entering the URL: www.yourdomain.com/import

Importer edit page overview

The Importer edit page offers four configuration options:

  1. Basic settings (name, where to attach importer, refresh time)
  2. Fetcher (whether to retrieve feed from a file, or web address)
  3. Parser (specify the feed format, eg. XML, CSV, OPML)
  4. Processor (whether to process the data as nodes, taxonomy terms, etc)
    • Processor settings and mapping

These options are described more fully, below.

Importer edit page details

Basic settings

  • Name: The name of default Importer, or the Importer you created
  • Description: What the Importer does
  • Attach to content type:
    • Use either a standalone form
    • or a node to create a feed subscription.
  • Minimum refresh period:
  • Import on create

Fetcher

The Fetcher is where your feed is fetched from.

  • File upload: To upload content from a local file or directory on your PC (which, by default, is located in sites/default/files/feeds).
  • HTTP Fetcher: Download content from a Web site. If you have opted to attach the Importer to "Use standalone form" (Basic setting, above), then you will be able to enter the Web address (URL) in the form. If you have optioned to attach the Importer to a Content type, the URL will be entered there.

Parser

The parser requires that you specify the format of the Feed, either:

  • Common syndication parser RSS 1, RSS 2 and Atom format(See RSS on Wikipedia)
  • CSV parser (See Comma Separated Values on Wikipedia)
  • OPML parser format ("Outline Processor Markup Language". See OPML on Wikipedia)
  • SimplePie parser: Only available if the SimplePie library file, simplepie.inc, has been extracted from the SimplePie download zipfile, and placed in your feeds/libraries folder. You may also have to clear your Drupal cache (Admin/Performance | Clear cache) for Feeds to note that new library.
  • XPath parser Which queries an XML or HTML document using XPath, or Regex expressions. Requires the Feeds XPath Parser module.

Processor

The options available are:

  • Feed Node processor: A Feed node is one that can import feeds themselves. This can be useful for instance when importing OPML feeds.
  • Node processor: The usual option for creating new nodes from feed items
  • Taxonomy term processor: For creating taxonomy terms from feed items
  • User processor: For creating Users from feed items
  • Data processor: For creating simple database records (requires the Data module)

More fetchers, parsers and processors are available through contributed plugin modules for feeds.

Mapping

The mapping option lets you choose how to map data from your selected feed, into the nodes to be created from the feed. For example, mapping the feed item "title" into your node "title", and your feed "description" into your node "body". The available mapping sources is dependent on the parser selected. For example, the third party SimplePie parser, currently offers more mappable sources that the supplied Common syndication parser. Available Mappings

|----------Default parsers----------| |-- Extra parsers --|
      Parser: →

↓ Element

CSV

parser

OPML

parser

Sitemap

parser

Common

syndication

parser

SimplePie

parser

XPath

parser

Feeds Ex
Source . . . . . .
Custom Yes - - - - Yes Yes
Feed/Item URL - Yes Yes - - -
Modification date - - Yes - - -
Priority - - Yes - - -
Title . Yes . Yes Yes -
Description - - - Yes Yes -
Author name - - - Yes Yes -
Published date - - - Yes Yes -
Item URL (link) - - - Yes Yes -
Item GUID - - - Yes Yes -
Categories - - - Yes Yes -
Author link - - - - Yes -
Author email - - - - Yes -
Category domains - - - - Yes -
Latitudes - - - - Yes -
Longitudes - - - - Yes -
Enclosures - - - - Yes -
-
Target . . . . . .
Title Yes Yes Yes Yes Yes Yes Yes
Body Yes Yes Yes Yes Yes Yes Yes
Published status Yes Yes Yes Yes Yes Yes Yes
Published date Yes Yes Yes Yes Yes Yes Yes
URL Yes Yes Yes Yes Yes Yes Yes
GUID Yes Yes Yes Yes Yes Yes Yes
Taxonomy: Tags Yes Yes Yes Yes Yes Yes Yes
pubdate: Start Yes Yes Yes Yes Yes Yes Yes
pubdate: End Yes Yes Yes Yes Yes Yes Yes

See also the list of implemented mappers http://drupal.org/node/856780 Custom mappings: If available, it is here that you can create custom mapping. For example, if your feed item has a field called "Pubdate", you might want to map the information into a date field in your node called "Publication date". Availability of the the SimplePie and XPath parsers are described above, in the section on parsers. Mappings from CSV files: If you are importing data from a CSV file, most of your mappings will be custom. Here are a couple of notes that might be helpful when doing CSV imports:

  • Make sure your mapping source matches the header row in your spreadsheet (not case-sensitive.)
  • When you're working with complex data, you might have your own GUID to supply. Select your GUID in the mapping by removing the default GUID in the sample node importer, then creating a custom mapping to GUID. Then it can be selected as a unique target. An example is described in this comment
  • See also "Feeds Glossary/CSV" especially for more information on imported date formats

Check out this Development Seed screencast for an example node import from CSV.

Importing images and other files

If you want to import images and store them in a Drupal 7 image field, or files and store them in a file field, you need to make sure that your source (CSV column, XML field, etc.) gives a URL or path where the image can be found. (This documentation may apply to Drupal 6 image and file fields also.) There are a few options for constructing this URL or path, as follows.

  • Provide full URLs to the images or files (accessible by the server) in your import source file, such as http://example.com/images/example.png. The files will be downloaded and stored in Drupal's file store.
  • Manually move the files to the server first, and then:
    1. Use the path from the drupal root in the import source file. For example: if your image resides in sites/default/files, the path would look like sites/default/files/myimage.jpg. Note: If you are using Field collection feeds to import a field collection, use this option.
    2. Or put the files into your public files directory (e.g. sites/default/files/import) and make your import source file use a link that looks like public://import/example.png.
    3. Or put the files into your private files directory (e.g. sites/default/private_files/import) or any directory under your configured private files directory, in which case the import source file would use a link like private://import/example.png

Importing multiple values and collections

Field collections can be imported using Field collection feeds. To do this, you first import the nodes, and then import the field collection values in a separate Importer. There is a documentation page for this module that explains a bit more: http://drupal.org/node/1831004 Multiple-valued fields can be imported from a CSV file by using the Feeds Tamper module. The steps, in brief, are: 1. Create a CSV file with all of your field values put together into one column. For instance, if you are importing news items and have a multi-valued Taxonomy field called "Topic", you might have a CSV column called "Topics" with individual topics separated by |, such as "Politics|National". 2. In your importer, map the CSV field to your Taxonomy field. 3. In the Tamper tab of the importer, find the field mapping. Apply the "Explode" tamper operation, and tell it to explode on the separator character "|".

Using your new importer

After you have finished creating/editing an importer, you can import a feed by clicking "Import" in the Admin menu, or by entering the URL: www.yourdomain.com/import

Help improve this page

Page status: No known problems

You can: