UI to import views-style exportable (a.k.a. How to import importers?)

The reason that there is no 'import' is that it is bad practice to not capture configuration in code. Moreover, there is not much you can gain from directly importing exported code.

However, if you really need it, you could very easily write an importer module with an import form accepting a feeds importer array.

I've posted this same request here: #781700: How to import importers? (sorry)

Here's a reply I had there:

clemens.tolboom - April 26, 2010 - 11:35

The export could be placed into a module. At least that's how I features.module use it this way.

See feeds_defaults.defaults.inc for a working example.

@Alex: please reconsider this request. If there is no time right now, please leave it to 'postponed' till you or someone else steps up(?)

btw, in my duplicate issue of this one I suggested to simply convert the simple text-box of the current export page to an input text-box + add a 'Save configuration' button:

Now that I think about it... it would be cool if the 'Export' link changed to a new 'Manual configuration' one. Then, in place of current ../admin/build/feeds/export/[name_of_importer] there could be a ../admin/build/feeds/edit_config/[name_of_importer]. This page would be the same as the current export page, but with the addition of a 'Save configuration' button that would be enabled if configuration code has been altered.

What do you people think?

People that simply need to export the configuration would still go to that page, but just copy the code instead of editing it. No sweat for them.

Hm. How exactly does it help to not export to code but import through the UI?

Nobody said anything about not exporting (I think?).

As for where it would help to import code from the UI, I've recently had a case where I needed to troubleshoot an importer by comparing it to the default Feed's code and then making minor changes one-by-one and then re-testing to see if the issue has gone. It was a proceedure that required multiple clicking through the interface to have these minor changes edited and saved, while if I could simply edit the config and then save from within the text-box, it I could spare me some minutes or even hours perhaps.

PS: the issue I was troubleshooting was #625196: Fatal errors (Unsupported operand types) and warnings (Argument is not an array) in FeedsConfigurable

...also consider the issues of people trying to import video or images (using Feeds Image Grabber). Some people like myself don't seem to be getting it right, while others seem to do. Imagine if the ones that got things working exported their configurations and we could simply import them and be ready to go (after perhaps some tweaking around)?

#6: klonos - you are not using Features with drush, right?

If you did, your workflow would have been:

1) Change setting (UI)
2) Re-export (drush features-update my_module)
3) Clear caches (drush cc all

I urge you to dive into Features before we keep discussing here. You will *love* it :-)

I have no doubt Features is great, but for now I would like to import my exported importers as simply as possible.

My use case is as follow :

I'm setting up a procedure to import data for a client, and he must configure his target existing Drupal site with all the Feeds stuff (XML mapping through Xpath parser).

It would be A LOT simplier if I could send him the exports in a txt file for him to import it straight away.

Anyway, doesn't an exporter imply an importer ? At least I understood it this way, and was very surprised not to find a menu entry to do that. And I guess I won't be the only one.

As a backup solution, could you tell us where we could find the simplest snippet possible to include the importer in a module ? Is it about a hook implementation ?

The functionnality for imports is absolutely amazing, mcuh more flexible and stable than other solutions, and I thank you for providing it.

OK, never mind, I found it :

You have to implement these two hooks :

hook_feeds_importer_default()
hook_ctools_plugin_api()

... as found in the files in modules/feeds/feeds_import.

Hope this helps.

Yeah, this is a bit confusing. I used the exporter because every field mapping you do rebuilds the menu_router table and on a big production site that's just not cool, it would take HOURS to recreate the config. I just want to import it straight, but there seems to be no way to do that.

+1 for a non-features based importer.

Features is great but I can see plenty of use for a simple text based importer like in Views.

+1 this. Why have an export button but no import button? You're going to keep getting issues created with a UI like that.

I +1 this as well, are you using ctool exportables? If you are it's simple feature you just have to turn on.

would not have believed if it had not read it !

it is bad practice to not capture configuration in code

Well, you've provided "capturing configuration in code" by providing an export UI.

There's just no corresponding way to capture the code back into configuration, which is what the best-practice is supposed to include.

I think a perfect use case for this is, someone provides their feeds export code here in the issue queue or elsewhere and I want to quickly test it to see how it works, what the settings are or even troubleshoot it. I was also confused by this, I kept looking for a code import text area.

I think we need to please reconsider this. There's definitely a valid use case argument for including non features based imports-- or views wouldn't do it (merlin's not exactly known for including superfluous features, lol).

But if thats' not enough, here's what I just ran into. I have a dev site installed in a mysql environment that unexpectedly needs to move to sql server (let's ignore the other ways of doing that migration for now-- i've tried them all and none worked). There's not much work done on the site yet so for now the fastest and easiest thing is to just recreate it in a new site in sql server. However, 2 of the main things that are done are views and feeds importers of which there are about 1/2 dozen of each.

Moving the views over took all of 2 minutes by exporting them from the mqsql site and importing them into the sql server site. Imagine my surprise when I went to feeds, exported the first feed, then went to the new site and could find no place to import it, lol. This 2 minutes job is easily going to take 1/2 hour or more as I click 1000 times through both sites to make sure I got all config duplicated (not to mention the very increased likelihood of fat fingering something incorrectly and having to redo one or more of the feeds.

Since the site is still in heavy development, I don't want to put them into modules yet-- that will just require enabling them, overriding them with customizations, and then recreating the module (a totally unnecessary waste of time).

As for features-- I get that a lot of shops and devs use it. That's great. For me and the small sites I do I found myself fighting with it and wasting lots of time on figuring out how to get it right then any time it might have saved me. Plus unfeaturizing stuff is a total PIA. And for whatever reason, the js in the ui frequently locks up in my environment on difference browsers and i have to recreate the feature all over again. Even so, features is an entire ecosystem of it's own that's not trivial to understand and implement correctly -- seems an unfair burden for a such a simple feature.

Not to mention the total WTF of having an export without an import like most other big time ctools plugin modules have (ie views, rules, page manager, panels, etc).

@Countzero if you did this any code you have would be much appreciated.

i'm actually surprised at how little code this requires-- i've the skeleton complete finished. I just have to grok the process of programmatically creating the feed importer from the import code and it'll be done.

almost there. So far for the feeds api stuff I've got:

  $importer = feeds_importer($form_state['feeds_importer']->id);
  $importer->setConfig($form_state['feeds_importer']->config);
  $importer->save();

$form_state['feeds_importer'] is the export code.

Which seems to be partially working-- though the plugins don't seem to get their proper config. I'm sure I'm missing something. If anyone has any info it would be much appreciated.

Based on code found in #1006292: Importing Feeds configuration through the UI, I've got a patch. Probably could use some more validation, but it's working great for me so far.

Hi WorldFallz,

I have installed it and tested with 5 feeds-configs and all worked well. I'm sure you will make many people happy.

Thanks a lot for this patch.

Hi WorldFallz,

I tested your code and work very fine for me.

Thanks a lot for your patch.

+1 - #22 works fine for me. Thanks for the patch.

Hi, Patch #22 worked great! Please commit; nice and easy way to import importers!
Greetings, Martijn

and here is same patch for D6 (since that was the original issue).

# 22 works. thank you! please commit

thx
div

Should be set to D6 to test for D6.

#27: feeds-ui-import_777888_27.patch queued for re-testing.

it will fail because it is not a GiT compatible patch - but the patch works should anyone need it.

Ah, that would explain the failing tests. Let's see how this attached git compatible patch does.
Also fixed a small spacing mistake.

Hi is #22 committed for D7. Isn't the path not first commit to D7 and then backport to D6?
Greetings, Martijn

This is not committed to 7.x-2.x yet, so moving back to 7.x-2.x-dev.

I found one security related issue with the patch in #22 so far:

The access check for the import page is if someone has the "administer feeds" permission:

+  $items['admin/structure/feeds/import'] = array(
+    'title' => 'Import importer',
+    'page callback' => 'drupal_get_form',
+    'page arguments' => array('feeds_importer_import'),
+    'access arguments' => array('administer feeds'),
+    'file' => 'feeds_ui.admin.inc',
+	'type' => MENU_LOCAL_ACTION,
+  );

But in feeds_importer_import_validate() user submitted PHP code is executed:

+  ob_start();
+  eval($form_state['values']['feeds_importer']);
+  ob_end_clean();

Thus, a check for if the user has the permission "use PHP for settings" should be done too. Views 7.x-3.x does this also for it's import function.

@MegaChriz, you are absolutely right, we needed an access check on execute PHP.

I've added some more validation to the patch as well.
7.x http://drupalcode.org/project/feeds.git/commit/717cd54

this doesn't seem to be working... I'm not sure what's up but i'm closing #1991448: Import Importer as a duplicate.

i've exported a view, went to import and got some odd validation errors unless i set the importer to override existing importers. then i got passed the validation errors, got my feed importer imported, but none of the settings or mappings made it in... i'm not alone. see the closed issue above for more details.

sorry wrong status

just tried getting latest dev from git and got the same issue... empty importer imported

So it looks like the issue relates to the code in the FeedsConfigurable constructor:

  /**
   * Constructor, set id and load default configuration.
   */
  protected function __construct($id) {
    // Set this object's id.
    $this->id = $id;
    // Per default we assume that a Feeds object is not saved to
    // database nor is it exported to code.
    $this->export_type = FEEDS_EXPORT_NONE;
    // Make sure configuration is populated.
    $this->config = $this->configDefaults();
    $this->disabled = FALSE;
  }

When you create a new instance of class, it overwrites any previously existing $this->config with $this->configDefaults()

So running this line:

  $importer = feeds_importer($importer->id);

re-writes $importer's config property to defaults.

Patch attached simply creates a temporary new instance to handle the save, leaving $importer->config; in tact.

I have committed this and 2 other fixes, I think it should be working quite nicely.

These new commits are working great! Thanks to everyone!

Patch in #40 worked great. Thank you very much.

@gamesfrager... Did you even bother to read #41?

After a bit over 3 years this is finally a reality! Yay :D

I am also having the problem where importing an importer results in an "empty" importer. I am working with version 7.x-2.0-alpha8. I tried applying the patch from #40 and I still get the same result, so the patch seems to be ineffective.

The date of alpha 8 is 2013-Apr-22, the date of the commit from #41 is 2013-May-14 -- therefore you need to use the dev to get the fix or patch the alpha.

In case this helps anyone else...

I applied this patch to the alpha.

When I imported I saw this message "Feeds importer already exists with that id." Potentially I am hitting #2153379: Feeds importer importer fails which is fixed in dev. I checked "Replace an existing importer if one exists with the same id." and it worked fine.