Hi,

I have created a custom entity through my module and have linked the same with users (like profile2 but i'm not using profile2 as i cannot afford to have so many tables for each field). Next, I start using Feeds to import users. Here is what I need:

While each user is getting imported, I create a corresponding entity for the same, using the uid returned from user_save. I do not want to to do it at the end of the entire import because that would mean the corresponding entity for user 1 will be attached to it only after 1500 (or more) such users are created. I need to have some fields from the CSV and corresponding to them I need a handler which will allow me to use those fields and the UID and save my entity.

I've read a lot about the hooks exposed using feeds but I could not really figure out a solution.

Comments

colan’s picture

StatusFileSize
new2.43 KB

In order to get #1183440: Multilingual Feeds - Make field import language-aware working (so that the translations can be recorded), this is rather necessary. Here's a patch as part of the work I've been doing for that issue. (#1364070: Record translations created by Feeds is now waiting for this to get committed.)

mukesh.agarwal17’s picture

Title: Hook into the user import » Provide entity pre/post-save hooks
Version: 7.x-2.0-alpha4 » 7.x-2.x-dev
Category: support » feature
Status: Active » Needs review

Thanks Colan!

I have not yet tried the patch, but from what I see, you have used module_invoke_all to expose the pre/post save hooks. Will try and let you know soon. But before getting into that, I'm thinking of a couple of solutions (please share your thoughts on how feasible/scalable these solutions are):

  1. Create a new processor for the user import which does all the typical user import stuff (cloning the current user import processor) and in that change the process function to incorporate my custom entity attachment
  2. Use hook_entity_insert and hook_entity_update to for type 'user'. From what I understand, the real difficulty is to make the fields in my csv available in $entity passed to these hooks

Anyways, I'm going to try out all the 3 solutions and study the pro's and con's of each.

colan’s picture

Adding tag.

chrisjlee’s picture

@colan - Trying to apply your patch. Which version of feeds should i patch this against? I did a patch of a clean clone from drupal (7.x-2.x-dev) and the patch failed.

$ git apply -v -p1 add_entity_save_hooks-1362378-1.patch
Checking patch feeds.api.php...
add_entity_save_hooks-1362378-1.patch:15: new blank line at EOF.
+
Hunk #1 succeeded at 239 (offset 15 lines).
Checking patch plugins/FeedsProcessor.inc...
error: while searching for:
          }
          $this->map($source, $parser_result, $entity);
          $this->entityValidate($entity);
          $this->entitySave($entity);

          // Track progress.
          if (empty($entity_id)) {
            $state->created++;

error: patch failed: plugins/FeedsProcessor.inc:126
error: plugins/FeedsProcessor.inc: patch does not apply

Should i be patching againsted the 7.x-2.x-alpha version?

colan’s picture

If it doesn't work against the latest dev version, then it'll need to be re-rolled. Set it to "needs work" if that's the case. Not sure you need the "-p1". Try it without it as well.

chrisjlee’s picture

Status: Needs review » Needs work

@colan Yeah i did a -p0 and it had produced a missing path issue:

$ git apply -v -p0 add_entity_save_hooks-1362378-1.patch
Checking patch a/feeds.api.php => b/feeds.api.php...
error: a/feeds.api.php: No such file or directory
Checking patch a/plugins/FeedsProcessor.inc => b/plugins/FeedsProcessor.inc...
error: a/plugins/FeedsProcessor.inc: No such file or directory

I also removed the -p0 and it has failed as well:

$ git apply -v add_entity_save_hooks-1362378-1.patch
Checking patch feeds.api.php...
add_entity_save_hooks-1362378-1.patch:15: new blank line at EOF.
+
Hunk #1 succeeded at 239 (offset 15 lines).
Checking patch plugins/FeedsProcessor.inc...
error: while searching for:
          }
          $this->map($source, $parser_result, $entity);
          $this->entityValidate($entity);
          $this->entitySave($entity);

          // Track progress.
          if (empty($entity_id)) {
            $state->created++;

error: patch failed: plugins/FeedsProcessor.inc:126
error: plugins/FeedsProcessor.inc: patch does not apply

Could you please re-roll your patch against the latest dev version?

colan’s picture

This is going to be non-trivial, as I wasn't aware of #1005128: Rules integration & enable modules to customize imports, whose commit conflicts with the way this is done. This is what happens when the issue queue gets too big. So this is a quasi-duplicate. That one has a pre-save hook (which has a different signature - that's the problem), but no post-save hook, which is still required.

It's necessary to have a discussion over there to determine how to proceed.

chrisjlee’s picture

@colan Do you think there is a way to roll an short-term fix to just get the post-save hooks working? I'm not really interested in rules integration with my import at this current moment. I'm more interested in solving these issues: #1183440: Multilingual Feeds - Make field import language-aware & #1364070: Record translations created by Feeds

colan’s picture

As I don't have time to merge all of this stuff right now, I recommend the following: Get the dev version of Feeds just before that commit got in, and then apply the language patches you mentioned (from around that time). This should allow you to get stuff working. This is what I did the other day.

colan’s picture

Title: Provide entity pre/post-save hooks » Provide entity post-save hook

The pre-save hook is done as of #1005128: Rules integration & enable modules to customize imports. With info in there, and an example implementation over at #1364070: Record translations created by Feeds, it should be possible for someone to redo this patch. I just don't have time right now, sorry.

rtdean93’s picture

I have modified FeedsProcessor.inc to run the pre and post save hooks. They do run, but function entity_translation_feeds_presave($source, $entity) and function entity_translation_feeds_postsave($source, $entity) are still not being called.

          // Allow modules to alter the entity before saving.
          module_invoke_all('feeds_presave', $source, $entity);
          if (module_exists('rules')) {
            rules_invoke_event('feeds_import_'. $source->importer()->id, $entity);
          }
          
         // Allow modules to alter the entity after saving.
          module_invoke_all('feeds_postsave', $source, $entity);
          
twistor’s picture

All for the issue, but this is not a release blocker.

js’s picture

I was searching for a way to hood entity_save because there isn't a feeds_postsave hook. I would like to pass the data table id to a queue for further processing.

Is this code current available? Or tips for hooking the entity event?

Thanks!

sutharsan’s picture

Status: Needs work » Needs review
StatusFileSize
new1.42 KB

Reworked the patch for latest head.

johnennew’s picture

Status: Needs review » Reviewed & tested by the community

Been testing this for a few weeks now and works great for my purposes (making modifications after a node is imported via feeds)

johnv’s picture

johnv’s picture

Marked #1962000: New hook hook_feeds_postsave() as a duplicate.
But please verify that issue, since it has a nice example in the patch.

gordon’s picture

I have merged in my patch #1962000: New hook hook_feeds_postsave() which as mostly a duplicate of this patch. I have included my example, and I have add the $entity_id as a parameter.

The main reason for doing this, is because there is no other way to tell if this is a new entity or an existing one. Yes on nodes you can use $entity->new_item, but this is not available on users or taxonomy.

gordon’s picture

Status: Needs work » Needs review

Forgot to change status.

Also see http://drupal.org/node/1939256 for how I have used this new hook.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 0002-Issue-1362378-by-colan-Sutharsan-gordon-mukesh.agarw_.patch, failed testing.

gordon’s picture

Status: Needs review » Needs work
StatusFileSize
new2.16 KB

oops, php error is the example.

johnv’s picture

Status: Needs work » Needs review
twistor’s picture

Version: 7.x-2.x-dev » 6.x-1.x-dev
Status: Needs review » Patch (to be ported)

Committed to 7.x

I've removed the entity_id. That can be found on $entity->feeds_item->entity_id as well.

gordon’s picture

As I said above the main reason for the $enitity_id was to use as an indicator that the entity is new. With nodes you can check new_item but with users or taxonomy you can't check that.

In the patch the $entity_id for a new item will but NULL, and not the actual id. There are a few ways to get the id, but none that I can think of to check if the entity is new.

With Feeds Delete this means I have to do an additional check to to know if I need to write it down.

If you have any ideas on how to reliably tell if an item is new.

Thanks.

twistor’s picture

$entity->feeds_item->entity_id will be 0 if the entity is new, or the entity_id if it previously exists.

gordon’s picture

Yes on a new item it is, but during the entity_insert the entity_id is updated to the new id. and because this is an object it is all past by reference and past back to hook_feeds_after_save() with this updated.

I was using this field as the key for my own table so I know this is never 0

twistor’s picture

Edit: I see. It's incredibly dumb that we move item saving into entity hooks just because item deleting is there.

hmmm...

megachriz’s picture

Version: 6.x-1.x-dev » 7.x-1.x-dev
Status: Patch (to be ported) » Needs review
StatusFileSize
new905 bytes

Back to 7.x-1.x then.

A possible way to fix this is by adding a property is_new on the feeds item, as it is that you want to know and not the entity id. Passing $entity_id to the hook can be confusing as I would expect it to be the saved id and not the id it had before saving. If you want it solved it that way, it should at least have been called $old_entity_id.
Another way is passing $is_new to the hook.

So, to sum up, I see three ways of fixing this so far:

  1. Add a is_new property on the feeds item.
  2. Pass a parameter called is_new to the hook.
  3. As gorden suggested in #18, except that the passed parameter is called $old_entity_id (to avoid confusion).

The attached patches implements solution 1 (add a is_new property).

Status: Needs review » Needs work

The last submitted patch, feeds-adds-is_new-property-1362378-28.patch, failed testing.

megachriz’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev

Oops, wrong version selected. Try again...

megachriz’s picture

Status: Needs work » Needs review
twistor’s picture

Version: 7.x-2.x-dev » 6.x-1.x-dev
Status: Needs review » Patch (to be ported)

Ok, I actually went ahead and added the $entity_id argument to all of the hooks as that was the only way to do them consistently.

I committed this as well, the is new attr could be useful in other places like mapping.

twistor’s picture

Issue summary: View changes
Status: Patch (to be ported) » Closed (outdated)
megachriz’s picture

Version: 6.x-1.x-dev » 7.x-2.x-dev
Status: Closed (outdated) » Closed (fixed)