Running a product display (node) import, I am not getting updates applied correctly. My data looks like so:

"sku1,sku2,sku3,sku4","displayNode1"

I am using tamper to explode the first field and setting as SKU

If displayNode1 does not exist it is created correctly, and it is populated with references to the correct sku's. If it does exist with the four references as above, and I run a new import with

sku1,sku2,sku3,sku4,sku5","displayNode1"

Response from feeds importer is "There are no new nodes.". The displayNode1 is not updated.

I do have update existing nodes set on the importer. Possibly just changing references to other nodes is not enough to trigger a difference in the import? Any ideas?

CommentFileSizeAuthor
#5 Screen shot 2012-03-18 at 9.11.08 AM.png29.1 KBNicholasR

Comments

NicholasR’s picture

NicholasR commented

Does anyone else experience this error? It seems to me a major problem? I'm not quite sure on how to approach a fix, but it currently requires me to delete every product display node before updating.

pcambra’s picture

pcambra
he/him
Primary language Spanish
commented
Category: bug » support
Priority: Major » Normal
Status: Active » Postponed (maintainer needs more info)

I'd say this is a support request more than a "major bug report".

Could you provide sample data and your import configuration? Note that you need to set your importer to replace or update nodes to get the update code firing.

You're probably using feeds tamper or something similar to import things like "sku1,sku2,sku3,sku4,sku5", right?

NicholasR’s picture

NicholasR commented
Status: Postponed (maintainer needs more info) » Active

I apologize if this is a support request, I had assumed that this functionality was in place, but possibly I was wrong! Regardless, thank you for the response.

my data looks like so:

"Style","Name"
"103D-14Y,103D-18Y,103D-14W,103D-18W,103D-14Rose,103D-18Rose","103"
"104SQD-14Y,104SQD-18Y,104SQD-14W,104SQD-18W,104SQD-14Rose,104SQD-18Rose","104SQ"

All of these values are already in the system. A separate import handles the 'style' (sku) records and works fine. Feeds tamper explodes the style record here to assign muliple products to the one display node (such as 103).

My node importer is 

$feeds_importer = new stdClass;
$feeds_importer->disabled = FALSE; /* Edit this to true to make a default feeds_importer disabled initially */
$feeds_importer->api_version = 1;
$feeds_importer->id = 'node_importer';
$feeds_importer->config = array(
  'name' => 'Product Display Node Importer',
  'description' => 'This importer creates the product displays and binds individual products to it',
  'fetcher' => array(
    'plugin_key' => 'FeedsFileFetcher',
    'config' => array(
      'allowed_extensions' => 'txt csv tsv xml opml',
      'direct' => FALSE,
    ),
  ),
  'parser' => array(
    'plugin_key' => 'FeedsCSVParser',
    'config' => array(
      'delimiter' => ',',
      'no_headers' => 0,
    ),
  ),
  'processor' => array(
    'plugin_key' => 'FeedsNodeProcessor',
    'config' => array(
      'content_type' => 'product_display',
      'expire' => '-1',
      'author' => '1',
      'mappings' => array(
        0 => array(
          'source' => 'Style',
          'target' => 'field_product:sku',
          'unique' => FALSE,
        ),
        1 => array(
          'source' => 'Name',
          'target' => 'guid',
          'unique' => 1,
        ),
        2 => array(
          'source' => 'Name',
          'target' => 'title',
          'unique' => FALSE,
        ),
      ),
      'update_existing' => '2',
      'input_format' => 'plain_text',
    ),
  ),
  'content_type' => '',
  'update' => 0,
  'import_period' => '-1',
  'expire_period' => 3600,
  'import_on_create' => 1,
  'process_in_background' => 0,
);

Any thoughts you may have would be fantastic. I'm not sure how the node processer checks to see if an update should occur to a record, but I would be happy to have low performance and automatically update every record regardless of if it has changed.

pcambra’s picture

pcambra
he/him
Primary language Spanish
commented

Please review my comments in #1

'update' => 0,

You need to set your importer to replace or update (depending on your needs) the data in order to be able to replace or update your data. Otherwise it will check for the GUID column and determine that the data you're trying to import is already there and then is when it throws the "no new nodes" message.

NicholasR’s picture

NicholasR commented
StatusFileSize
new Screen shot 2012-03-18 at 9.11.08 AM.png29.1 KB

Yes, I noticed that line, and thought it was odd that '0' would mean update, however, in my Node Processor settings I have 'Update existing nodes' selected (see screenshot below). I'm not sure what I could be doing wrong there?

I also see the 'update_existing' => '2' line. Are there 2 different spots in the Feeds interface I need to specify that updating should be occurring? I guess I'll go through all the screens to try and locate something.

Again, thanks for your time, I appreciate it.

smokinggoat’s picture

smokinggoat commented

Hey Pedro -

Sorry to say, as far as I can tell, I agree with NicholasR :P (unless there is something both of us misunderstand). I was helping "charged" on IRC with this, and saw the same behaviour - essentially for a Display importer, any existing nodes to not get updated *particularly updating the node reference field*.

And same as NicholasR, we set "Update existing nodes (slower than replacing them)" (which seems like it's 'update_existing' => '2' and not the simple 'update=0' line).

Using:
- Commerce Feeds 7.x-1.2+7-dev
- Feeds 7.x-2.0-alpha4+40-dev
(Drush pmi info)

smokinggoat’s picture

smokinggoat commented

OK, so here are the steps to reproduce my issue above. You'll see it starts with an error, and perhaps the real issue is the way of avoiding the initial error. See my comments at the end for this latter discussion. I don't think this helps NicholasR's situation, but I think it could be related.

To reproduce the problem:

  • Create a CSV import of products that will also be used to import product displays (one CSV)
  • Create a product importer
  • Create a display importer, mapping SKU to GUID (and set as your unique target). Also map SKU to product reference field. Both of these are normal mappings for this kid of import.
  • Set the display importer to "Update nodes"
  • Run your display importer *first*. I know this is *wrong* but bear with me (and see below). Nodes are created, but without any product reference.
  • Run your product importer. You now have products.
  • Run your display importer - the existing nodes *do not* get updated - the product reference field is not updated.
  • Note: "Replace nodes" also seems to fail. :-(

So what happens is that the displays get created missing product references, and then when I try to run the display importer again with the products in place, I'm using SKU as the unique target - but it does not update existing node (of course, because there is no matching SKU) *nor* does it create new nodes (expected behaviour). Is it because the product reference value is stored on the node table (so passes the "matching" test and does not need an update) but is not working properly to reference the right product?

Initial problem : Why am I running display first? Because when trying to trigger imports programmatically with cron it's unclear how to force product importer to run first, then display importer (this is the issues "charged" is facing when we discussed in IRC).

pcambra’s picture

pcambra
he/him
Primary language Spanish
commented

Ok, got it. Let me try to provide a good explanation for this.

First of all, this is a behavior coming from Feeds itself, commerce feeds doesn't take care of node import/sync, it only provides a product reference mapper which is kind of standard. Excepting the update behavior which is custom, but replace is "failing" too, and that's feeds standard.

Feeds calculates a hash when you import something and stores it in the feeds_item table using the file structure and row contents for the importer, so if nothing changes in the origin of the data, you'll always get the "There are no new nodes" message, it doesn't look to the current node or whatsoever. This is happening at FeedsProcessor.inc on ::process method.

Now a real, hopefully useful answer: What can you do with this scenario.

You've got many options actually. I'd recommend as the first and main one try to import everything in the right order. You can do this for example with drush, see #608408: Drush integration for Feeds for getting a patch to do that and call the drush commands in sequence in a cron process.

Another option that comes to my mind is forcing the update by changing the hash in the feeds_item table, you can do that in a process after the import and you can use the "empty" hash, i.e: d41d8cd98f00b204e9800998ecf8427e

A last option could be create your own processor same as commerce product processor is but only replace the process method, changing the behavior of the hash check to whatever fits your use case.

NicholasR’s picture

NicholasR commented

Thanks for paying attention to this guys.

I'm going to look into changing the hash in the feeds_items table after import. I'll post back my findings later to help any future users.

pcambra’s picture

pcambra
he/him
Primary language Spanish
commented
Status: Active » Fixed

Marking as fixed, feel free to reopen for more details

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.