The node title XXXXXX is not unique for this node type.

budda,
good question. i suppose one issue is that if you were to import the same file twice you would get double the number of nodes. in many cases (e.g. every node that i want to import) you do not want to import nodes of the same title twice. perhaps we should make this a configurable option at import time.

Yes, this should be configurable. Maybe even an option to do an "update" of the node rather than creating a new node if the title is the same. I'll work on this.

update option would be majorly cool but a little tricky. Which fields to overwrite? I suppose all or none would be the options? I think that would be OK to permit overwrite all fields or no fileds.

OK while we're dreaming here are some ideas:
In field mapping form, do this
(1) switch form so that the CSV fields appear in the dropdown and each node field appears in its single row
(2) have 1 column to map a node field to a CSV field
(3) have a 2nd column to permit entry of a static value to populate globally for all nodes imported
(4) have a 3rd column w/ dropdown to indicate that the field value should be unique or not.
And really dreaming here: (5) have a 4th column to specify what to populate with if it is not unique, where %fieldname becomes substituted with that field's value

he he. Robrecht?

Guys - I want to affirm the direction you are going. I am wanting to use node-import with the ability to update fields. I'd like to enqourage you to have a column which can be specified as the inique field identifer which is not the title field.

Otherwise, I'll have to delete all of the records and the reimport every time - not a big problem, but then they will all be clasified as new (because of a new created date)

Overall - thanks for the great work and your willingness to improve the module.

I have the same problem with legitimate nodes that have non-unique title text (uniqueness in my case is a combination of other fields).

Sorry about this but, being a newie to Drupal, the naive suggestion below may seem dumb but I am a completely new to Drupal so there may be a much better way of achieving the same result:

A nice feature in the import module would be to allow the values that are written to the node's database fields to be 'derived' from values in the import field set.

For example, I have an import file with the following:

legacy_system_id
asset_title
asset_author
ISBN

It woud be great to be able to specify the title as an 'interpretted' text string. i.e. some sort of macro substition syntax (I'm sure Drupal has a standard way of doing this already)

At the moment, there are drop-down boxes to select the field value. If a text box was included too, it may be possible add 'macros'. Let me explain:

As a simple case, we could just use:

%asset_title%

Resulting in the imported node's title being simply:

Animal farm

Not very exciting. But, if we used the following macro:

%asset_title%, %asset_author% - ISBN: %isbn% (%legacy_system_id%)

The resulting title imported into the database in this case would look something like:

Animal farm, George Orwell - ISBN: 0140278737 (ZZaNzza)

Of course, it would be great to eventually have full and arbitrary field mapping, but that's a LOT of work.

I'd be happy to co-operate on a full requirements specification for all sorts of much more exciting things in the import module. I've had too many years at financial institutions connecting their systems together and encountered many nasty situations and nice features to solve them. However, my coding experience with Drupal is a bit like a company CEO trying to write an Excel macro (i.e. VERY, VERY BAD!)

Cheers,
Doug

Ah the joy of feature requests :-)

A nice feature in the import module would be to allow the values that are written to the node's database fields to be 'derived' from values in the import field set.

Doug, I think I'll have to say "no" to this feature request. At least for the moment. BTW: this is something you could do in Excel quite easily *before* you give the CSV file to node_import.

Guys - I want to affirm the direction you are going. I am wanting to use node-import with the ability to update fields. I'd like to enqourage you to have a column which can be specified as the inique field identifer which is not the title field.

Yes, this seems to be the right way to go. This hardcoded behaviour for title is not nice at all. It should be removed or improved upon.

(1) switch form so that the CSV fields appear in the dropdown and each node field appears in its single row
(2) have 1 column to map a node field to a CSV field
(3) have a 2nd column to permit entry of a static value to populate globally for all nodes imported
(4) have a 3rd column w/ dropdown to indicate that the field value should be unique or not.
And really dreaming here: (5) have a 4th column to specify what to populate with if it is not unique, where %fieldname becomes substituted with that field's value

The problem with (1) is: where would you put the sample data?

Other than that, great to see ideas popping out... I'll see what I can do to implement them :-)

suggest, if possible, using combination of 2 fields (title + other field) to determine node uniqueness.

Yes, I'm working on this. I'm reworking the node_import code a bit. This is one of the features that will be included. I hope to release it somewhere this week. The hardcoded "title must be unique" will be gone. I'm playing with a couple of ideas on how to do "this combination of fields must be unique". I think I have the right idea to implement it, but need to test it (and implement it).

First off, Robrecht, great stuff. Good code. It makes sense.

I just hit this problem a couple of times so I googed it and found this posting. Good thoughts here.

For checking that the combination of fields be unique, is it feasible to make a hash of the combined fields, i.e. hash a concatenated (serialized?!) string of it all and store that in a node_import_hash table? Or is that just too crazy?!

One thing I added last night in working with it is to return a listing of the nodes created. I have other scripts that run based on those and I needed those node numbers. Your code is clean and clear so really all I did was everytime success is set to true, I just added the newly created node id to an array of success_list (yah bad naming convention on my part) and did an implode with the result message to get a comma delimited list of added nodes.

Many thanks again!

Regarding your hash idea, the trouble is that node_import needs to be able to indentfy/query from the db any records that already exist. Unless those records' hash is also stored in the DB then calculating a hash on fields to be imported would not be helpful, correct?

One thing I added last night in working with it is to return a listing of the nodes created. I have other scripts that run based on those and I needed those node numbers. Your code is clean and clear so really all I did was everytime success is set to true, I just added the newly created node id to an array of success_list (yah bad naming convention on my part) and did an implode with the result message to get a comma delimited list of added nodes.

The thing you did does work of course, but it might have been easier if you just implemented hook_node_import_postprocess() in some module you maintain. This way you didn't need to change anything inside node_import itself so things are maintainable in long term more.

Eg:

function somemodule_node_import_postprocess($node, $preview, $error, $getlist = FALSE) {
  static nids = array();
  if ($error && !$preview) {
    nids[] = $node->nid;
  }
  if ($getlist) {
    return $nids;
  }
}

This way all the nids of the created nodes are stored in the nids array, and you can get the nids created by calling somemodule_node_import_postprocess(null, null, null, true).

dado asked for this hook_node_import_postprocess, and this is exactly a case where one could use it. Probably he has better examples for usage.

But it would avoid changing the code of node_import itself. Which I think is always a good thing.

For checking that the combination of fields be unique, is it feasible to make a hash of the combined fields, i.e. hash a concatenated (serialized?!) string of it all and store that in a node_import_hash table? Or is that just too crazy?!

Yes it is too crazy :-)

Let's say you want "title" and "some_other_field" to be unique. This means the combination of the two needs to be unique before you consider a node to be a new node instead of an update of an old node. In order to use a "hash" I first have to calculate for all nodes the hash?? (because otherwise I don't know if any other node doesn't have the same hash for this) I may as well query every time I create a node whether it has the same title and some_other_field as all other created nodes - this is just as fast (I suspect).
This can be done by creating a SQL query on demand. I have some code on my PC that does exactly that, but I haven't committed it yet (code heavily based on the views.module btw). Need to test some more.

Regarding your hash idea, the trouble is that node_import needs to be able to indentfy/query from the db any records that already exist. Unless those records' hash is also stored in the DB then calculating a hash on fields to be imported would not be helpful, correct?

Right.

I suggest making the simple case (intentional duplicate titles) work before bothering with this giant head of hair under discussion here. Consider a calendar with events that recur all the time. They will have duplicate titles. I suggest commenting out the check for now, and adding complexity back later when someone gets around to it.

Unique titles is now an option. If the option is not checked, several nodes will be created with the same title.

I will be rewriting node_import for 5.0, possibly using Jaza's work, so I'm setting this as "postponed" for now, although the initial bug report was fixed.