Data transformation -- can we expose $data to a hook?

Well, there is a hook_node_import_values_alter() and a hook_node_import_postprocess(). You could insert your logic there.

See http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/node_import... - there is a doxygen.txt file that could help you convert it into something more readable.

I must agree that the documentation is not complete yet, but if you have questions you know how to find me...

Or did you have some other idea on how this client-supplied transformation function would be supplied?

Firstly: you want to save a import type definition so you can reuse it.

One of the planned features was to to be able to save a node import task. This means that you would go through the wizard and do the mapping, set defaults, etc but instead (or in addition) to doing the import with the supplied file, the node import task is also saved as a "profile". Profiles would use the "name" property of the task more intensively (eg to reference it).

Next time a user wants to do an import with the same parameters, he would be able to select an already created profile (or create a new one which would go through the wizard again like now).

This has not yet been implemented, but maybe I should move it up on the todo list. The task is already saved but only to do the import, not for use (as a profile) for a new import. So this would involve changing some steps on the form but is not so big a job.

Secondly, you want to attach business logic.

This means (in your example): category of source 1 → drupal taxonomy term. Without you probably realising (or more likely, not really documented as it should) node_import already does some data transformation. For example, in your CSV file you can specify "webmaster" as "authored by" and node_import will convert this to a uid if needed. This is just the same kind of data transformation you would want, except you have a more special case.

The data transformation of a taxonomy term to a taxonomy tid is similar, and probably the thing you would want to extend. At present you would need to write a module does almost the same as node_import_check_taxonomy_term(), but then for your category of source 1 → drupal taxonomy term → drupal taxonomy tid.

Something like (untested):

/**
 * Implementation of hook_node_import_fields_alter().
 */
function yourmodule_node_import_fields_alter(&$fields, $type) {
  // Check if the type is a node content type.
  if (($node_type = node_import_type_is_node($type)) !== FALSE) {
    // Now you would need to alter the field. Let's say the Drupal vocabulary has
    // ID 42. What you want to do is add a new 'preprocess' function to the
    // array of already present 'preprocess' functions. This function would
    // convert the input value from the CSV to a tid that taxonomy can use.
    if (isset($fields['taxonomy:42']) {
      $fields['taxonomy:42']['preprocess'] = array_unshift('yourmodule_business_logic', (array) $fields['taxonomy:42']['preprocess']);
    }

    // If you would want to add the same business logic preprocess function
    // to all taxonomy forms, you could do a:
    // foreach ($fields as $fieldname => $fieldinfo) {
    //   if ($fieldinfo['input_format'] = 'taxonomy_term') {
    //     // same thing
    //   }
    // }
    // Note that yourmodule would need to run after the taxonomy
    // module currently. That's because supported/taxonomy.inc
    // does more or less the same thing in taxonomy_node_import_fields_alter().
    // I'll change that because it can do it just as well in
    // taxonomy_node_import_fields().
  }
}

// What you have done now is told node_import that before submitting
// the value from the CSV file, you would first like to run a
// "yourmodule_business_logic" function on the value.
// See from the hook_node_import_docs.php:
/**
 * ...
 * - \b "preprocess" : Array of callback functions. Each of the functions
 *   can preprocess the mapped value. This is a way to validate and
 *   preprocess the input. These preprocess functions can
 *   be used in companion with the hook_node_import_values_alter().
 *
 *   The signature of the callback is:
 *     @code
 *       $return = $function(&$value, $field, $options, $preview);
 *     @endcode
 *   and it should alter the $value passed. Note that if the field
 *   "has_hierachy", the value passed will be an array (grandparent,
 *   parent, child).
 *
 *   The $return value should be FALSE if there is an input error,
 *   NULL if there was not, but not a valid value could be found or
 *   TRUE if a valid value could be found and so other preprocess
 *   functions can be skipped.
 *
 *   See @ref node_import_preprocess for examples.
 *
 *   Defaults to array().
 */

/**
 * A preprocess function that converts a value to the correct
 * taxonomy term.
 */
function yourmodule_business_logic(&$value, $field, $options, $preview) {
  // The function is given the $value (from CSV file or from a previous
  // executed preprocess function) for the given $field (as returned by
  // node_import_fields()). It also passes some options for the field if
  // you have implemented hook_node_import_options().

  // The $field includes $fields['vocabulary'] which is the vocabulary
  // object by the way.

  // Now you need to lookup the $value in your "category source 1"
  // and convert it to a "drupal taxonomy term tid".
  $value = yourmodule_lookup_source1($value);

  // You could give some errors if the value does not appear to be
  // valid in source 1. Look at some the other examples such as
  // node_import_check_user_reference, etc (all *_check_* functions).

  // If you have succesfully looked up the value you can return
  // TRUE here. If you still want the builtin preprocess functions
  // to run (which convert a term name to a term tid) you would
  // return NULL. If there was an error, you would return FALSE.
  return TRUE;
}

This is a pretty small module to implement. OK, it is not totally clear right now how to use the hooks together... I'll need to write some clearer "overview" docs (maybe even including this example).

The question is now... how can I allow users filling in the wizard to submit "business logic" on the fly. They probably will have to specify it in PHP. Need to think about it, but it is more longterm for sure. Maybe it should be possible to have modules (or users) to define extra preprocess functions and let the user choose from them in the wizard.

If you still have questions, or it is not as clear... let me know.

[Edit: array_shift() needed to be array_unshift() in the example]

I too would be interested in data transformation features. My needs are a little simpler, though. I just want to map values on a 1-1 basis.

For example, if I have a column in the .csv file that stores "Y" and "N", and a field in a Drupal content type that allows "Yes" and "No", then I'd like to be able to tell the import process that Y means Yes and N means No.

Similarly, if I have column in the .csv file that holds a pipe-delimited list of 3-digit codes, I'd like to be able to map those to more verbose taxonomy terms.

Business rules would be great as well, but even a simple 1-1 conversion would save a lot of SQL/scripting gymnastics on the export side.

Thanks for the great module!

I'm creating a php transformation module (or could be a patch to this module) for node_import which adds a computed php field to the "map" screen for each row. The idea is to allow an admin user to specify transformations on the csv data right from the ui.

I've achieved basic transformations where there is a 1:1 mapping of the CSV row to a field value (such as, say, performing a ucfirst() on the "Title" field) by adding the php string to be executed to the $options array, and then performing the transformation using hook_node_import_values_alter().

What doesn't seem to be possible with the current hooks is to reference other fields in the CSV data. (A basic use case would be to concatenate two csv columns into the Title field). Is there some way we can expose the CSV data that's in the $data var in node_import_values() for reference in a hook?

I've gotten it working by changing the following in node_import.inc:

drupal_alter('node_import_values', $values, $type, $defaults, $options, $fields, $preview);

... to ....

drupal_alter('node_import_values', $values, $type, $defaults, $options, $fields, $preview, $data, $map);

Is there another way to do this without having to hack the module? Or - should I submit this as a patch?

Thank you for your help and for the awesome module.