Field import/export

I'm currently testing this and going through the code. This is rather spectacular.
As I answered to Moshe in http://drupal.org/node/58153#comment-156928, I thought that implementing import / export trhough the (yet to complete) CRUD API would be more solid than through programmatic forms. I'm not so sure right now, I guess we can debate this when the API is done (will have to get done one day, right ?)

Anyway, _this_ is working now, and quite beautifully too. We should probably ask for JonBob's approval, but IMO this belongs to the core CCK package. Kudos, and thanks to Moshe as well.

I'm marking http://drupal.org/node/58153 as a duplicate.

As my previous post shows, I was so baffled at this that I could not resist to dive into the code.
Here are a few remarks / suggestions - some of them are related to the code in devel's original 'generic' macros.
Maybe you're working yourself on this and I'm not bringing anything new, but, well, here's my review - and a modified version of the module for a more tangible illustration.

- I don't get the if ($step < 2){ } part : moving the declarations in this branch to the switch ($step) case 1: seems to work the same ?

- It seems the 'recorded' form does not need to go through its regular '_content_admin_field_submit' callback.
In the _form_alter, the '#submit' property could just be overridden to have the form go directly to content_copy_record_macro.

- I'm not sure I get the field_name generation part in content_copy_record_macro.
Don't we already have a name for the field ? I'm probably missing something, though.

- In the exported code :
The 'parameters' and 'form_id' parts could be removed : we already know the form id, and we can get the other info from the 'field' part.
Plus in the 'field' part, the 'submit', 'form_id' and even 'type_name' can be removed.
So the export result would be something like

$field['field_field1'] = array(
  'widget-type' => ...
  'label' => ...
  ...
);
$field['field_field2'] = array(
  'widget-type' => ...
  'label' => ...
  ...
);

That is, nothing more nothing less than a list of fields definitions. Cleaner to the user's eyes, and more open to subsequent (?) module-defined content types...

- the include of install.inc during import does not seem required

- and that's all, actually... I'm not sure what additional validation could be made during import, aside from what is already done. You get the field validation errors in case something is wrong (even though the error messages might be puzzling when you don't actually have the form in front of you, but I guess this is a downside of programmatic form submission anyway)

Any chance there is a content export/import module for 4.7?

Most probably not. As Karen said, this module relies heavily on 5.0 forms API.
But you should be able to get the feature using Jaza's much more complete (and massive...) 'Import / Export API' module : http://drupal.org/project/importexportapi

i just learned about this effort. very nice work. i will try to provide feedback soon.

UI looks terrific.

I exported the 'page' content type after adding a node ref field to it. then i got errors on importing to a new site (i deleted page there before anything). Here is what I saw. If need be, I can debug further ...

* warning: array_merge() [function.array-merge]: Argument #2 is not an array in /Library/WebServer/contributions/modules/cck/content_copy.module on line 239.
* warning: Missing argument 1 for drupal_execute() in /Library/WebServer/htdocs/bpr/includes/form.inc on line 152.
* warning: Missing argument 2 for drupal_execute() in /Library/WebServer/htdocs/bpr/includes/form.inc on line 152.
* warning: array_key_exists() [function.array-key-exists]: The second argument should be either an array or an object in /Library/WebServer/contributions/modules/cck/content_copy.module on line 287.
* An illegal choice has been detected. Please contact the site administrator.
* warning: array_merge() [function.array-merge]: Argument #1 is not an array in /Library/WebServer/contributions/modules/cck/content_admin.inc on line 693.
* warning: array_merge() [function.array-merge]: Argument #1 is not an array in /Library/WebServer/contributions/modules/cck/content_admin.inc on line 694.

* Created field parentref.
* The field parentref (field_parentref) was added to the content type , but an error has occured updating the field settings. Please check the errors displayed for more details.

tried again and still have same error. here is what i am pasting. i also see some additional status messages. see below ... this will be more readable via web than email:

$macro[page]['form_id'] = 'node_type_form';
$macro[page]['values']  = array (
  'node_options' => 
  array (
    'status' => true,
    'promote' => true,
    'sticky' => false,
    'revision' => false,
  ),
  'comment' => 2,
  'upload' => 1,
);
$macro[page]['parameters']  = 'a:1:{i:0;O:8:"stdClass":14:{s:4:"type";s:4:"page";s:4:"name";s:4:"page";s:6:"module";s:4:"node";s:11:"description";s:83:"If you want to add a static page, like a contact page or an about page, use a page.";s:4:"help";s:0:"";s:9:"has_title";s:1:"1";s:11:"title_label";s:5:"Title";s:8:"has_body";s:1:"1";s:10:"body_label";s:4:"Body";s:14:"min_word_count";s:1:"0";s:6:"custom";s:1:"1";s:8:"modified";s:1:"1";s:6:"locked";s:1:"0";s:9:"orig_type";s:4:"page";}}';

$macro[field_parentref]['form_id'] = '_content_admin_field';
$macro[field_parentref]['values']  = array (
  'widget_type' => 'nodereference_select',
  'label' => 'parentref',
  'weight' => '0',
  'description' => '',
  'group' => false,
  'required' => '0',
  'multiple' => '0',
  'referenceable_types' => 
  array (
    'poll' => true,
    'group' => true,
    'page' => true,
    'story' => true,
  ),
  'advanced_view' => '--',
  'submit' => 'Save field settings',
  'type_name' => 'page',
  'field_name' => 'field_parentref',
  'field_type' => 'nodereference',
  'module' => 'nodereference',
  'form_id' => '_content_admin_field',
);
$macro[field_parentref]['parameters']  = 'a:2:{i:0;s:4:"page";i:1;s:15:"field_parentref";}';

Added field parentref.
The field parentref (field_parentref) was added to the content type , but an error has occured updating the field settings. Please check the errors displayed for more details.

FYI, the bug I reported before happens only when importing as new content type. importing into an existing content type works well.

I am seeing group=false in the export even though a field is in a group. Ideally, we would export field group info as well.

I suggest using radio buttons on the initial page of the Export form. Is a bit faster than choosing a type from a select. Similarly, use checkboxes on the second page (field selection). Without looking at code, it appears you are using a multistep form for export. If you want to simplify code, you could add an export link beside each content type in the content types list. Either way is fine. Import should stay as own tab.

minor bug: when i do an export, i see three confusing status messages as below. Maybe a little refactoring would get them out of the export code path:

    * The content type story has been updated.
    * Saved field qty.
    * Saved field Assigned to.

even more minor are these nearly duplicate status messages after successfully adding fields to an existing content type. I guess removing them would make normal field creation more confusing. So lets ignore it:

    * Created field qty.
    * Saved field qty.
    * Created field Assigned to.
    * Saved field Assigned to.

@KarenS - ideally, the non existant node types should not create an error and import proceeds.

I tried exporting a field of type=date.module and it worked!

so, the major outstanding issue is my inability to create a new node type during import. i am using latest HEAD CCK and core. I removed og from the export site but still no luck. (the import site is plain HEAD). i'm gonna bust out the debugger and see whats up. i will hopefully learn from your code that way too.

about removing the status messages: you have a couple of options which may or may not be worth it. You could unset the #submit handlers ... less easy would be to take control of the form processing yourself. basically, you will run through your own drupal_execute() and drupal_process_form() with the intent of skipping drupal_submit_form().

@karen - i do think this is a php5 problem. i am getting much further by changing your cast in content_copy to an array: $param = (array) unserialize($imported_type['parameters']);. i will keep fooling with this, since i still am getting an error (but the types/fields look good).

FYI, drupal_execute() returns an $errors array that you can display as desired.

I will return to this in a few hours. have to assume dad duties for a while.

thanks for looking into fieldgroups. swat those pesky arg() calls away!

FYI, drupal_execute() returns an $errors array that you can display as desired.

Unfortunately, that feature was removed. you still have to call form_get_errors() to see what went wrong. Just one more step, though. :)

@eaton - thanks. the function comment on drupal_execute still talks about errors as return value. i looked at it and i think the return value is now the redirect path. i will patch the docs if you agree.

All I had to do was step away for a few hours and lo and behold, it is importing perfectly now. Just be sure to change that cast to array() as I highlighted earlier. i am doing so because in php5 it is invalid to array_merge() when one of the arguments is an object. maybe try calling drupal_execute() directly instead of through call_user_func_array()? anyway, if array() cast works for you, i think this is RTBC. Fieldgroups would be (big) icing on the cake.

This is some lovely code. Mad props to Eaton and chx and Adrian and Karen. Watching this in the debugger is magical.

@all - my tour in the debugger did take me through a strange section where we "// Save or reset persistent variable values." at bottom of node_type_form_submit(). That section of code ended up doing variable_set('status', 0) and variable_set('promote', 0) which is harmless cruft, except it does cause us to reload the variable cache each time. maybe some content type expert can comment on this code. anyway, not related to this issue.

it is possible that some of yched's suggested simplifications in #2 are still valid. i haven't looked closely.

@karen - I got rid of the status messages during export ... Just use this line in content_copy_form_alter() where you currently add a the record_macro submit handler. defining it this way causes fapi not to add the default submit handler.

$form['#submit'] = array('content_copy_record_macro' => array($form));

i'm ok with just leaving the import status messages as they are.

interesting - subscribing

subscribing

Thanks again Karen for this.

I briefly tested export - will try imports (and the related CCK patch) shortly :-)

- on exporting a content type with no cck fields, i get
Invalid argument supplied for foreach() in content_copy.module on line 196.

- cosmetic :
why not group the two exported definitions :

$macro[type]  = array (
...
);
$macro[fields]  = array (
...
);

in one :

$content_type = array (
  'type' => array(
  ...
  ),
  'fields' => array(
  ...
  ),
);

Or maybe, if you do want to keep both separate :

$type  = array (
...
);
$fields  = array (
...
);

(or anything to get rid of the "$macro" variable, whose reasons for being named that way is strictly internal)

- even more cosmetic (and probably a matter of personal and arguable taste :-) ) : the numeric keys in the 'fields' array sort of seem out of place (see Views exports, or views tables definitions). Maybe those keys could be ditched altogether, or the field (machine) name be used instead ?

I can't reproduce the "no hyphen in key names" issue. but maybe the hyphen in group names should be replaced in the first place, just to make them coherent with the 'field_myfieldname' pattern.
(I can provide the patch and the upgrade function if you like - and if this is agreed)

Sorry,cross posted with your comment 33.

Alright - this numeric keys thing is really minor anyway. I'll see if I can propose a patch later on that removes the keys altogether (like views export does : avoids var_export for arrays and manually build a string like "Array(".$values.")").

I'll propose a patch for hyphens in group names in a separate thread.

Anyway, I guess you're mostly expecting reviews on the import process :-)

Looking real good. Only found 1 bug: field weights look proper in the export but are not recognized during import.

what form has hidden fields weight? why are they hidden? anyway, #default_value sounds just fine. i will try this again soon.

works like a dream.

Still having "Warning : Invalid argument supplied for foreach() in content_copy.module on line 196"
on exporting a content type that has no cck fields (the export seems to happen just fine, though)

Hi,

Where is this module to be found please?
EDIT: I found it on CVS, but not on drupal.org/project/content_copy http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/cck/content...

Will it be placed on drupal.org/project/content_copy ?
Greetings,
Martijn