For those attempting to import calendars a date import supporting the entirety of the date_api (including from date, to date, repeats, etc) is needed.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

zabelc’s picture

FileSize
1.34 KB

I dug into this problem a bit more and I managed to come up with the attached solution. It will let you migrate to and from dates. I've managed to get it to pull in the rrule associated with repeating dates, but that part needs a bit of extra work.

somanyfish’s picture

@zabelc: I'd love some help implementing your code. I found something which looks similar in the beer example:

$arguments = MigrateTextFieldHandler::arguments(array('source_field' => 'excerpt'));
$this->addFieldMapping('body', 'body')
         ->arguments($arguments);

so this is what I tried in my Migrate implementation:

$arguments = MigrateDatetimeFieldHandler::arguments(array('source_field' => 'examdate'));
$this->addFieldMapping('field_exam_date', 'examdate')
      ->arguments($arguments);

This does not successfully import the examdate field to field_exam_date, so any help would be much appreciated.

Thanks!
Lisa

zabelc’s picture

HI Lisa,

This is what my mapping looks like:

    $date_args = MigrateDatetimeFieldHandler::arguments( array('source_field' => 'start_date'), array('source_field' => 'end_date'), array('source_field'=>'rrule'));    
    $this->addFieldMapping('field_event_date', 'start_date')->arguments($date_args);
OliverMcc’s picture

Hi,

I'm having the same problem with importing dates. Does the date have to be provided in a particular format?

thanks in advance

Oliver

OliverMcc’s picture

Think this code from the migrate beer example is potentially relevant regarding the need to preprocess datetime values.

// If you define a prepare() method in your migration, it will be called
  // after the mappings have been applied and before the destination object is
  // saved. The first argument is the destination object as built up using
  // the mappings; the second argument is the raw source data.
  public function prepare(stdClass $account, stdClass $row) {
    // Source dates are in ISO format.
    // Because the mappings above have been applied, $account->created contains
    // the date/time string now - we could also pass $row->posted here.
    $account->created = strtotime($account->created);

I'm struggling to know what to replace $account and $row with, any help much appreciated.

thanks

somanyfish’s picture

@zabelc: Thanks very much for posting your mapping, but when I do something similar it does not work. However, I have successfully imported start dates with this prepare method in my Migration extension class:

public function prepare(stdClass $account, stdClass $row) { 
  $account->field_exam_date = _set_cck_date($account->field_exam_date, $row->examdate);
  $account->field_sent_results_date = _set_cck_date($account->field_sent_results_date, $row->sentresults);
  $account->field_sent_certificate_date = _set_cck_date($account->field_sent_certificate_date, $row->sentcertificate);
} 

and this helper function:

function _set_cck_date($cck_date, $row_date) {
  //Don't set the date if it has the NULL value coming from the database
  if ($row_date != '0000-00-00') {
    $cck_date['und'][0]['value'] = $row_date . 'T04:00:00' ;
    return $cck_date;
}
zabelc’s picture

@OliverMcc all of my dates are coming from a mysql datetime field so I didn't need to use the prepare method. That said, my guess would be that you'll want to do that if you have string formatted dates. I'm afraid that I'm not certain of the particular PHP function.

zabelc’s picture

@somanyfish I'm building on Drupal 7 so I used the field api. It appears that you're using Drupal 6/CCK, so we would both have to do slightly different things to store our fields...

somanyfish’s picture

@zabelc: I'm on D7 too, the cck you see referenced in my code was just what I named my function -- wasn't thinking about the fact that it wasn't a good name in this case!

BTMash’s picture

@zabelc: Does this also handle a multi-date field?

BTMash’s picture

I was trying to work with what @zabelc wrote but was having trouble importing multi-value date fields (want to stress that not the same as date repeat where you can add a new row for the date and I have this so far:

// Define a way to migrate the time stuff
class MigrateDateFieldHandler extends MigrateFieldHandler {
  public function __construct() {
    $this->registerTypes(array('date', 'datestamp', 'datetime'));
  }
  
  static function arguments($start_date = NULL, $end_date = NULL, $rrule = NULL, $separator = NULL) {
    $arguments = array(
      'start_date' => $start_date,
      'end_date' => $end_date,
      'rrule' => $rrule,
      'separator' => $separator,
      'timezone' => variable_get('date_default_timezone', @date_default_timezone_get()),
      'date_type' => 'date',
    );
    return $arguments;
  }
  
  public function prepare(stdClass $entity, array $field_info, array $instance, array $values) {
    if (isset($values['arguments'])) {
      $arguments = $values['arguments'];
      unset($values['arguments']);
    }
    $date_type = $field_info['type'];
    $timezone = $field_info['settings']['tz_handling'];
    $timezone_db = $field_info['settings']['timezone_db'];
    $migration = Migration::currentMigration();
    $destination = $migration->getDestination();
    $language = isset($arguments['language']) ? $arguments['language'] : $destination->getLanguage();
    $separator = FALSE;
    if (!empty($arguments['separator'])) {
      $separator = TRUE;
    }

    $return = array();
    foreach ($values as $value) {
      if ($separator) {
        $date_values = explode($arguments['separator'], $value);
      }
      else {
        //@TODO Figure out base behavior in this scenario.
      }
      $return[$language][] = array(
        'value' => (isset($arguments['start_date']) && isset($date_values[$arguments['start_date']])) ? $date_values[$arguments['start_date']] : '',
        'value2' => (isset($arguments['end_date']) && isset($date_values[$arguments['end_date']])) ? $date_values[$arguments['end_date']] : '',
        'rrule' => (isset($arguments['rrule']) && isset($date_values[$arguments['rrule']])) ? $date_values[$arguments['rrule']] : '',
        'timezone' => $timezone,
        'timezone_db' => $timezone_db,
      );
    }
    return $return;
  }
}

I get the date object come through correctly if you format it in the mapper as:

$generic_date_arguments = MigrateDateFieldHandler::arguments(0, 1, 2, '|');    $this->addFieldMapping('field_content_date_time', 'event_dates')
      ->separator('##')
      ->arguments($generic_date_arguments);

So the 4th parameter would allow you to have a multi date setup working. However, I haven't tested or implemented the scenario where there is no separator (this is where maybe zabelc and my code could converge to fill this aspect) and the other part I haven't figured out is what to do in the scenario where the date field does not have a timezone associated with it (so it should keep the current date of the site). I have a series of dates under that scenario and they kept going back by 1 day until I changed the from YYYY-MM-DDT00:00:00 to YYYY-MM-DDT09:00:00 (where 9 is the offset in time from UTC). Any help in clarifying this would be helpful and probably help us get something out of full date field support out more quickly :)

zabelc’s picture

@BTMarsh I hate to say it but I really don't know anything about multi-valued date fields. One thing that might make the code cleaner is if your version of the static MigrateDateFieldHandler::arguments function took an array of dates rather than a string with a separator. Then the prepare method could branch based on the contents of the arguments array.

I'm not at all sure what would happen if the TZ wasn't supplied. My dates were coming from datetime columns in a table so I believe they ended up with a default timezone. My guess is that they would be the default of the site or UTC. This was where I found the import/rollback feature of migrate 2 so useful. I just kept trying something until it worked...not ideal but not too bad an option in a system as complex as drupal...

mikeryan’s picture

Title: Support for Event-type Date Import » Support for date module in Drupal 7
mikeryan’s picture

Status: Active » Needs work

As a starting point, I've committed a straight forward port of the D6 date support. For Date even more than the other modules supported by Migrate Extras, we really need simpletests to cover all the possibilities - I'll try to setup at least a broad framework before DrupalCon (no guarantees).

BTMash’s picture

@mikeryan, I saw what you posted (and I see that you would provide an array of values); I was wondering how someone would set up a date migration with from (value) and to (value2) fields. I had some ideas on how mine could be changed so it doesn't use a separator (and I agree with you, @zabelc - the migrate rollback saved me lots of times :))

FrequenceBanane’s picture

I am also interested in in the possibility to do this without a seperator, but I cannot figure how to do it !
Moreover, how could $date_values[$arguments['start_date']] (associative array, since $arguments['start_date'] is a string) be working after $date_values = explode($arguments['separator'], $value) (explode creates numeric array, doesn't it ?).

So... subscribe

fangel’s picture

Mike, the straight-forward D6 port you talk about in #14 has some faults in it..

You start the prepare-method with

  public function prepare(stdClass $entity, array $field_info, array $instance, array $values) {
    $language = $this->getFieldLanguage($entity, $field_info, $arguments);
    ...
  }

But it should really be

  public function prepare(stdClass $entity, array $field_info, array $instance, array $values) {
    $migration = Migration::currentMigration();
    if (isset($values['arguments'])) {
      $arguments = $values['arguments'];
      unset($values['arguments']);
    }
    else {
      $arguments = array();
    }
    $language = $this->getFieldLanguage($entity, $field_info, $arguments);
    ...
  }

(Okay $migration = .. ain't really needed, but it could be later on)

The reason is pretty obvious - in the code as it is now, it uses the variable $arguments without even having set it somewhere first.

-Morten

agentrickard’s picture

Status: Needs work » Needs review
FileSize
581 bytes

@fangel's approach in #17 worked for my simple migration to a single date field. Patch attached.

I'm just importing to a single (no repeats, 1 row, Year - Month - Date), and I can now pass:

// Where $first is a UNIX timestamp
$row->broadcast = date('Y-m-d', $first) . 'T00:00:00';

Where the mapping is:

    $this->addFieldMapping('field_story_broadcast_date', 'broadcast')
      ->description(t('Computed field. See prepareRow()'))
      ->issueGroup(t('DNM'));
agentrickard’s picture

FileSize
1.44 KB

The original patch throws an undefined index error. $instance['type'] is now $instance['widget']['type'].

New patch assumes timestamp unless overridden.

agentrickard’s picture

FileSize
1.44 KB

That last patch is not correct. $value in my case (type = 'date_select') should be kept as is. This patch restores the normal behavior.

FrequenceBanane’s picture

It would have been greate if it had been committed to the new stable release, since it makes it unstable (it always throws errors about this undefined $arguments value whenever I try to migrate dates !)

agentrickard’s picture

Without proper reviews, patches don't get in. The patch is #20 is likely incomplete.

Complaining about the process is not helpful. Reviews are.

mikeryan’s picture

Sorry I haven't been giving migrate_extras much attention lately - there will be a beta of Migrate 2.1 released soon, then I'll turn my attention to catching up on the migrate_extras queue.

BTMash’s picture

I'm still confused on how this method captures the various other fields (rrule, timezone, timezone_db, value2) that zabelc processes (and mine used the not-so-good separator method). zabelc's suggestion on the array already having all that info in there made sense but maybe I'm missing something.

agentrickard’s picture

Status: Needs review » Needs work

I don't believe it does ATM, so Needs Work.

mikeryan’s picture

I've committed the patch in #20 (eyeballed but untested) - I finally have time for migrate_extras, my first priority now is building out simpletests (starting with date support).

mikeryan’s picture

OK, for Drupal 7 I've committed support for all three date field types, accepting timezones, To values, and rrules as arguments. There's also a new migrate_extras_examples module (implemented as a Feature) to help guide you to using these fields. I'm still working on simpletests, but this should get people going.

FrequenceBanane’s picture

Waiting for the next dev, then :-)

mfb’s picture

don't see migrate_extras_examples so I guess it's not yet pushed..

mikeryan’s picture

Indeed, I committed and forgot to push. It's there now.

The example feature module is actually migrate_extras_date - migrate_extras_examples is the folder that contains it (and will contain more examples for other contrib destinations).

mikeryan’s picture

Version: 7.x-2.x-dev » 6.x-2.x-dev
Component: CCK » Date
Status: Needs work » Patch (to be ported)

OK, I've committed (and pushed:-) tests for date import, plus tweaks to the example to show import of timezones. Now to backport to D6...

BTMash’s picture

Hmm, I'll need to update some of my docs; I'm a bit confused by how the date would be imported from the db if you have a table with value and value2 (let's say the columns are called 'source_value_from' and 'source_value_to' that as well in that table). Would the code be something like:

$this->addFieldMapping('field_date_range', 'source_value_from')
->arguments(array('to' => array('source_field' => 'source_value_to')));

And how does it handle multiple data values (if we were gathering the multiple dates in the prepareRow() implementation (would we have $current_row->source_value_from = array() be an array with a bunch of values and then $current_row->source_value_to be an equally sized array with its necessary values?

mikeryan’s picture

Yes, that's precisely the code - see the examples in migrate_extras_date.migrate.inc.

And... Good point, a multiple-value date field with more than From dates isn't going to work as the code is now. The date field handler could recognize that $current_row->source_value_to is an array and iterate it alongside the primary values, but perhaps it would be better to use the same JSON technique as filefields. What do you think?

BTMash’s picture

I'll need to get myself more familiar with the JSON technique introduced in filefields but from what I saw, I think that would be the best way to go (the argument handler could either just handle the timezone or even that could come into the json fields).

mikeryan’s picture

Status: Patch (to be ported) » Needs work

D6 version committed. So, back to D7 to handle the args as JSON...

mikeryan’s picture

Version: 6.x-2.x-dev » 7.x-2.x-dev
mfb’s picture

Version: 7.x-2.x-dev » 6.x-2.x-dev
Status: Needs work » Patch (to be ported)
$this->addFieldMapping('field_date_range', 'source_value_from')
->arguments(array('to' => array('source_field' => 'source_value_to')));

didn't work for me. I got
"strtotime() expects parameter 1 to be string, array given File migrate/includes/base.inc, line 949". Either I did something wrong or the raw array is getting passed to MigrationBase::timestamp()?

mfb’s picture

Version: 6.x-2.x-dev » 7.x-2.x-dev
Status: Patch (to be ported) » Needs work

argh (why doesn't d.o. warn you when the issue has been commented on since the comment form was created)?

mfb’s picture

Ah it looks like this is the error message if source_value_to is NULL. In this case, the source_field mapping is ignored (although seems like Migration::applyMappings() should support NULL values here?) I fixed in prepareRow() by setting source_value_to = source_value_from if source_value_to is NULL.

mikeryan’s picture

@mfb: Looks like you've uncovered a flaw in applyMappings(), when there's a value in the source_field it normally overwrites the array('source_field' => 'my_field_name') with the value from my_field_name, but when NULL it leaves that array in there. I've added an issue to the Migrate queue: #1161482: Use of source_field in arguments fails with NULL source value.

Thanks!

mikeryan’s picture

Status: Needs work » Fixed

OK, the JSON approach to date properties is committed for D6 and D7 - I've added examples and tests for the JSON and for multiple values. I'm closing out this issue - any specific bugs found in the date support should be opened up as distinct issues.

Thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

grendzy’s picture

Priority: Normal » Minor
Status: Closed (fixed) » Needs work

Could someone please update the migrate_extras project page? "Date" is still listed as built-in, instead of supported by this module. Thanks!

mikeryan’s picture

Status: Needs work » Closed (fixed)

Built-in is correct - migration support is built-in to the Date module.

grendzy’s picture

Ah - sorry. I was confused by MigrateDateFieldHandler vs DateMigrateFieldHandler. :O