Tips for Migrating from Storm to PM

With the stormmigrate module planned, it should not be necessary to use any other modules.

To be honest, I think it is a little soon for everyone to try upgrading. It should be a lot smoother when the stormmigrate module is in place, and also there are still bugs / remaining items in the Drupal 7 version that it would be better to fix (so that people don't upgrade and regret doing so!)

However, I think we are getting very close to this point, and so very soon...

In terms of where I plan to spend my time, I intend to try to push over the next week to close the remaining "miscellaneous bugs", as I think these will make a big difference to the perception of the coming alpha - which will be released once these are done.

After the alpha is released, I will focus all of my coding efforts on the migration from Storm (although will continue to respond to other PM issues).

You may have already noticed this, but I've actually started committing the migration module to Storm.

I realised that having too many patches on the go at once for PM would just cause rework later.

My priorities will still be as per #3.

The migration path is now ready to test.

To migrate from a Storm installation to a Project Management installation:

1) Remove the entire Storm module directory.

2) Upgrade your version of Drupal core and run update.php

3) Download Project Management but DO NOT ENABLE.

4) Download Storm 7.x-1.x, and enable "Storm Migrate" (the only module present).

From here on, all will be done for you, and you will end up with Project Management modules enabled on your website (only the modules that were once installed on your previous site).

I'll mark this one as fixed now - but if you notice any bugs (testing of the upgrade path MUCH appreciated) then please open an issue in the Storm issues queue.

Drush Site Upgrade could probably be used for the rest of your site, but it will not automatically pick up on the change.

However, I think the Storm to PM part is quite simple. You could even do 2) above using drush_sup, then do 1) 3) 4).

I understand the eagerness to migrate to PM, but am little worried if this is the correct path. Since we have decided not to create a separate branch for fields api, then we would at later time...

1. Whenever a new field is added, provide a migrate code in update hooks. This would make every single minor update a major update.

2. Provide no migrate path from PM without fields to PM with fields version until everything is converted to fields. After that provide migration procedure using "feeds" api.

Am particularly very interested in providing a "Feeds" migration path for PM since it would enable migration of content from "other project management" softwares too, which provides a csv output of data as well.

Inshort, my major concern is, How do we migrate data when fields are in PM?

@D34dMan

I understand the eagerness to migrate to PM, but am little worried if this is the correct path

I'm keen to explore your concerns so that we can manage this better.

To clarify your points:

1. How are you defining a major update? An additional dependency? A user facing change?

I would respond by saying that we're talking about these changes when we are in an alpha status - alpha means to me that changes may be bigger than you'd normally expect.

2. I would suggest that as we replace custom fields with core fields, there is an upgrade function at each stage that ensures no data is lost.

I'm not sure why we would need Feeds at this stage, but note that by using core Fields, import from CSV would automatically be possible (another advantage of the change to core fields rather than our custom versions). I don't think it makes sense for us to define specific importers unless there are particular formats of CSV (perhaps other PM systems that allow an export) that would be useful.

To clarify on the update function, this is how I would see it going:

node_load ($node);
db_select([custom field from database]);
$core_field = $custom_field_from_database;
node_save ($node);
db_drop_field([custom field]);

Anything I've missed / reasons why this might not work in practice?

I see, feeds adds more complexity that required. I thought it would be easier for creating a CSV export and just import to PM. But i forgot that PM is just storm renamed. So upgrade path should be just like converting any other 6.x to 7.x. Also handling node references would require additional care. Ok no feeds.

Regarding,

node_load ($node);
db_select([custom field from database]);
$core_field = $custom_field_from_database;
node_save ($node);
db_drop_field([custom field]);

Where do we do it?

If it had been just renaming of tables then it would have been accomplished well within php timouts. But i am afraid hunting and saving each nodes would be a different ball game alltogether.

If in hook_update_N, then wouldn't it be a risky business?

I know hook_updates uses drupal's batch_api to run each process. But am not sure if it batch api supports nested batch process. If it does it would be awesome :). Otherwise we would need to provide a different UI based option. This is something that CCK did.

Optimizing the code further (not tested):

$pmnotes = _pmnotes_get_stormnotes_field_vaule_by_querying_database_directly_as_array('LIMIT_NUMBER_PER_BATCH');
foreach ($pmnotes as $nid => $field_value) {
  $node = new stdClass();
  $node->id = $nid;
  $node->bundle = 'pmnote';
  $node->field_name[LANGUAGE_NONE][0]['value'] = $field_value;
  field_attach_insert('node', $node);
  _pmnotes_delete_stormvalue_pmnote($nid); // Not sure where this function should be present.
}

The idea is to save only the field and not the entire node, keeping the node updated date and other things intact.

Also the above code doesn't take into consideration if revision is enabled or not. in that case we would also have to provide revision id.

Or am i trying to make it more complicated o.o?

Yes, hook_update_N can cope with doing the migration in stages if required:

If your update task is potentially time-consuming, you'll need to implement a multipass update to avoid PHP timeouts. Multipass updates use the $sandbox parameter provided by the batch API (normally, $context['sandbox']) to store information between successive calls, and the $sandbox['#finished'] value to provide feedback regarding completion level.

(https://api.drupal.org/api/drupal/modules!system!system.api.php/function...)

Yes, it would be great if we could avoid hitting the last updated time etc. - let's try testing this with one of the real updates - the code posted here looks like it might be the sort of thing that would work.

We've moved a bit from the original topic now - so I'd suggest we move this conversation over to #1915320: Move to a Field API based approach.