Upgrade path from D6 to D7

We are also looking for a solution to this. Has anyone had any experience yet with migrating a D6 Kaltura site to D7?

We have a large number of videos uploaded to our D6 site, and really don't want to have to start from scratch again with D7. At the same time, we are hesitant to go ahead with the upgrade, if it is going to be error-laden.

Does Kaltura upgrade smoothly to D7? Does anyone have experience/advice?

Thanks!

Success!!

Ok, I just finished the upgrade. It was messy, but I learned a lot, and figured others could benefit from this. (This is not for the faint of heart, but will get the job done.)

Just to be clear of the scenario I was in-- we had a drupal 6 site running the Kaltura module, connecting to a Kaltura server (on another machine). We were upgrading the site to D7, and didn't want to lose all of the data stored on the Drupal-side by the Kaltura module. (Kaltura is the one module whose D7 version didn't provide a method to upgrade the datastore from D6.)

Someone else set up the original D6 site, and yet another third party did the bulk of the migration from D6 to D7. My understanding is that it was a very basic Drupal upgrade. They upgraded the rest of the site as per normal, upgraded all the modules, ran update.php, etc. As for Kaltura, they simply replaced the module with the D7 version. The D7 version doesn't attempt to modify the tables. So I inherited a D7 site with Kaltura data that was being stored partly still in the D6 format. (In other words, the database didn't match the code, and so things were a bit corrupt. I had to uncorrupt them.)

In D6, there had been many CCK fields added to the Kaltura Node type, and these fields seemed to convert to the Drupal 7 format correctly during the upgrade.

On the old site, we had Kaltura module 6.x-2.0-beta2 installed. Modules that were enabled:

• Kaltura
• kaltura as CCK field
• Kaltura Media Node
• Kaltura Media Views (though I don't think this one has anything to do with anything)

After the upgrade, it appeared to me that the nodes had *mostly* migrated correctly. The old site had a node type called "Kaltura Media Node" (auto-generated by the module of the same name, I presume), and the machine name for this node type was "kaltura_entry", which I think was also hard-coded in the module. Many extra CCK fields had been added onto this content type. These nodes still existed post-migration, and if you edited them, you could see all the fields were still there.

After the migration, everything seemed to work except:

• This content type no longer showed up in the list of content types.
• you could view one of the migrated nodes, but not edit it
• the media portion of the nodes was not visible.

(Before you proceed, BACK UP YOUR DATABASE! My solution here involves a lot of hacking, and lots of opportunity to mess things up if you're not paying attention.)

To fix the first two points there, all I had to do was this:

    INSERT INTO node_type ( type,            name,            base,           has_title, 
                               title_label,    custom,  modified,  locked, orig_type, module, disabled)                
       VALUES             ('kaltura_entry', 'Kaltura Entry', 'node_content',  1,        
                              'Title',         1,       1,         0,      '',       'node',  0       );

(for some reason, this row didn't get created as part of the data-conversion during the D6->D7 upgrade).

For the last point ("the media portion of the nodes was not visible"), this is where the conversion needs some extra help.

The "node_kaltura" table in the database contains the data for your field. It is not really a "node" table anymore -- that name is a holdover from the D6 version, where the Kaltura Media Node module created that table to store the kaltura info in its own custom node format. Anyhow, that table is still there in D7 with the same name, and it still contains all the kaltura data. However, now you want it to be a *field* that you attach to a node, rather than a *node*, onto which you can add other fields.

You have to go into your "Kaltura Entry" content type in the drupal admin UI, and add a new field to it. Ironically, the node-type that your kaltura nodes were converted to, doesn't have a kaltura field attached to it! So you have to do that: Go in, and add he kaltura media field to the Kaltura Entry node type. I called the field "Media", and gave it the machine name "field_kalmedia". Doing that creates a table called "field_data_field_kalmedia" in the database. This table basically just acts as the linkage between the node and the kaltura field values (which are in the "node_kaltura" table). It contains the nid field as a foreign-key into the node table, and the "field_kalmedia_entryid" as the foreign-key into the node_kaltura table.

However, that linkage table is currently empty, which means all of your Kaltura Entry nodes have nothing in their new Media fields. That data is orphaned over in the node_kaltura table. We need to de-orphan it.

So we need to populate this new linkage table. The D6 version of "node_kaltura" (which should still be how it looks in your database at the moment) contains the 'nid', and 'vid' fields. We'll be dropping these fields for the D7 version, but we need to use that data first. Here is how to move this important legacy data from "node_kaltura" into "field_data_field_kalmedia":

INSERT INTO field_data_field_kalmedia 
        (entity_type, bundle,          deleted,  entity_id,  revision_id, 
            language,   delta,  field_kalmedia_entryid, field_kalmedia_mediatype) 
SELECT  "node",      "kaltura_entry",  0,        nid,        vid,         
            "und",      0,      kaltura_entryId,        kaltura_media_type 
FROM node_kaltura;

Voila! Your nodes now have the media re-attached. You should be able to view them now. (But don't mess around with them, as there are still aspects of the database that are inconsistent with the D7 code.)

Next, I made a custom module to run the following code:

          // Primary key was 'vid' in D6; it's Kaltura EntryID in D7.
          db_drop_primary_key('node_kaltura');
          db_add_primary_key('node_kaltura', array('kaltura_entryId'));

          // nid and vid fields no longer exist in this table in D7.
          db_drop_field('node_kaltura', 'nid');
          db_drop_field('node_kaltura', 'vid');


          // Drop and recreate "Media Date" field.  Has changed from 'date' to 'int'.  
          //   (We didn't have any data in this field anyhow, so this way was easy.  If you have data, you'll
          //    have to modify this to migrate it.)
          db_drop_field('node_kaltura', 'kaltura_media_date');

          $new_fields = array(

              'kaltura_media_date' => array(
                  'description' => 'Date of the image from exif data',
                  'type' => 'int',  // used to be 'date'
                  'not null' => FALSE,
              ),

              'kaltura_created_date' => array(
                  'description' => 'The date and time the entry was created',
                  'type' => 'int',
                  'default' => 0,
                  'not null' => FALSE,
              ),
              'kaltura_updated_date' => array(
                  'description' => 'The date and time the entry was updated',
                  'type' => 'int',
                  'default' => 0,
                  'not null' => FALSE,
              ),
              'kaltura_title' => array(
                  'description' => 'The entry title',
                  'type' => 'text',
                  'default' => NULL,
                  'not null' => FALSE
              ),
              'kaltura_description' => array(
                  'description' => 'The entry description',
                  'type' => 'text',
                  'default' => NULL,
                  'not null' => FALSE
              ),
              'kaltura_puser_id' => array(
                  'description' => 'Uploader user id',
                  'type' => 'int',
                  'default' => 1,
                  'not null' => TRUE
              ),
          );

          foreach ($new_fields as $field_name => $spec) {
            db_add_field('node_kaltura', $field_name, $spec);
            $fields_added[] = $field_name;
          } 

Then I ran this SQL to copy the created/changed dates out of the node (which used to represent the media entry), into the field (which now represents it). This seems like redundant data, but the idea is that nodes containing media information can be created and updated independantly of creating and changing the media field that it contains.

      UPDATE node AS n
        JOIN field_data_field_kalmedia 
                  AS fk ON (n.nid = fk.entity_id)
        JOIN node_kaltura              
                  AS k  ON(fk.field_kalmedia_entryid = k.kaltura_entryId)
      SET k.kaltura_created_date = n.created,
          k.kaltura_updated_date = n.changed,
          k.kaltura_puser_id = n.uid
      WHERE n.type='kaltura_entry';

(I'm not sure what to do with the title and description fields. You could copy in the title and body from the node. We are leaving them blank, for now.)

Then more custom code:

// Add a temporary field, to use during type conversion.
db_add_field('kaltura_notifications', 'received_at_NEW', array('type' => 'int'));

Another SQL query :

      /* Copy old field ('date') to the new one ('int'). */
      UPDATE drup_kaltura_notifications 
         SET received_at_NEW = UNIX_TIMESTAMP(received_at);

Even more custom code:

// Get rid of the old field, then rename the new one to have the name of the one you just deleted.
db_drop_field('kaltura_notifications', 'received_at');
db_change_field('kaltura_notifications', 'received_at_NEW', 'received_at', array('type' => 'int'));

THE END
-----------------------------

Clearly this could be a LOT more elegant. Example, at the very least, the SQL could be in db_query() calls. If someone would care to package this up as some sort of an auto-updater, or even if you can package chunks of this procedure, please post it here.

subscribe - going to attempt #2 from Dan3H and report back here