Migrating from Movable Type

The Import TypePad module should be able to handle the Movable Type export format.

Here's a few scripts for updating Movable Type to a 4.7.x or 5.x Drupal install. These scripts will let you migrate your Movable Type blog to Drupal with comments and all settings intact. Feel free to tweak the code as you like.

NOTE: To use this you will need to be running PHP 5. The simplexml commands are not supported on PHP 4.

Overview of the steps involved:

Install and "rebuild" MT Theme
Create a drupal page (ie: hit the path 'node/add/page') with supplied PHP snippet
Patch comment.module to allow comments to get proper dates (you can remove this patch after you finish migrating)
Done!

First, follow the section of this guide titled "Extract Movable Type content as xml" except use the MT template below:

<?xml version="1.0" encoding="UTF-8"?>
<items>
  <MTEntries lastn="1000" sort_order="ascend">
  <item about="<$MTEntryLink$>">
    <title><$MTEntryTitle encode_xml="1"$></title>
    <description><$MTEntryBody encode_xml="1"$></description>
    <link><$MTEntryLink$></link>
    <subject><$MTEntryCategory encode_xml="1"$></subject>
    <creator><$MTEntryAuthor encode_xml="1"$></creator>
    <date><$MTEntryDate format="%Y-%m-%dT%H:%M:%S"$><$MTBlogTimezone$></date>

    <comments>
      <MTComments lastn="1000" sort_order="ascend">
      <comment>
        <author><$MTCommentAuthor default="Anonymous" encode_xml="1"$></author>
        <email><$MTCommentEmail encode_xml="1"$></email>
        <homepage><$MTCommentURL encode_xml="1"$></homepage>
        <date><$MTCommentDate format="%Y-%m-%dT%H:%M:%S"$><$MTBlogTimezone$></date>
        <body><$MTCommentBody convert_breaks="0" remove_html="1" encode_xml="1"$></body>
        <title><$MTCommentTitle encode_xml="1"$></title>
      </comment>
      </MTComments>
    </comments>

  </item>
  </MTEntries>
</items>

Once your finished with that, rebuild the template, and download it (ie: enter something like http://yourblog.com/drupal.rdf in your web-browser and do a "Save As").

Put the drupal.rdf file somewhere sensible. My code assumes you're putting it in the scripts folder of your Drupal install. If you put it somewhere else then be sure to edit the PHP code.

Now, we're ready to import it into Drupal. To do this, create a new node of type 'page' on your site and give it the input format type of PHP (you might want to unpublish it as well so people don't stumble upon it).

The content of this new page should be the following code:

<?php
  /**
   * This snippet will scrape the 'drupal.rdf' MT XML export file and create
   * nodes and comments from the content.
   *
   * @author James Andres
   * @version 2007-03-17
   */
  if ($_GET['start'] == 1) {
    // ****** Change the 'scripts/drupal.rdf' line to the location of your MT XML export ******
    $xml = simplexml_load_file('scripts/drupal.rdf');
    global $user;

    foreach ($xml->item as $item) {
      $comment_count = count($item->comments->comment);

      echo "Saving <strong>$item->title</strong> with $comment_count comments.<br />\n";

      // Create a node
      $node = (object) array();
            node_object_prepare($node);
            $node->type = 'blog';
      $node->status = 1;
      $node->promote = 1;
      $node->uid = $user->uid;
      $node->format = 3; // Full HTML
      $node->created = strtotime($item->date);
      $node->updated = $node->created;
      $node->path = str_replace('http://www.davidrdgratton.com/', '', $item->link);
      if ($item->subject) {
        // Save the tags
        $term = taxonomy_get_term_by_name($item->subject);
        $term = current($term);
        $node->taxonomy[] = $term->tid;
        $node->taxonomy_term = (string) $item->subject;
      }

      $node->title = $item->title;
      $node->body = $item->description;
      $node->description = $item->description;

      node_save($node);

      foreach ($item->comments->comment as $comment) {
        $edit = array();
        $edit['pid'] = 0;
        $edit['nid'] = $node->nid;
        $edit['uid'] = (
          strtolower($comment->author) == strtolower($user->name) ?
          $user->uid :
          0
        );
        $edit['timestamp'] = strtotime($comment->date);
        $edit['name'] = $comment->author;
        $edit['subject'] = $comment->title;
        $edit['comment'] = $comment->body;
        $edit['format'] = 2;
        $edit['mail'] = $comment->email;
        $edit['homepage'] = $comment->homepage;
        comment_save($edit);
      }
    }
  } else {
    echo l('start', $_GET['q'], array(), 'start=1');
  }
?>

Hit save. BUT before you hit the 'start' link you might want to patch the comment.module.

How to patch the comment module:

If you're on Drupal 4.7 or 5.0, find your comment.module and look for the line $edit['timestamp'] = time();. Replace that line with the following code ...

  if (!$edit['timestamp']) {
    $edit['timestamp'] = time();
  }

Last but not least, log in as the user you want to be the "author" of all the blog posts you migrate (ie: In my case that user was 'James'). If that author matches the author of your MT blog the script will detect that and act accordingly.

Finally, you should be able to hit the 'start' link on the page you created with all that PHP code in it and the migration will run.

mt2drupal

[jseng: mt2drupal is another trick you can use to migrate MT to Drupal. It is written in perl as an MT plugin utilizing MT libraries to extract from the database and then feed it into the MySQL database directly. It will import:

all your bloggers
all your defined categories
all your entries including body, excerpt, extended (in 4.4.1 & CVS, it is stored as bodyextended and in D4B formatting rules are also preserved)
all your comments (with anonymous support, in CVS and D4B)
all incoming trackbacks (stored as comments)
all your outgoing trackbacks (D4B only)
all your trackbacks trackers (D4B only)
keep all your old archives as url_alias, including your RSS feeds so permalink is preserved

» Login or register to post comments

Importing your MT categories into Drupal

aspankie - June 6, 2007 - 16:19

Thank you for providing these scripts. They were SO HELPFUL! Since my MT posts had multiple categories, I had to take the scripts a step further and update them to import the node's taxonomy from the MT categories. It works great and thought I should share. Here is what I did to accommodate MT posts with multiple categories.

In your Movable Type drupal.rdf template as mentioned, change:

<subject><$MTEntryCategory encode_xml="1"$></subject>

to:

<MTEntryCategories>
   <subject><$MTCategoryLabel encode_xml="1"$></subject>
   </MTEntryCategories>

That will put all of your categories into the rdf file rather than just one.

Then, to loop through them and create them as Drupal categories change the following in your MT import php page in Drupal:

Change:

if ($item->subject) {
        // Save the tags
        $term = taxonomy_get_term_by_name($item->subject);
        $term = current($term);
        $node->taxonomy[] = $term->tid;
        $node->taxonomy_term = (string) $item->subject;
      }

To:

if ($item->subject) {
        // Save the tags

	 foreach ($item->subject as $subject) {
	        $term = taxonomy_get_term_by_name($subject);
	
		//if the term already exists, update the node
		if($term){
		        $term = current($term);
		        $node->taxonomy[] = $term->tid;
		        $node->taxonomy_term = (string) $subject;
	
	        }
		else {
                 //else create a new category and update the node      
			$newEntry = (string) str_replace("'", "''", $subject);
			db_query("INSERT INTO {term_data} (name, vid) VALUES ('$newEntry', '4')"); 
			$newtermID = mysql_insert_id();
			db_query("INSERT INTO {term_hierarchy} (tid, parent) VALUES ('$newtermID', '0')"); 
		        $node->taxonomy[] = $newtermID;
		        $node->taxonomy_term = (string) $subject;
		}
	   }
	}

The '4' in the insert is just the type I used for my categories. You can switch out this number as you need to. There might be a function you could call to insert the new category to make things easier, but this is the way I went.

Hope this helps someone.

Abigail

Update taxonomy module to fix after migrating

aspankie - June 28, 2007 - 02:30

I am on 4.7.3 and was unable to reproduce the problem you describe. I have new categories from the migration, pre-migration categories and post-migration categories and all seem to be co-existing well. At times, I did get the error message you are referring to during migration test runs, but since all the categories got created without issue, so I didn't worry about it. I just verified and I am able to create new terms in all vocabularies. However, looking at the database I see it created them using numbers lower than the highest migrated number because there were open slots. So I started wondering what will happen when the gap spaces are filled (I had records 60 - 102) open. So now I am so glad you pointed this out!!

I did some playing around and was about to modify the taxonomy module to simply auto-increment the table instead of looking for the tid of the last Drupal-created category (which is what it's doing right now for some reason).

In your taxonomy.module file, go to lines 429 - 430 (it may be different in yours).

Look for the lines:

    $edit['tid'] = db_next_id('{term_data}_tid');
    db_query("INSERT INTO {term_data} (tid, name, description, vid, weight) VALUES (%d, '%s', '%s', %d, %d)", $edit['tid'], $edit['name'], $edit['description'], $edit['vid'], $edit['weight']);

And change them to:

    db_query("INSERT INTO {term_data} (name, description, vid, weight) VALUES ('%s', '%s', %d, %d)",  $edit['name'], $edit['description'], $edit['vid'], $edit['weight']);   
    $edit['tid'] = mysql_insert_id();

This will stop the module from looking at the database for the last non-migrated category and will just auto-increment it instead. I just tested it and it works beautifully.

For 5.1 It should be changed to:

$form_values['tid'] = db_next_id('{term_data}_tid');
db_query("INSERT INTO {term_data} (name, description, vid, weight) VALUES  ('%s', '%s', %d, %d)", $form_values['name'], $form_values['description'], $form_values['vid'], $form_values['weight']);
$form_value['tid']= mysql_insert_id();

Note: Thanks NCXT!

I will post back if there is a better way to prevent this in the first place. I have to find some time to look into it.

Hope this helps you!

Abigail

No need for sql statements

ivanb - November 9, 2007 - 01:02

Just use the Drupal API. The original code did not update the sequences table.

Change:

db_query("INSERT INTO {term_data} (name, vid) VALUES ('$newEntry', '4')");
$newtermID = mysql_insert_id();
db_query("INSERT INTO {term_hierarchy} (tid, parent) VALUES ('$newtermID', '0')"); 

To:

$term = array('vid' => '4', 'name' => $newEntry);
$status = taxonomy_save_term($term);
$newtermID = $term['tid'];

Use can check the status variable to verify the term has been added correctly.

mt2Drupal Script for Drupal 4.4.1

dgorton - August 6, 2007 - 16:29

The mt2Dupal script sounds interesting. In case that link disappears someday, though, it should be noted that it was written for Drupal 4.4.1 (circa mid 2004). The last comments on that page are from mid 2005 and describe import issues to Drupal 4.6.

So - unless that code is updated pretty significantly it's probably a safe guess that it won't work for recent versions of Drupal.

Drew Gorton
_{Gorton Studios

Some of our Drupal Sites}

Some Troubleshooting Tips

dgorton - August 6, 2007 - 23:24

First - big thanks to everyone above. This has been very helpful. For anyone else attempting this, you may run into a few issues. Here are some that I found:

Required Modules (this may or may not be obvious to others)

* Blog (required as currently written - more below)
* Comments
* Taxonomy (if automatically adding subjects / taxonomies )

Required Access

The user running the import must have permissions to create comments.

Correct URLs / Paths for Your Site

$node->path = str_replace('http://www.davidrdgratton.com/', '', $item->link);

should be changed to reflect your blogs location - e.g.

$node->path = str_replace('http://www.my-old-mt-blog.url/', '', $item->link);

or - if you want a slightly deeper URL (like putting all your blogs into a subdirectory):

  $node->path = str_replace('http://www.my-old-mt-blog.url/', '', 'blogs/'.$item->link); // (replace 'blogs/' with your dir name)

Want to Ix-Nay the 'Published to Home Page'?

Change:

$node->promote = 1;

$node->promote = 0;

CCK type instead of Blog

If you don't want to use the built-in blog type and module, change

$node->type = 'blog';

$node->type = 'my_cck_type_name';

Timeout Issues

If you're having trouble importing all your entries, you can toss in a loop. This is a quick, dirty hack -- a real solutiuon would be to actually flag the status of each node and/or check for the existence of a duplicate before importing.
Replace:

 if ($_GET['start'] == 1) {
    // ****** Change the 'scripts/drupal.rdf' line to the location of your MT XML export ******
    $xml = simplexml_load_file('scripts/drupal.rdf');
    global $user;

    foreach ($xml->item as $item) {
      $comment_count = count($item->comments->comment);

      echo "Saving <strong>$item->title</strong> with $comment_count comments.<br />\n";

with

  $row_start = 1;
  $items_per_run = 50; // arbitrary number - adjust to fit your needs
  if ( isset($_GET['rowNum']) ) {
    // ****** Change the 'scripts/drupal.rdf' line to the location of your MT XML export ******
    $xml = simplexml_load_file('scripts/drupal.rdf');
    global $user;
		
    $item_number = 1;
    $row_start = $_GET['rowNum'];

    foreach ($xml->item as $item) {
      $item_number++;
      if ( $item_number <= ($row_start ) ) {
        continue;
      }
      if ( $item_number > ($row_start + $items_per_run) ) {
        break;
      }


      $comment_count = count($item->comments->comment);

      echo "Saving <strong>$item->title</strong> with $comment_count comments.<br />\n";

And, then later, replace:

  } else {
    echo l('start', $_GET['q'], array(), 'start=1');
  }
?>

with

  }
  $next_row_start = $row_start + $items_per_run;
  if ( isset($_GET['rowNum']) ) {
    echo l( 'continue import: rows ' . $next_row_start . ' to ' . ( $next_row_start + $items_per_run - 1), $_GET['q'], array(), 'rowNum='. $next_row_start );
  }
  else {
    echo l( 'start import: rows ' . $row_start . ' to ' . ($next_row_start - 1), $_GET['q'], array(), 'rowNum=1' );
  }

Again, though, this is NOT pretty in addition to being horribly insecure and yucky. But, it's a starting point. And, depending on the scope and results of my current tests, I may post an updated version later. Regardless, though, I hope this prevents headaches for someone out there...

Drew Gorton
_{Gorton Studios

Some of our Drupal Sites}

Another Troubleshooting Tip - Extended Entries

jenna.tollerson - March 17, 2008 - 00:58

First off, this page and all the above comments have been extremely helpful in migrating my site to Drupal, so thanks everyone!

If you have used the "Extended Entry" field in any of the entries on your Movable Type site, the drupal.rdf code above will not capture that text. The fix is easy, however.

Change this line:
<description><$MTEntryBody encode_xml="1"$></description>
to this:
<description><$MTEntryBody encode_xml="1"$><MTIfNonEmpty tag="EntryMore" convert_breaks="0"><$MTEntryMore encode_xml="1"$></MTIfNonEmpty></description>

Before uploading drupal.rdf to your Drupal site, open it up in a text editor do a quick find and replace to remove the ]]><![CDATA[ text between the two values.

Updated information for MT4 and Drupal 6.x

woytek - May 30, 2008 - 19:56

I have a blog that is running under MT4, and I've been working on some conversion scripts to get it moved over to Drupal 6. Through the generosity of the people who created these pages and added their comments, I was able to get things partially working. There are some oddities in MT4 and API changes in Drupal 6 that were preventing some things from working.

First, MTCommentTitle apparently doesn't exist in MT4. I left that field in the output from the template, but it is blank. I also use free-form tagging on my blog, but not categories, so I added the information to output the list of tags for each entry. In the import code, I used the node API to add the tags to the taxonomy of the node. I also had to work-around some deprecated calls being used in the creation of the node, and moved some things to use the published node API (thanks to "blogapi_blogger_new_post" for that code example).

First up, here is the updated template that I used to generate the XML in MT. Note that I increased the number of entries to 10k, modified the encoding in the XML header to list whatever encoding MT is actually using (helps with odd characters), added the tags element, and removed the call to the non-existent MTCommentTitle template tag.

<?xml version="1.0" encoding="<$MTPublishCharset$>"?>
<items>
  <MTEntries lastn="10000" sort_order="ascend">
  <item about="<$MTEntryLink$>">
    <title><$MTEntryTitle encode_xml="1"$></title>
    <description><$MTEntryBody encode_xml="1"$></description>
    <link><$MTEntryLink$></link>
    <subject><$MTEntryCategory encode_xml="1"$></subject>
    <tags><MTEntryTags glue=", "><$MTTagName encode_xml="1"$></MTEntryTags></tags>
    <creator><$MTEntryAuthor encode_xml="1"$></creator>
    <date><$MTEntryDate format="%Y-%m-%dT%H:%M:%S"$><$MTBlogTimezone$></date>

    <comments>
      <MTComments lastn="1000" sort_order="ascend">
      <comment>
        <author><$MTCommentAuthor default="Anonymous" encode_xml="1"$></author>
        <email><$MTCommentEmail encode_xml="1"$></email>
        <homepage><$MTCommentURL encode_xml="1"$></homepage>
        <date><$MTCommentDate format="%Y-%m-%dT%H:%M:%S"$><$MTBlogTimezone$></date>
        <body><$MTCommentBody convert_breaks="0" remove_html="1" encode_xml="1"$></body>
        <title></title>
      </comment>
      </MTComments>
    </comments>

  </item>
  </MTEntries>
</items>

Next up is the modified PHP code for the importer page. I moved all of the variables that one might have to set to the top and commented them appropriately. I also used some of the hints in blogapi_blogger... to configure some aspects of each node according to the defaults that are set in Drupal for that node type. For instance, this controls if comments are allowed, type of text in the node, etc. Tags are processed and saved during node creation. The date format accepted in the node API seems to be different from the scripts on this page, too--the existing scripts changed this to a timestamp, which did not play nicely with the node API. I pulled that code so that the date fields are filled-in with a qualified date entry.

<?php
/**
 * This snippet will scrape the 'drupal.rdf' MT XML export file and create
 * nodes and comments from the content.
 *
 * Original by James Andres.
 * Updates by Jonathan Woytek [woytek (at) dryrose (dot) com].
 *
 * @author James Andres/Jonathan Woytek
 * @version 2008-05-30
 */

if ($_GET['start'] == 1) {
        //******Change the 'drupal.rdf' line to the location of your MT XML export ******
        $xml = simplexml_load_file('drupal.rdf');
        // Set oldurl to the old URL to your mt blog (fully-qualified).
        $oldurl = 'http://www.jennyandjonathangetmarried.com/lifeblog/';
        // Set tagVid to the vocabulary ID for your freeform tag taxonomy.
        $tagVid = 1;
        // Set nodeType to the type of entry you want to create (page, story, blog, etc.).
        $nodeType = 'blog';

        global $user;

        foreach($xml->item as $item) {
                $comment_count = count($item->comments->comment);

                //Create a node
                $node = (object) array();
                $node->type = $nodeType;

                $node_type_default = variable_get('node_options_'.$node->type, array('status', 'promote'));
                $node->status = 1;
                $node->promote = in_array('promote', $node_type_default);
                $node->comment = variable_get('comment_'.$node->type, 2);
                $node->revision = in_array('revision', $node_type_default);
                $node->uid = $user->uid;
                $node->name = $user->name;
                $node->format = FILTER_FORMAT_DEFAULT;
                $node->date = $item->date;
                $node->updated = $node->date;
                $node->path = str_replace($oldurl, '', $item->link);
                $node->title = $item->title;
                $node->body = $item->description;
                /*
                 * save the tags.
                 */
                if ($item->tags) {
                        $node->taxonomy['tags'] = array($tagVid => $item->tags);
                }

                /*
                 * Save the node using the API.
                 */
                node_invoke_nodeapi($node, "mtimport");
                $newnode = node_submit($node);
                node_save($newnode);

                /*
                 * Save the comments.
                 */
                foreach($item->comments->comment as $comment) {
                        $edit = array();
                        $edit['pid'] = 0;
                        $edit['nid'] = $newnode->nid;
                        $edit['uid'] = (
                                        strtolower($comment->author) == strtolower($user->name) ?
                                        $user->uid :
                                        0
                                );
                        $edit['timestamp'] = strtotime($comment->date);
                        $edit['name'] = $comment->author;
                        // note:  mt4 apparently doesn't have the MTCommentTitle attribute
                        $edit['subject'] = $comment->title;
                        $edit['comment'] = $comment->body;
                        $edit['format'] = 2;
                        $edit['mail'] = $comment->email;
                        $edit['homepage'] = $comment->homepage;
                        $cid = comment_save($edit);
                }
                echo "Saved <strong>$item->title</strong> with $comment_count comments.<br />\n";

        }
} else {
        // I had problems with the l() call below, so I replaced it.
        echo "<a href=\"/".$_GET['q']."&start=1\">start</a>";
        //l('start', $_GET['q'], array(), 'start=1');
}
?>

Note that this was reformatted with indent(1), which might bother some folks. Feel free to reformat at will.

Also note that I don't do anything with categories. I don't use them on my current blog anymore, so I decided to drop them entirely when I move the blog and site to Drupal. Some slight modifications to the code above to handle tags would probably handle categories as well. I think one would need to essentially copy the code, change "taxonomy['tags']" to just "taxonomy[]", set the vid appropriately, and move on. If you have multiple categories on posts, then one might also need an iterator to step through them and add them all. Since these categories are being added automatically, one could use "taxonomy['tags']" with a separate vid, include all of the categories in one line with commas separating them, and use exactly the same code. This should add all of the categories automatically to your selected vocabulary. That is left as an exercise for someone who actually uses categories, though. :)

Hopefully this helps others, and thanks to those who posted here who helped me to get this far!

jonathan

Migrating from Movable Type

mt2drupal

Importing your MT categories into Drupal

categories

Update taxonomy module to fix after migrating

No need for sql statements

mt2Drupal Script for Drupal 4.4.1

Some Troubleshooting Tips

Required Modules (this may or may not be obvious to others)

Required Access

Correct URLs / Paths for Your Site

Want to Ix-Nay the 'Published to Home Page'?

CCK type instead of Blog

Timeout Issues

Another Troubleshooting Tip - Extended Entries

Updated information for MT4 and Drupal 6.x

Developing for Drupal

Handbook license

Quick links

User login

Contributor links