drush command: "drush migrate-import --all" fails

I have an import with status "importing" ...I tried forcing it to continue by running the above drush command and got following error:

Notice: Undefined property: WordpressMigrateStreamWrapper::$? in /mnt/www/html/xxx/docroot/includes/stream_wrappers.inc on line 512

Fatal error: Call to undefined function ??() in /mnt/www/html/xxx/docroot/includes/stream_wrappers.inc on line 512

I am on the Acquia network.

Am I doing the right thing with drush to get it to continue importing? It got stuck on 47 attachment out of 6k.

Comments

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

First off, if you see a migration stuck in the "Importing" state, you can clear the status to unstick it:

drush mrs FoobarBlogEntry

What's at (and around) line 512 of /mnt/www/html/xxx/docroot/includes/stream_wrappers.inc? In my local install, that line is in the middle of a comment, so you must be on a different D7 version than I am.

vacholino’s picture

"First off, if you see a migration stuck in the "Importing" state, you can clear the status to unstick it:"

I don't want to clear the status of "importing" because it's not finished importing yet? Blog entries, categories and tags are all imported perfectly. I am happy there:) But the problem is all the attached files. It's about 10 000 attached files and it runs until 1009 and stops.

I get following error when I try command: "drush mi --all"

There is already an active process on VertigoGraphicContentAttachment
Drush command terminated abnormally due to an unrecoverable error.       [error]

Notice: Undefined property:  WordpressMigrateStreamWrapper::$? in /mnt/www/html/xxx/docroot/includes/stream_wrappers.inc on line 512

Fatal error: Call to undefined function ??() in /mnt/www/html/xxx/docroot/includes/stream_wrappers.inc on line 512

It says an import is in progress, but it has not made any progress for days. How do I make it continue downloading attachments? Is there a documentation page will all the drush migrate commands and some details on how they work?

Thank you once again for a great module and the support you are giving!!

vacholino’s picture

One more question, what does "drush mrs FoobarBlogEntry" exacly do?

It just sets the status to "complete" even though it's not completed? can you be a little more detailed on what it does? Maybe it's just what I need if I want to stop it from importing so I can run "drush mi --all" to get it to run again...

Also, what triggers it to continue importing in the backend? Cron? or something else? please advice.

mikeryan’s picture

So, a little background... If your import takes longer to run than the maximum execution time for a page on your site, it will run the rest of it in the background using drush. While each step of it is running, it is put into the Importing status to indicate that it's busy, and prevent another operation from happening at the same time on that step. Now, if something happens that it just plain dies, it will be stuck in that Importing status and won't be able to continue. You can tell this if you repeatedly do a drush ms (migrate-status) or refresh the dashboard in the UI and the number of imported items doesn't change. When you're in this state, you get that "There is already an active process" message. In this case, you do a drush mrs (migrate-reset-status), which sets the status back to Idle. If you then run drush mi --all, it will be able to run the import, and it picks up right where it left off.

The stream wrapper message is entirely different (and may be the reason the migration got "stuck" in the first place), and it's a mystery to me. For one thing, it's show "$?" and "??" instead of the actual function names involved. For another, when I look in that region of my copy of stream_wrappers.inc, there's nothing there I can possibly imagine calling an undefined function. If I could see what's in your copy of stream_wrappers.inc in the area of line 512, that might give me some hint.

Thanks.

vacholino’s picture

511: public function stream_flush() {
512: return fflush($this->handle);
513: }

That's from my stream_wrappers.inc

hope that helps and thanks for helping with the migrate being stuck. I am going to try your suggestions right now.

vacholino’s picture

Using ''drush mrs'' o reset the import and running "drush mi --all" totally fixed my problem. Is there a page where I can find out about all the migrate dependencies? It seems like the publish dates are not getting mapped over correctly and user profiles (I have them with identical name in drupal) are not getting mapped over. Is this because I am missing dependent modules?

thanks

mikeryan’s picture

Glad you were able to get the migration working. I have no explanation for the stream wrapper error you were seeing, but it seems you're past that now.

Required modules are listed as dependencies in the .info files, whatever issues you see are not a matter of module dependencies.

By "publish dates" I'm assuming you mean the changed date on the nodes (the date that appears in the listing at admin/content)? I can see that that is not being set correctly, although the created date (the one that is displayed on the node page itself) appears correct. I will open a separate issue for the changed date.

Users are not part of the Wordpress export file, and thus are not migrated.

vacholino’s picture

I am re-running the import becuase it got stuck.. and now I am getting the following errors:

/mnt/www/html/xxx/docroot/includes/common.inc:6975)
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry                                                         [error]
'public://files/2009/08/titi_20_dylux-15-copy.jpg' for key 2: INSERT INTO {file_managed} (uid, filename, uri,
filemime, filesize, status, timestamp, type) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1,
:db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5,
:db_insert_placeholder_6, :db_insert_placeholder_7); Array
(
    [:db_insert_placeholder_0] => 129
    [:db_insert_placeholder_1] => titi_20_dylux-15-copy.jpg
    [:db_insert_placeholder_2] => public://files/2009/08/titi_20_dylux-15-copy.jpg
    [:db_insert_placeholder_3] => image/jpeg
    [:db_insert_placeholder_4] => 274631
    [:db_insert_placeholder_5] => 1
    [:db_insert_placeholder_6] => 1330908310
    [:db_insert_placeholder_7] => image
)
 (/mnt/www/html/xxx/docroot/includes/common.inc:6975)
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry                                                         [error]
'public://files/2009/08/titi_20_dylux-16-copy.jpg' for key 2: INSERT INTO {file_managed} (uid, filename, uri,
filemime, filesize, status, timestamp, type) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1,
:db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5,
:db_insert_placeholder_6, :db_insert_placeholder_7); Array
(
    [:db_insert_placeholder_0] => 129
    [:db_insert_placeholder_1] => titi_20_dylux-16-copy.jpg
    [:db_insert_placeholder_2] => public://files/2009/08/titi_20_dylux-16-copy.jpg
    [:db_insert_placeholder_3] => image/jpeg
    [:db_insert_placeholder_4] => 279795
    [:db_insert_placeholder_5] => 1
    [:db_insert_placeholder_6] => 1330908310
    [:db_insert_placeholder_7] => image
)
 (/mnt/www/html/xxx/docroot/includes/common.inc:6975)
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://files/2009/08/bnsm0204net.jpg' for  [error]
key 2: INSERT INTO {file_managed} (uid, filename, uri, filemime, filesize, status, timestamp, type) VALUES
(:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3,
:db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7); Array
(
    [:db_insert_placeholder_0] => 129
    [:db_insert_placeholder_1] => bnsm0204net.jpg
    [:db_insert_placeholder_2] => public://files/2009/08/bnsm0204net.jpg
    [:db_insert_placeholder_3] => image/jpeg
    [:db_insert_placeholder_4] => 475613
    [:db_insert_placeholder_5] => 1
    [:db_insert_placeholder_6] => 1330908310
    [:db_insert_placeholder_7] => image
)

Looks like it's trying to download and install the image on the server, but they already exists.. so it fails. Any easy way to fix this?

thanks!!

mikeryan’s picture

Not sure offhand, I'm going to have to investigate the duplicate files. With the fits and starts you've had, you were probably left with file_managed entries from earlier attempts, but the migration should really be reusing (updating) them rather than trying to insert new entries for the same files.

mikeryan’s picture

Status: Postponed (maintainer needs more info) » Closed (cannot reproduce)
hockey2112’s picture

I am also receiving that line 512 error message, in version 7.x-2.2. Any ideas on how it can be fixed?

An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /~globalwl/?q=batch&id=12&op=do StatusText: parsererror ResponseText: {"status":true, "percentage":"60", "message":"Imported 5 in 0.1 sec (2097\u002Fmin) - done with \u0027WebsiteNameAuthor\u0027\u003Cbr \u002F\u003EImported 57 in 0.7 sec (4922\u002Fmin) - done with \u0027WebsiteNameBlogEntry\u0027\u003Cbr \u002F\u003EImported 0 in 0.1 sec (0\u002Fmin) - done with \u0027WebsiteNamePage\u0027\u003Cbr \u002F\u003E"} Notice: Undefined property: WordpressMigrateStreamWrapper::$ in /home/mysite/public_html/includes/stream_wrappers.inc on line 512 Fatal error: Call to undefined function () in /home/mysite/public_html/includes/stream_wrappers.inc on line 512