Hi all,

I'm hoping someone can point me in the right direction here. The situation is I'm trying to port my WoW guild's website into Drupal. First things first, I'd like to convert all the forum users, posts, avatars, etc, etc into Drupal before I even worry about the theme.

Upon setting up the phpbb2Drupal 7.2 module and using migrate to begin the port, I'm getting errors importing the Avatars. More specifically, the 'PhpbbAvatar' step is reporting several errors importing avatars, such as:

Could not copy file http://thefabled.net/forum/images/avatars/upload/b1b048b89e301e0f596e614a474e943a_57.gif

Basically that, multiplied by 91, leaving 210 unimported Avatars (according to 'migrate'). Now when you try to go to that image address you'll see it's a 404. I'm guestimating that's why it can't import it :cP . So my question is, two fold:

- Is there some setting in PHPBB that stores a list of Avatars in the DB that is old and out of date that I can purge so it won't be reporting Avatars with URL's that don't actually exist?

- Or, how can I just skip this step? Trying to move on to importing 'PhpbbForums' seemed to work... but then moving on to 'PhpbbUser' failed with "Skipped PhpbbUser due to unfulfilled dependencies".

Possibly related or unrelated, just as an FYI I'm also getting a warning of "Warning: Call-time pass-by-reference has been deprecated in /sites/all/modules/phpbb2drupal/phpbb2drupal_core.inc on line 556"

I tried to find that line in that file and @ it out to tell it to shut up but it didn't seem to do the trick. I'm thinking it has no real relevance to my issue, but you all may know better. Thanks for any help you can provide, and let me know if you'd like to see any more settings & such.

Comments

Psykoral’s picture

Psykoral commented

I love when I can answer my own questions :c). I'll post my findings in case anyone else is having the same issue.

So the Avatar thing was a permissions issue on the images themselves. They weren't Group readable (664) so that fixed that for the most part. Also, temporarily rename any .htaccess files you have that might be blocking 3rd party image downloads or directory access. I know I had one at the root of my site, as well as in /forum/.

It didn't exactly import all of them still; it was still getting some file names wrong (missing the userID after the underscore), trying to go to b1b048b89e301e0f596e614a474e943a_.jpg instead of b1b048b89e301e0f596e614a474e943a_1030.jpg for example, but I don't really care all that much. The bigger joy was that I could proceed to forums, users, etc and import all of those.

When I fixed the other 'Warning' issue that allowed the Users import to go through all the way. So if you're getting this error in the same spot I was, open /sites/all/modules/phpbb2drupal/phpbb2drupal_core.inc on line 556 you'll see:

$result = @preg_replace($pattern, $tag, $field, 1, &$count);

Remove the & in front of $count.

$result = @preg_replace($pattern, $tag, $field, 1, $count);

Psykoral’s picture

Psykoral commented

I also found out why it was trying to import the avatars without userID's on them. Many of the users who had no avatar, phpbb set the user_avatar field to empty. So, instead of skipping it (bug!), phpbb2drupal is trying to down load b1b048b89e301e0f596e614a474e943a_$USER.jpg

So thinking "OK, it's trying to grab the image that is in the user_avatar field" I updated all the empty user_avatar fields to '1.gif', made made a 1px transparent image and uploaded it as b1b048b89e301e0f596e614a474e943a_1.gif. Should work, right? Wrong. It was essentially completely ignoring the user_avatar field and literally trying to find a $userID.jpg or .gif

So since it was determined to grab b1b048b89e301e0f596e614a474e943a_.jpg and b1b048b89e301e0f596e614a474e943a_.gif, I uploaded both images as 1px transparent's with those file names and ran it again. A few more popped up with random chars after the _ (like 80x80, g, bf) so I just kept making blank images with those file names until it shut up and imported all of them. Not pretty, but it works!

Despite pretty much talking to myself this entire thread, maybe someone will find these troubleshooting tips useful in the future.

challenge’s picture

challenge commented

I just want to write "thank you" for that report, you're not writing only to yourself.

I have a phpbb forum with 2500 members to migrate and posts like this help me to find the motivation to try this module :)

surely i don't want to hand writing a thousand of blank avatars names mmm

the other things (posts, users, nesting) are imported fine or you found other issues?

Psykoral’s picture

Psykoral commented

Ah well you're certainly welcome for whatever help I've provided.

To answer your question, currently I'm basically back where I started with issues getting the Avatars to import. After I imported the users successfully (I should have left well enough alone!) I realized that I hadn't setup any 'roles' in Drupal yet. In the hopes that the import would transfer all my 'groupID' numbers from phpbb into roles I deleted all the users, created roles with ID's matching those in PHPBB and tried again.

Well, I regret that cause basically now I'm in the same boat of 'Skipped PhpbbUser due to unfulfilled dependencies' and PhpbbAvatar only importing 124 of 276 avatars. Though this time its actually more frustrating because it's giving NO errors. In fact, those temp files I mentioned above with the _.gif and the other weird file names... I deleted them from the phpbb upload avatars folder, but somehow Drupal is still copying them to /sites/default/files/pictures/forum/images/avatars/upload ! How it's doing that or where it's getting them from I don't know, but they certainly no longer exist in the folder I thought it was reading from. I've even gone as far as to delete the entire Drupal install folder AND the database it read from, since it's pretty easy to set it up again. Didn't do the trick :c(.

So once again I look for any assistance anyone can provide. Is there some sort of cache of these images somewhere? Perhaps somewhere it copied them temporarily first, and then over to the location above? I've purged the general cache in Drupal's admin/config/development/performance to no avail, it's still getting those images, which shouldn't exist, from somewhere.

darksnow’s picture

darksnow
Location Scotland
commented

Good debugging work there. ;)

As you found out, there is an issue with the .htaccess file and file permissions, it's mentioned in the readme file somewhere, but documentation for this project could be better.

The file name it's coming up with isn't as straight forward as just the UID, the function getRealFilename() in phpbb2drupal_code.inc generates the real file name, based on the avatar_salt and the user_avatar field in the database. Line 121 of the same file should check the user_avatar filed and skip missing ones, allowing the import to continue. There is no caching of this file. It can pull the image directly from the PHPBB installation is you've given it the URL, as you've seen from the error messages. If they're no longer there it will just complain that it couldn't find the files and fail.

Perhaps the avatar import shouldn't be a prerequisite of the user import, so that it's optional.

The pass by reference error is very useful, thanks for that. I'll patch the code and upload when I next get round to making any significant changes to this code.

Finally, group membership. I wasn't entirely sure how to handle this. On my PHPBB forum I added a mod to display the group icon under every user in that group, so I created loads of groups just to add little icons to people. Importing those as Drupal Roles seemed like the wrong way to go so I simply left that facility out. I think it's perhaps a good thing to have, but should be in another thread if you want to help decide how to proceed with this.

Thanks for the bug reports. There's still a lot of work to be done on avatar imports, some issues I have to resolve and some for the migrate team. Once they fix various issues, I'll get back on the case.

Cheers.

Martin...

Psykoral’s picture

Psykoral commented

It's been about a week or so since I last tried, but now I'm kind of stuck again. It was taking incredibly long (like 2 days) importing Phpbbtopic's to I tried to stop it. But for a week now it's been in "Stopping" mode and I can't continue.

Any suggestions?

Psykoral’s picture

Psykoral commented

Well now it's a whole other ball game. I saw there was an update for Migrate so I started it up, but upon updating the DB entries for it, it results in the following error:

Fatal error: Class MigrateDestinationFlagSimple contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (MigrateDestination::__toString) in /sites/all/modules/migrate_extras/flag.inc on line 78

Can't seem to get past this now. >.<

snlnz’s picture

snlnz commented

If it's any consolation I went through this last night and wrote some documentation as to how I got it working.

You can check it here: http://drupal.org/node/57197

Hope that helps as I had a hell of a time with the avatars/images and then again at phpbbattachments.

Psykoral’s picture

Psykoral commented

Thanks! I'll poke through this and try again soon. Just for my own sanity I'll probably wipe my previous instance out and try from scratch, just to help eliminate all the things I've done previously as possible snags at this point. Then I'll look into your notes for what worked for you and see if I can't get it completely migrated over.

snlnz’s picture

snlnz commented

Let me know how you get I've been pushing along nicely with this migration since getting over that obstacle.

ghankstef’s picture

ghankstef commented

I was able to work around it in this way: comment out line 22 of phpbbdrupal_core.inc that says this:

$this->dependencies = array('PhpbbAvatar');

Not sure that should be a requirement anyways.

Psykoral’s picture

Psykoral commented

Yep I did that to eliminate the dependency of Avatars importing completely before users as well, but as you may see in the above posts I ran into other issues with the rest of it. I'll try again one night this week when I can start all over, take the notes from my failures & successes combined with that of @snlnz and hopefully get a full board imported.

darksnow’s picture

darksnow
Location Scotland
commented

The reason it's a dependancy is because of the way the avatar import works. All it does is import the images. The user import then looks for an appropriate avatar to attach itself too, so the in order to get avatars to import at all, you need to run it first.

Now that I've done other things with the attachment import, perhaps it's time to rewrite the avatar import so it's dependant on users which makes it a more logical optional import.

Seneschall’s picture

Seneschall commented

You're writing to me, too, Psykoral.

Just a few questions (to the general phpbb2drupal audience): Since I only have about 200 users, 20 to 30 of them active at any given time, would it work better (or would it work at all) to simply turn avatars OFF on the phpbb forum, therefore giving it nothing to have to import?

Also, I'm getting "3 unmapped, 6 mapped" under destination. UID is one of them. Would this be another reason why it can't fetch the avatars?

I'm working with a fresh copy of D7, so not too worried about experimentation here.

Seneschall’s picture

Seneschall commented

The reason I ask is I'm getting a whole lot of this during av migration:

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'public://pictures/betaforums/images/avatars/upload/0145f6ef84711' for key 2: INSERT INTO {file_managed} (filename, uri, filemime, filesize, status, timestamp) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array ( [:db_insert_placeholder_0] => 0145f6ef84711fafbb1d9f6d6a40de8e_.jpeg [:db_insert_placeholder_1] => public://pictures/betaforums/images/avatars/upload/0145f6ef84711fafbb1d9f6d6a40de8e_.jpeg [:db_insert_placeholder_2] => image/jpeg [:db_insert_placeholder_3] => 47747 [:db_insert_placeholder_4] => 1 [:db_insert_placeholder_5] => 1336459880 ) (/<strong>(((site)))</strong>/includes/common.inc:7013)

Not sure what's causing this.

snlnz’s picture

snlnz commented

@skyline5k it looks like you have already attempted to import the avatars previously, just delete those records out of the tables in question "file_managed" and try again.

phreadom’s picture

phreadom commented

Some expansions on the bugs already noted...

The avatar import fills in the website address even when you try to leave it blank, then uses that instead of the directory you set for it to import from, which obviously fails.

Eg; even though I left the website address field blank, it ignores this and puts in "http:///" anyway when you try to save settings, which it then prepends to the avatar location, leaving something like "http:///sites/default/files/pictures/phpbb/images/avatars/upload", which fails.

I ended up having to just use the live website because I didn't feel like trying to dig through the code to figure out why it was doing this.

Next, avatar import places files in "/sites/default/files/pictures/phpbb/images/avatars/upload" and still using the full original filenames as well, just tacking on the entire forum avatar path and original filename after the normal drupal avatar location path... rather than checking if the user already has an avatar and using that instead, or mapping the avatar to the same kind of filename and location like the ones already in use, eg; "/sites/default/files/pictures/picture-2.jpg"

snlnz’s picture

snlnz commented

Not sure exactly what your problem is though if it is database related then you can always do an sql query executing a string replace on the paths. A very broad example might be
UPDATE tablename SET filepath = REPLACE(filepath, 'sites/default/files', 'sites/www.yourdomain.com/files');

However, I agree the whole migration of Avatars is a complete nightmare clearly someone needs to sponsor the coder to get it cleaned up.

Psykoral’s picture

Psykoral commented

I'm back! lol

So I'm trying this again, this time a fresh install of the latest Drupal 7 and phpBB2Drupal. Most of the import has gone quite well, many thanks to the "ignore dependencies" option. Here's what's going on now:

Upon import of the 'Phpbbpost', some of the imports go in successfully (19615 out of 78912):

"An AJAX HTTP error occurred. HTTP Result Code: 403 Debugging information follows. Path: /batch?render=overlay&id=3217&op=do StatusText: Forbidden ResponseText: Access denied"

And it goes on beyond that which seems like the start of the HTML template, complaining about @import CSS and such... but I think that's the jist of the actual error. It can happen at any point, sometimes right away, sometimes it'll process a batch of say 5 sets of posts and import them correctly before it shoots that error. For some additional background info, I'm pulling from www.mysite.com into stage.mysite.com, so while that error doesn't *look* like a cross scripting issue, could it be? I thought it was pulling directly from the database... It also almost seems like a $_SESSION time out issue... possible? Would my phpbb3 forums still being active at the time of import be contributing to the issue (should I lock them temporarily)?

Any ideas?

JeremyFrench’s picture

JeremyFrench commented
Status: Active » Closed (duplicate)

I think this is a dupe of #1672936: User Avatar Import Fails which has had some progress, it would be nice to see someone test the latest code (other than me)

Psykoral’s picture

Psykoral commented

Well it was started as an overall "Several steps in the process fail" thread. I was just going through the progression of which items are failing at what points in the hopes that someone would have suggestions, or if I found a solution I might help someone else by posting it.

Looks like the latest dev code is only updated since 2012-Jan-31. As of about two weeks ago I still couldn't import Avatars, users and posts smoothly. Is there some secret code release section that's been updated beyond January?