| Project: | phpBB2Drupal |
| Version: | 6.x-2.1 |
| Component: | Code |
| Category: | support request |
| Priority: | major |
| Assigned: | Unassigned |
| Status: | needs work |
Issue Summary
Hi there,
First thanks for a great module. I have a number of 'problems' with my conversion. I quite willing to believe operator error may be at work here though lol. Im converting first from smf to phpbb3 and then on to drupal. The first conversion step goes without a hitch (smf --> phpbb3) manually checked tables and forums. Attachments/avatars all fine. Now, when i convert from phpbb3 to drupal (with attachments check box ticked) when i view my forum they are not displaying. Also user pics arnt transfereing as expected either.
I should mention that at some point previos someone switched the attachment setting in smf from real file name to numericCode_Hash format, however phpbb3 has converted all to the hash type file (i think my terminology is right).
I'm not afraid to dive right into phpmyadmin and check whats in the tables after running your convertor and it all appears fine. Backtracing from drupal comments all the way to original smf posts reveals that the right attachment is associated with the right post in the database.
This leads me to believe there may be something up with the way i've moved the files over and their new path and/or my drupal configuration.
I created a new post in drupal with attachments and posted to the imported board and it shows up fine.
On a side note, I have noticed that the default signiture field type in the drupal user table is varchar(255) and some of my members signitures exceed this. If i converted the type to TEXT before i run your convertor will it still function as expected? (im also quite happy to mod the code is necassary).
If you need anymore info dont hesitate to ask.
Kez
Comments
#1
#2
Is it just the attachments that are not showing?
#670424: Missing posts after migration?
To see what the problem is, compare the data in the files table with what is actually on the filesystem - pick any attachment and have a look at what path is being given for it, and then check if it is the correct one.
#3
Hi,
Thanks for the speedy response. Ok here we go, i choose a message with an attachment from the converted forum running in drupal.
That gave a cid of 2702 which check outs in the comments_upload table. It refers to file swat2.jpg ( i could not see any path information ). Now come the interesting bit. Checking the file stock of attachments i have; sods law obviously dictates its not there. So a cursory glance at the same forum message in the phpbb3 tables show infact that the file has indeed been given a hash code type name. So i identify that file and rename it swat2.jpg and move it to htdocs\sites\default\files (drupal is installed in root of htdocs). For completeness sake i also copied to htdocs\sites\default\files\pictures also. No joy still (if we can get this too work a bottle of scotch is heading your way :p ).
Now here comes the kicker, i check my drupal settings and try and change default file store only to be told that the permission on drupals default folder is read only. So i do what any sane person (if you count running xp as sane) would do and right click said folder remove read only. Guess what? Its a known issue (or feature as microsoft calls it) where in order to preserve there thumbnail folders etc you cannot remove the read only attribute from the folder. Check google for many many many angry people lol.
thanks for you help again
EDIT EDIT EDIT EDIT
Ok, so idiot me just worked out that there is also an FID column in comment_upload which relates to the file table.
here is the line
176 0 swat2.jpg files/1_501c1ae9df445fe2fb84483312f7619d image/jpeg 116579 1 1180240514
So i can confirm that that does exit in that directory (assuming that directory is htdocs\sites\default\files\).
My limited memory is telling me that there was some setting somewhere which limited the filetypes the forum would display. Given that the numericCode_Hashcode filename doesnt have any extension could that be the root of the problem?
END EDIT END EDIT END EDIT
Kez
#4
For your conveniance (spelling? lol) grabbed this screenshot for you (with my da vinci-eqse masterstrokes)
#5
I can see that the "files/1_501c1ae9df445fe2fb84483312f7619d" bit is wrong.
You should be able to fix this in phpmyadmin with an sql script to replace the filepath with the real file path plus whatever is in the filename column.
(if you have uploaded ohter files using other means, you may want to then limit the query for the relevant fid's)
#6
thanks for that screenshot - it shows that I was wrong with my last instructions. If you have the attachments in the files folder, then it should work as expected.
If you have them in some other folder, then you need to replace the "files/" bit with whatever the correct path is.
EDIT - just to check if it is not the filename, edit one file to give it an extension, and then edit the files data to point to the correct place and see if that gets it shown.
Chances are that you have uploaded the files to the wrong place. Have you also checked the permissions to see if people have the permission to see the uploaded files?
#7
Hi again mate,
Thanks again, for help, using the limited grey matter i have left at this time of night i bothered to check the file path for the attachment i uploaded as a test. You were bang on the money (check screen shot).
I'm going to move all files to root/files and see what happens.
I'm unsure if this will fix avatars but we'll see lol.
Were getting close to that booze bonus now :))
Also, i had to hack your phpbb2druapl.module and i'll tell you why. I was in the situation where i am migrating on behalf of a third party. This means I am not THE admin in the sense of the board founder/amdin. What was happening was my drupal admin account was overwriting the true admin/founder lol ( i dont think hed be too happy!! :) ) So i removed some code and lowered the user id grab to 1 from 2, removed any reassginment of posts etc from user 2 (true admin of phpboard in this case was user 2). Now im no code guru and it hasnt quite worked as i expected but it has worked. One artefact of my efforts was all users ids were bumped up by one. Not a bother as that occured before the mapping. I'm sure there maybe other users in this situation so maybe providing a yes/no option to link the admin account with the board admin may be useful for future releases.
Kez
#8
Ok experimentation has led me to discover moving the folders doent work, but adding sites/default to the begging of the file path does work. You wouldnt happen to know the mysql querry to add sites/default/ to the beginning of every entry would you?
Kez[
#9
in phpmyadmin, try something like:
UPDATE files SET filepath = REPLACE(filepath, "files/", "sites/default/files/") WHERE fid < X(where X is teh maximum fid for the import... you can remove the WHERE statement if there are no other files... and change it if there were other files before the import.)
#10
here you could have created a new user and then in the phpbb2drupal settings page chosen to map the details to the new user without hacking the module.
#11
I could have created a new user but from my experience the drupal user take precedence over the board admin, i.e. i lost the board admins details. I assume what ur saying is that if there is a 'blank user' the board owners details wil transfer to the drupal account rather than the other way round. Is it possibly to do with the fact the the drupals admin usersname was admin?
Once again thanks for your speedy and helpful responses.
Kez
#12
Your sql query worked a charm thanks. Next task, avatars lol, i shall report back once i've gone through the same steps as we have here.
Thanks so much
Kez
#13
#14
Hi mate.
Ive reinstalled from scratch and the db fix we discussed above works like a charm. Now im experienceing another problem :(
Passwords dont work. I did some basic investigation which revelaed the following:
(this is the pass for my account on the original forum)
SMF 2f8c6da0d40ae61b11edaaa8d1ea2dae928967e3
PHPBB3 2f8c6da0d40ae61b11edaaa8d1ea2dae928967e3
DRUPAL 2f8c6da0d40ae61b11edaaa8d1ea2dae
drupal user_table appears to set password as varchar(32);not long enough it appears. So i edited the structure and replaced the password in the table. Still doesnt work. Another bizarre thing happens. Every so often it will take the admin password with my username. can't replicate that with any regualitry though. It just happens sometimes.
Once i get passed this i'll continue with the avatar check.
Any thoughts?
Kez
#15
do you have the phpass module installed? (there is a "fixed" working version downloadable from my site).
What encryption scheme does smf use? phpbb3 uses phpass, same with the upcoming drupal 7. but standard drupal 6 only uses md5, so the extra module is needed to allow the phpass passwords to be used.
#16
Hiya,
Yes i have that module installed. The crazy think is ive been through this conversion/install process alot now, and it has worked before.
I should add that the passwords do work in phpbb3 from smf, i have all three installed and on the same database so it seems its getting confused after phpbb3.
Still looking for smf encryption infor, it appears they change between versions :(
Any ideas?
Kez
#17
try copying the password into the phpass table to see if it makes any difference.
#18
Also, could you advise if i need to replace the passwordhash.php if i use the module at the bottom of your site?
Kez
#19
you can replace it if you wish - I have not checked to see if there is an updated version available, but I had made no changes to that file, so it does not hurt to replace it - it was just a matter of convenience.
#20
Just an update for you,
It appears id overwritten your modded phpass from your site with another causing the errors. All ok now. Gonna have a bash at those avatars. Thanks again
#21
More oddeties im afraid. althought my user id pass carried over fine, some other members who tried didnt. User name ok, password is cut short though. druapls native field pass field is varchar(32) yet phpbb3 is varchar(40). So i changed the field type to varchar40 and appened the missing bit of password but still no joy.
any ideas?
Kez
#22
Figured it out i think, smf doesnt use md5 to encrypt. what ws happening was when i logged in to phpbb3 it was taking password and converting it to md5, therefore it worked on drupal. This explains why only i could log in as only my password had been through that middle conversion process.
thanks for all your help.
If you have any thoughts let me know other wise cheers
Kez
#23
ok, that is probably not something that can be avoided - you may need to force all the users to reset their passwords.
Either that, or you can create a new module that overrides the authentication proces a'la phpass and then use that to authenticate the users, but this will be more effort and you will need to find out exactly how smf encoded its users.
Or you can ask all the users to request a new password.
#24
Just to give u an update. I replaced authentication ala phpass and it works great now. Thanks alot for all your help.
Kez
#25
#26
Hi,
I also have a attachment import problem. I've made sure that the files are at the correct location. But all the "filepath"s are like "files/1_501c1ae9df445fe2fb84483312f7619d". When I click the link, my browser asks for downloading the file, but it sees the file as "1_501c1ae9df445fe2fb84483312f7619d" without any attachements (and the file name doesn't make any sense, although on the filename column I see that the file name is "something.pdf").
So there are two problems here:
1. The files does not have attachments (and since there are different types of files -jpg, gif, pdf, etc.) I'm not sure how I can give those files correct extensions (maybe using the data from the filename column, but how?).
2. The file names, while downloading, does not make any sense. Although this is not a major issue, I feel these two issues can be solved at once. I only lack the knowledge to do so.
Some more info:
When I rename "1_501c1ae9df445fe2fb84483312f7619d" to "1_501c1ae9df445fe2fb84483312f7619d.pdf" on the filepath, it downloads as a PDF file, just normal (only the file name is gibberish). But when I rename it to "something.pdf", the download link takes me to the File Not Found page.
Thanks in advance
#27
Is there any way you can find out what all the file types are? are they stored anywhere? You will need to compare in the database.
As for the filename, that is a different issue and since the files are named in such ways before, that bit cannot be fixed by this module, though it may be possible to create a mass renaming script?
#28
Hi,
Thanks for the response. For the file types there are two sources as far as I can see:
1. The filename column has the filenames as "something.pdf", so there is the information about the file extension
2. There is a filemime column in the table which includes info like "image/jpeg", "application/octet-stream", "application/pdf", "application/zip" etc about the file type. This second one maybe more difficult to use.
If you think the first info can be used, could you also help me creating the phpmyadmin SQl query?
Cheers
#29
are the actual filenames in the director ending in eg ".pdf" or do they have no file endings?
If they do have the filetype endings, then you can redo the import and in phpbb2drupal.module, on lines 662 and 822 (in both cases jsut before the db_query line and after the
db_set_active('default');) add<?php$file->physical_filename = $file->physical_filename . '.' . end(explode(‘.’,$file->real_filename));
?>
amd then re do the import.
If however the files do not have the filetype in their physical file names, that will not work.
(there may also be a phpmyadmin query to update the filepath using the filename data, but I am not sure how you would get a substring there...)
#30
Thanks for the info. I'll try this approach.
(Yes, the filename column includes the extensions.)
#31
(I meant the actual filenames in the filesystem as opposed to the filenames as stored in the database)
#32
No the actual files don't have the file extensions either.
Hmm, this is getting trickier that I thought it would be.
#33
That complicates things as then you need something to change the actual filenames too. This would require a more complicated script to be written, one that manipulates the files too.
#34
I don't have access to phpmyadmin right now, so I don't know if this is possible or not.
Maybe I can try to rename the files before starting the migration. If such info exists in the phpbb tables I can do the rename first and then use the un-hacked version (excluding the hack you shared a few comments ago) of php2drupal migrate to do the migration.
I'm just noting the idea here. I'll have a look in a few hours.
#35
Hi again,
Sorry for the late response, but I couldn't find any time for this.
I've just checked the PhpBB tables and in table phpbb_attachments I found two coulmns, physical_filename and real_filename.
phyical_filename is something like this: 2_a8b580ad650e236892ba294fd13f565b
corresponding real_filename is: play.jpg
So, the PhpBB database contains the file extensions as part of real_filename. Do you think this will help?
#36
We are already importing the physical filename as the filepath and the real filename as the filename.
Now the issue is how to display the file with the correct filename - drupal shows the filepath filename as the real filename... and that is not an issue I know how to fix.
#37
The only way I can think of is to rename all the files using a bulk file renaming utility to their "real_filename"s. And then replacing the filepath in Drupal with the filename. In that case, filepath, filename and the actual file name of the file would be all the same.
I only need to export the PhpBB table in a file (if I can) and run the renaming utility using that file to do the renaming. This may be tricky...
I just wonder why I don't have the attachments of the files...
#38
I am unsure why it does not work, but changingthe filepath to inclide the .pdf etc to add an attachment does seem to indicate it is more an issue of what types of atttachment are allowed and not allowed attachments will probably not show. But this is a total guess.
#39
After a year there is something new about this problem with attachments "real names" vs "physical name"?
I imported from phpbb 3.08 yesterday and have this problem with attachments.
By reading around the issues (and applying a patch) I was able to import attachments into posts (I can see their real name in drupal posts), but they get not rendered because when I click on them they goes to the physical name.
example:
in a post I see attachment "img_1234.jpg", but when I click on it goes to "/sites/default/files/files/53_ddc2953156972e826ff9c6160757f53c" and the browser can't render that file without extension.
I copied all the PHPBB3 attachment folder to where it points, so it finds them. But the browser can't render a non extension file (under windows)
It's a problem in the module...
How this can be solved? without mass file renaming & relative sql editing?
I need attachments working, otherwise phpbb2drupal it's not useful in my case.
thanks,
Roberto.
#40
I renamed all attachments files with a script, giving them the real name extracted from phpbb3 table.
Now I can see attachments and they all open fine in the browser.
I have some issues with post count, need to double check if it imported everything.
#41
to help other users, do you mind posting your script that you used?