Hi,
I have re-posted this here (from the 'Converting to Drupal' forum) because I think it's more relevant here. I'm sorry but I couldn't delete the other post.
I searched for similar topics to this and couldn't find anything which helped.
We've just successfully ported a postnuke (.723) website to Drupal (5.1)... almost. We have all the users, news items, comments, forum categories, forums and ALL of the forum posts. Most are displaying in the right sequence and all the right message content. Unfortunately a number of posts (in each forum) will not display. That is, they are in the Drupal database (you can search and find them) but they are not displayed in the forum (any forum).
I've compared the records of each database (postNuke and Drupal) and they messages don't contain any dodgy data and to all intents and purposes, seem like any of the other messages that are displaying correctly.
I've estimateld that between 5% and 10% of the messages are missing from the forum listings. I am very confident that the data was transferred from the PostNuke database successfully to the Drupal database. I think this is a Drupal forum configuration issue.
Background and example:
The PostNuke forum was a pnphpbb2 module of PostNuke. We adapted a script that was on this site and everything else has moved across without any problems (we fully intend to post the migration script here once we have successfully migrated the content).
Example of the problem.
A message entitled "That's Good Soup" is in the old PostNuke 'General Sci Fi' forum (about 20 posts down) here:
http://www.sadgeezer.com/html/phpBB2+viewforum-f-1-sid-16e5f44512853a5dd...
It's not an unusual or complex message. See for yourself here:
http://www.sadgeezer.com/html/phpBB2+viewtopic-t-5606-sid-16e5f44512853a...
On the new Drupal site (theme not ported over yet) you cannot see the message in the 'General Sci Fi' forum here:
http://sadgeezer.org/?q=forum/45
But you CAN see it if you do a search for 'Good Soup':
http://sadgeezer.org/?q=search/node/good+soup
You can click on the link and it opens the message up and even tells you that it's posted in the right forum!!! See Here:
http://sadgeezer.org/?q=node/5991
The forum message and subsequent comments were successfully transferred to the Drupal site but will not display in the forum.
I notice after counting all the posts in a particular forum that the message counter includes the forum messages not displayed and the last couple of pages in a forum are usually empty (as though Druplal knows that it should be displaying some forum posts but .... doesn't).
Strangely, if I search for the message and add a comment, the comment is correctly appended and the message is correctly displayed and promoted to the top of the forum (as you would expect). NOTE: Please don't do this with the example as it will spoil it for others reading this message.
Stranger still, if I go and delete the comment, the topic and comments will then be displayed in it's correct placing (about 20 posts down the forum listing!! What's up with that?!
We've tried everything to work out what the problem is but we're completely stumpted! It would be impractical to go through and add and delete comments from the missing posts (around 5000). Any ideas how we can get the rest of the 'hidden' posts to appear?
Comments
We think it's definately
We think it's definately something to do with the message rather than any subsequent comments, but when doing a before and after comparison, the record dosn't seem to have changed.
Any ideas on what we could try to identify the problem?
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
Sounds like the database has
Sounds like the database has counters that needs to be updated or re-indexed.
Here are some things to try ...
Find a container/forum with missing posts and go into Content Management >> Forums: go into the container and forum and re-save them. See if the missing posts appear. Try changing or re-saving the settings too.
Try turning off anonymous access to forum. Try to view forum as anonymous. Log back in and turn anonymous access back on.
You might have to visit update.php
Turn off cache. (under 'performance')
Turn the site off then back on (site maintenance)
Disable the forum module then turn it back on.
Quint
Motorcycle News
Cron
Maybe running cron (Logs >> Status Report) might do some good.
Quint
Motorcycle Forum
Thanks for the suggestions....
Thanks very much Quint for your advice. I tried each one with details below:
>>Find a container/forum with missing posts and go into Content Management >> Forums: go into the container and forum and re-save them. See if the missing posts appear.
The missing post didn't appear. It did appear at the top of the forum when I added a comment. When I deleted the comment it still appeared (in the right order).
>>Try changing or re-saving the settings too.
I changed the forum setting (renamed one of the forums) and resaved the settings, but posts didn't appear.
>>Try turning off anonymous access to forum. Try to view forum as anonymous. Log back in and turn anonymous access back on.
I can't actually turn off anonymous access to the forum (not via admin>user>access at least). I accessed the page anonymously then logged in but this didn't show the posts.
>>You might have to visit update.php
I tried to run this but got a page not found error (might be because my Drupal friendly ISP has taken it off). I'll dig a bit further and report on the outcome.
>>Turn off cache. (under 'performance')
It was already disabled. I turned it back on and played around a while then turned it back off. I was still unable to see the missing posts.
>>Turn the site off then back on (site maintenance)
Posts still didn't appear.
>>Disable the forum module then turn it back on.
I disabled the forum Module and then turned it back on but the posts didn't appear.
I'm hopeful of the update.php script as that seems to go through all nodes and make sure they are stored in the correct way (saving and re-saving as it goes).
Any other suggestions would be a great help.
I'll also post the code snippet that was used to import the files from my old site at sadgeezer.com to the test site at sadgeezer.org. Maybe that will shed some light on this.
Thanks again Quint, for your consideration.
Regards,
Tony
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
update.php - error display
You might want to make this 'true' when you run it.
ini_set('display_errors', FALSE);
Quint
www.Shalla.Net
Depressing
Depressingly, update.php ran fine without any errors but the forum posts are still missing.
I think the next step is to post the script snippet (I say snippet, - it's a big script!). Hopefully that will shed some light.
In the meantime, I'll see if I can do anything more to identify the record changes before and after it has been updated (ie. while is invisible and after it takes it's rightfull, visible place in the forum).
Thanks again for your consideration.
Saddy.
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
Also
I noticed in the forums, that its as though Drupal knows that there are a lot of missing posts. The forum counts the messages and allocates enough pages to display them (the right number of topics are displayed on the main forums page) but the last few pages of the larger forums are usually empty of posts.
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
Last few pages?
Are the missing posts only at the end? Is there some cutoff date?
Similar/same thing here on Drupal.org
You know ... now that you bring it up, Right here on Drupal.org, under _ My Account >> Track _ I have a blank post at the end. It says:
Anonymous (not verified) 0 37 years 25 weeks ago
Which puts it around Jan 1, 1970. I don't think I've been using Drupal this long. :P
Quint
I get that if I checkout 'My
I get that if I checkout 'My recent posts'
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
No, posts are missing from most pages
No, that's the strange thing. It seems completely random.
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
Code used to import from pnphpbb2 to Drupal forums
This is the code that was used to import the data from a pnphpbb BS to the Drupal Forums.
As I said above - EVERYTHING has worked very well except for a few missing posts (which are in the database (on Drupal) but missing from the diplayed forum message listings).
Any help with what could have gone wrong would be much appreciated.
--------------------------------------------------------
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/
As long as you are backed
As long as you are backed up, and have something to revert to, go into your mySQL table and see if you can find a variable that could be something like
forum node count
forum node max
forum display number
I bet phpBB and Drupal have a similar-looking variable, but calculate it differently.
It's noteworthy that if you go to the empty pages and change the sort, you still don't see any posts. So Drupal apparently has a max number of posts stored, but still allocates the display pages based on the actual number of posts in the database.
Your General Sci Fi Stuff forum is allocated for 28 pages of posts, at 25 posts per page = 7000, but the last page is probably not all 25, so let's say 6990
That same forum is displaying 22 (x 25 = 5500 posts) full pages and page 23 has 20 posts, so you have 5520 posts displayed, and around 6990 posts allocated.
Look in the tables and see if there is a value around 5520, (or maybe figure the number for the whole forum count.)
Can you change the number of posts per page? You just need to get Drupal to re-figure that number somehow, or maybe you'll have to go into the table manually and find the number and change it yourself.
Quint
experiment
Also, it would be interesting if you can determine. by changing the sort, if the missing posts change. Since the missing posts are at the end, say you sort by date, and post "AAA help drivers" posted Jan 1, 1999 is at the end and invisible. If you sort by title, does this post show up at the beginning? Or are all the invisible posts always invisible, regardless of sort?
If you find a mod that adds something to every node it might fix that 'broken' counter. Maybe CCK?
I think we've cracked it....
I tried to do some more analysis of the records that are changed once a comment is added, especially after your comment that phpbb and drupal may handle the same type of variable differently.
To cut a long story short, I noticed that the "last_comment_uid" field in the "node_comment_statistics" table was set to '2' on one of the missing topics. This id theoretically shouldn't have existed as in pnphpbb the admin user id is '2' and that was renamed to '1' when moving to Drupal.
I changed all the last_comment_uid's from 2 to 1 (changed about 550 records) and suddenly all the posts that I had noticed missing, re-appeared.
I don't think that the problem is completely fixed. As you noticed Quint, the last page of some forums (especially large ones) is empty which to me, means that we are still missing some posts in the topic listing, but after a brief check, the forums list the most recent posts (last 12 months).
I think that the others are missing because of some discrepency between user id's from legacy systems (pre-pnphbb2). I'm looking into this a little more and will report any new findings.
Quint, thanks very much for your help. Your tips kept me encouraged and the support and willingness to help was simply excellent! Cheers.
Regards,
Saddy :)
The SadGeezers Guide to Cult Sci Fi
http://www.sadgeezer.com
SadCAST - The Sci Fi Video Podcast
http://sadgeezer.libsyn.com/