Some forum posts not displayed. Please help!

SadGeezer - June 14, 2007 - 19:25

Hi,

I searched for similar topics to this and couldn't find anything which helped.

We've just successfully ported a postnuke (.723) website to Drupal (5.1)... almost.

We have all the users, news items, comments, forum categories, forums and the vast majority of forum posts all displaying in the right sequence and all the right message content. Unfortunately a number of posts (in each forum) will not display. That is, they are in the Drupal database (you can search and find them) but they are not displayed in the forum (any forum).

I've compared the records of each database (postNuke and Drupal) and they messages don't contain any dodgy data and to all intents and purposes, seem like any of the other messages that are displaying correctly.

I've estimateld that between 5% and 10% of the messages are missing from the forum listings.

Can anyone tell me why this is happening and if there is something I can do to fix it?

Background and example:

The PostNuke forum was a pnphpbb2 module of PostNuke. We adapted a script that was on this site and everything else has moved across without any problems (we fully intend to post the migration script here once we have successfully migrated the content).

Example of the problem.

A message entitled "That's Good Soup" is in the old PostNuke 'General Sci Fi' forum (about 20 posts down) here:
http://www.sadgeezer.com/html/phpBB2+viewforum-f-1-sid-16e5f44512853a5dd...

It's not an unusual or complex message. See for yourself here:
http://www.sadgeezer.com/html/phpBB2+viewtopic-t-5606-sid-16e5f44512853a...

On the new Drupal site (theme not ported over yet) you cannot see the message in the 'General Sci Fi' forum here:
http://sadgeezer.org/?q=forum/45

But you CAN see it if you do a search for 'Good Soup':
http://sadgeezer.org/?q=search/node/good+soup

You can click on the link and it opens the message up and even tells you that it's posted in the right forum!!! See Here:
http://sadgeezer.org/?q=node/5991

The forum message and subsequent comments were successfully transferred to the Drupal site but will not display in the forum.

Strangely, if I search for the message and add a comment, the comment is correctly appended and the message is correctly displayed and promoted to the top of the forum (as you would expect). NOTE: Please don't do this with the example as it will spoil it for others reading this message.

Interestingly, if I go and delete the comment, the message is displayed in it's correct placing (about 20 posts down the forum listing!!

We've tried everything to work out what the problem is but we're completely stumpted! Any ideas how we can get the rest of the 'hidden' messages to appear?

Code used to import from pnphpbb2 to Drupal forums

SadGeezer - June 19, 2007 - 12:50

# start pnPhpBB stuff
# Written by Feodor (feodor [at] mundo.ru)
#
# Modified for drupal 4.5.2 and phpbb 2.0.11-2.0.13
# by Alexander Mikhailian
# modified for postnuke pnphpbb conversion by Keizo
# screwed up for drupal 5 by me
# Drupal variables for node counts
#
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);

SELECT @forum_title:='Forums';
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';

#
# Create a vocabulary (called "Forum")
#
DELETE FROM vocabulary WHERE module='forum';
INSERT INTO vocabulary (vid,name,description,help,relations,hierarchy,multiple,required,module,weight)
SELECT @vocabulary_vid+1,@forum_title,'','','0','1','0','1','forum','0';

#
# Set up the vocabulary "Forum" as the forum vocabulary
#
DELETE FROM variable WHERE name = 'forum_nav_vocabulary';
INSERT INTO variable (name, value) VALUES ('forum_nav_vocabulary', CONCAT('i:',@vocabulary_vid+1,';'));

#
# The default comment order in phpbb2 is "oldest first"
#
DELETE FROM variable WHERE name = 'comment_default_order';
INSERT INTO variable (name, value) VALUES ('comment_default_order', 's:1:\"2\";');

#
# Import categories from nuke_phpbb_categories as terms
#
INSERT INTO term_data (tid,vid,name,weight)
SELECT @term_data_tid+cat_id,@vocabulary_vid+1,cat_title,cat_order
FROM phpbb_categories;

#
# Import terms from nuke_phpbb_forums
#
INSERT INTO term_data (tid,vid,name,description,weight)
SELECT @term_data_tid+@phpbb_cat+forum_id,@vocabulary_vid+1, forum_name,forum_desc,forum_order
FROM phpbb_forums;

ALTER TABLE term_data ORDER BY tid;

#
# Import forum hierarchy
# Drupal allows topics to be created at this level while phpbb2 does not. If you want to disallow topics below
# categories, mark them as containers in the forum configuration dialog
#

# BROKE HERE!!!!

#this block to rem later
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);

INSERT INTO term_hierarchy (tid,parent)
SELECT @term_data_tid+cat_id,'0'
FROM phpbb_categories;

INSERT INTO term_hierarchy (tid,parent)
SELECT @term_data_tid+@phpbb_cat+forum_id, @term_data_tid+cat_id
FROM phpbb_forums;

ALTER TABLE term_hierarchy ORDER BY tid;

#
# Create temporary tables for sorting topics and comments.
#

DROP TABLE IF EXISTS temp_posts;
CREATE TABLE temp_posts (
post_id mediumint(8) UNSIGNED NOT NULL auto_increment,
topic_id mediumint(8) UNSIGNED DEFAULT '0' NOT NULL,
forum_id smallint(5) UNSIGNED DEFAULT '0' NOT NULL,
poster_id mediumint(8) DEFAULT '0' NOT NULL,
post_time int(11) DEFAULT '0' NOT NULL,
post_edit_time int(11),
post_subject blob,
post_text text,
PRIMARY KEY (post_id),
KEY forum_id (forum_id),
KEY topic_id (topic_id),
KEY poster_id (poster_id),
KEY post_time (post_time)
);
DROP TABLE IF EXISTS temp_node;
CREATE TABLE temp_node (
post_id mediumint(8) UNSIGNED NOT NULL auto_increment,
topic_id mediumint(8) UNSIGNED DEFAULT '0' NOT NULL,
PRIMARY KEY (post_id),
KEY topic_id (topic_id)
);

#
# Copy into temporary table topics without comments
#
INSERT INTO temp_node (post_id,topic_id)
SELECT MIN(post_id), topic_id
FROM phpbb_posts
GROUP BY topic_id;
INSERT INTO temp_posts (post_id, topic_id,forum_id,poster_id, post_time,post_edit_time,post_subject,post_text)
SELECT c.post_id, c.topic_id, a.forum_id, IF(a.poster_id='-1','0',a.poster_id), a.post_time, a.post_edit_time,
REPLACE(b.post_subject, CONCAT(':',b.bbcode_uid),''), REPLACE(b.post_text, CONCAT(':',b.bbcode_uid),'')
FROM phpbb_posts AS a, phpbb_posts_text AS b, temp_node AS c
WHERE c.post_id=a.post_id AND c.post_id=b.post_id;

#
# Insert nid and tid from temp_posts into term_node
#
#this block to rem later
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);

INSERT INTO term_node (nid,tid)
SELECT @node_nid+topic_id,@term_data_tid+@phpbb_cat+forum_id
FROM temp_posts;

ALTER TABLE term_node ORDER BY nid;

#
# Insert forum topics from temp_posts into node
#

#this block to rem later
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);

#INSERT INTO node (nid,type,title,uid,created,comment,body,changed)
#SELECT @node_nid+topic_id,'forum',post_subject,poster_id,post_time,'2',post_text,
#IF(post_edit_time<>'NULL',post_edit_time,post_time)
#FROM temp_posts;

SET @default_story_format = 3;
INSERT INTO node_revisions (nid, title, uid, timestamp, body, format)
SELECT @node_nid+topic_id,post_subject,poster_id,post_time,post_text,
@default_story_format
FROM temp_posts;

INSERT INTO node (nid, type, title, uid, created, comment, changed, vid)
SELECT @node_nid+topic_id,
'forum',
post_subject,
poster_id,
post_time,
'2',
IF(post_edit_time<>'NULL',post_edit_time,post_time),
node_revisions.vid
FROM temp_posts
Inner Join node_revisions on @node_nid+topic_id=node_revisions.nid;

ALTER TABLE node ORDER BY nid;

#
# Insert nid into forum
#

#this block to rem later
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);

DELETE FROM forum;
INSERT INTO forum (nid,vid,tid)
SELECT @node_nid+topic_id, vid, @term_data_tid+@phpbb_cat+forum_id
FROM temp_posts
Inner Join node_revisions on @node_nid+topic_id=node_revisions.nid;
#
# Insert comments into comments for topics from temp_posts
#
INSERT INTO comments (nid,uid,subject,comment,hostname,timestamp,users)
SELECT @node_nid+a.topic_id,
CASE WHEN a.poster_id='-1' THEN '0' ELSE a.poster_id END,
REPLACE(c.post_subject, CONCAT(':',c.bbcode_uid),''),
REPLACE(c.post_text, CONCAT(':',c.bbcode_uid),''),
CONCAT_WS('.',CONV(SUBSTRING(a.poster_ip,1,2),16,10),
CONV(SUBSTRING(a.poster_ip,3,2),16,10),
CONV(SUBSTRING(a.poster_ip,5,2),16,10),
CONV(SUBSTRING(a.poster_ip,7,2),16,10)),
a.post_time,'a:1:{i:0;i:0;}'
FROM phpbb_posts AS a LEFT JOIN temp_posts AS b ON a.post_id=b.post_id,phpbb_posts_text AS c
WHERE b.post_id IS NULL AND a.post_id=c.post_id;
ALTER TABLE comments ORDER BY cid;

UPDATE comments,node
SET comments.subject=IF(comments.subject='',
CONCAT('Re:',node.title),comments.subject)
WHERE comments.nid=node.nid;

#
# Update thread in comments
#
#this block to rem later
SELECT @phpbb_terms:=MAX(forum_id) FROM phpbb_forums;
SELECT @phpbb_cat:=MAX(cat_id) FROM phpbb_categories;
SELECT @term_data_tid:=id FROM sequences WHERE name = 'term_data_tid';
SELECT IF( @term_data_tid>0, @term_data_tid, @term_data_tid:=0);
SELECT @comments_cid:=id FROM sequences WHERE name = 'comments_cid';
SELECT IF( @comments_cid>0, @comments_cid, @comments_cid:=0);
SELECT @vocabulary_vid:=id FROM sequences WHERE name = 'vocabulary_vid';
SELECT IF( @vocabulary_vid>0, @vocabulary_vid, @vocabulary_vid:=0);
SELECT @node_nid:=id FROM sequences WHERE name = 'node_nid';
SELECT IF( @node_nid>0, @node_nid, @node_nid:=0);
DROP TABLE IF EXISTS comments_tmp;
CREATE TABLE comments_tmp (
cid int(10) NOT NULL default '0',
nid int(10) NOT NULL default '0',
thread int(10) NOT NULL auto_increment,
PRIMARY KEY (nid,thread)
);

INSERT INTO comments_tmp (cid,nid)
SELECT cid,nid
FROM comments
WHERE cid>@comments_cid;

UPDATE comments,comments_tmp
SET comments.thread=CONCAT(CONCAT(REPEAT(9,
LEFT(comments_tmp.thread,LENGTH(comments_tmp.thread)-1)),
RIGHT(comments_tmp.thread,1)),'/')
WHERE comments.cid=comments_tmp.cid;

#
# Update history
#
#INSERT INTO history (uid, nid, timestamp)
# SELECT a.uid, b.nid, a.timestamp
# FROM users AS a, node AS b
# WHERE a.uid>0 AND b.nid>@node_nid;

#
# update topic statistics
#

INSERT INTO node_comment_statistics
(nid,last_comment_timestamp,last_comment_name,last_comment_uid,comment_count)
SELECT @node_nid+pt.topic_id, pp.post_time, ppt.post_subject, IF(pp.poster_id=-1,0,pp.poster_id),
pt.topic_replies
FROM node dn, phpbb_topics pt, phpbb_posts pp, phpbb_posts_text ppt
WHERE dn.nid = @node_nid+pt.topic_id
AND pt.topic_last_post_id = pp.post_id
AND pp.post_id = ppt.post_id;

#put something into node_counter for read counts (totalcount is the read count, daycount seems to be the #same thing?)
INSERT INTO node_counter (nid,totalcount,daycount,timestamp)
SELECT @node_nid+topic_id,
topic_views,
topic_views,
unix_timestamp(topic_time)
FROM phpbb_topics;

#update for user 2
update node set uid=1 where uid=2;
update node_revisions set uid=1 where uid=2;
update comments set uid=1 where uid=2;

#
# Delete all temp tables
#
DROP TABLE IF EXISTS temp_posts;
DROP TABLE IF EXISTS comments_tmp;
DROP TABLE IF EXISTS temp_node;

#
# Update Drupal variables
#
SELECT @term_data_tid:=MAX(tid) FROM term_data;
SELECT @comments_cid:=MAX(cid) FROM comments;
SELECT @node_nid:=MAX(nid) FROM node;
SELECT @users_uid:=MAX(uid) FROM users;
SELECT @vocabulary_vid:=MAX(vid) FROM vocabulary;

DELETE FROM sequences WHERE name="term_data_tid";
DELETE FROM sequences WHERE name="comments_cid";
DELETE FROM sequences WHERE name="node_nid";
DELETE FROM sequences WHERE name="users_uid";
DELETE FROM sequences WHERE name="vocabulary_vid";

INSERT INTO sequences (name,id) SELECT "term_data_tid", @term_data_tid;
INSERT INTO sequences (name,id) SELECT "comments_cid",@comments_cid;
INSERT INTO sequences (name,id) SELECT "node_nid",@node_nid;
INSERT INTO sequences (name,id) SELECT "users_uid",@users_uid;
INSERT INTO sequences (name,id) SELECT "vocabulary_vid",@vocabulary_vid;

forum

vitovonantwon - December 25, 2007 - 15:43

Did you check your mysql database node, vocabulary_node_types. From what I've gleaned from trying to solve my own problem; The node should have the vocabulary id of your forum type. So if your If you have several types like image, weblink,forum they are assigned a unique number in the vid mysql record. I've assigned tags to have all my node types. So below the 9 (which is tags(keywords) is linked to 7 vocabulary items but forum category is linked to forum and image.)

Full Texts nid vid type title uid status created changed comment promote moderate sticky
Edit Delete 1 1 blog Nyc Lightwave lives again 1 1 1147931155 1147931155 2 1 0 0

So in the above... If the word wrap doesn't mess it up, that node has a vid type of 1 Which is forum.
vid type
1 forum
1 image
2 image
7 Weblink
9 event
9 forum
9 image
9 massmail
9 mass_contact
9 page
9 story

Maybe that info is missing from those posts? Use phpmyadmin and check.

I'm having a problem where new forum posts show up on recent_posts page and front page but not in the containers of the forum.

www.nyclightwave.com

Vito

SELECT *
FROM `vocabulary_node_types`
LIMIT 0 , 30

 
 

Drupal is a registered trademark of Dries Buytaert.