CSC-SY.net Migrated to Drupal

CSC-SY.net has been recently migrated from PHP-Nuke to Drupal, the migration process involved moving stories, forum topics, polls, posts and comments, private messages, and user accounts, we also had to port the theme we were using with PHP-Nuke to PHPTemplate, localize Drupal into Arabic, and finally configure Drupal to provide PHP-Nuke's features (and much more), details of the reasons behind the migration, and of the process itself follow.

Intro

CSC-SY.net is an Arabic community website for computer science students, it was started one year and a half ago, we chose PHP-Nuke in the beginning for several reasons, it was easy to flip the layout to RTL, which is a must for an Arabic site, PHP-Nuke had an Arabic translation, and its features were close to what we had in mind when we planned for the site. we were aware of PHP-Nuke's security issues, but with 3rd-party patches and our own modifications it went well in the beginning.

During the period of time we used PHP-Nuke, we realized how poorly-written PHP-Nuke was, as the site evolved we needed new features to be added, and with PHP-Nuke it became a nightmare, because it usually involved modifying so many files, making version updates another nightmare in turn, we had to update individual files by hand to patch security vulnerabilities, exploits were often found and updates had to be done often, every while we thought of moving to a more secure CMS, but the site was doing OK overall and we could live with PHP-Nuke, until several months ago, when the site's control panel started acting strangely, most of the strings were missing and after some debugging we couldn't fix it, we decided it wasn't worth the hassle, it was time to move to a new CMS.

The site was aging and missed all the new web features, like blogs and RSS feeds. We wanted to end the PHP-Nuke security nightmare. Web standards played a role in the decision, the new CMS had to be XHTML/CSS complaint, use UTF-8 encoding, and needed to be active in development.

After some investigation, our choices were narrowed down and finally we decided to choose Drupal, the main reason behind it was the tight integration of all site sections, more specifically forum.module, although it looked basic in the beginning, several sites customized it to look and act like other popular forum packages, so we thought it would be possible with our site, on the other hand, using a CMS that features a forum module instead of integrating another forum solution meant better security and compatibility.

After some investigation and testing, we realized that the migration process had three major points: data migration, theme creation, and localization.

Data Migration

Data migration seemed easy in the beginning, given that many scripts were available to do the task, but after further investigation, it turned out that none of them did what we wanted exactly, each solution missed something that had to be migrated, our plan was to migrate almost all of the site's content stored in PHP-Nuke's database, the migration involved moving the following items:

* ~770 users
* ~240 stories and polls
* ~360 comments
* ~17500 forum posts
* ~7110 private messages

Data migration made us research how PHP-Nuke and Drupal store those items, and research the best way to move data from one system to the other, given the large amount of data needed to be migrated, we had to develop a set of scripts to migrate our data, we made use of the already available scripts, and improve upon them, to create a semi-automatic script that fixes PHP-Nuke's database oddities (like date/time being stored as text, or having duplicate usernames, ...), migrates the data to Drupal's database, and we had make sure our work to fix things after running the script is minimal.

In addition, we had several private sections in the PHP-Nuke site, like admin forums and private message, we had that in mind while working on the script, a bug in the script could expose private data to the public.

We opened a beta section on our site to let users test the migration process, the script evolved as members reported bugs, and in the final stages before the migration, it had all what we needed for the data migration to be complete.

We had minor problems while working on the data migration other than moving across databases, data in PHP-Nuke is stored in windows-1256 encoding, but Drupal uses UTF-8, unfortunately MySQL 4.0.x doesn't support UTF-8 natively, so we had to figure out some workarounds to get over this, Arabic characters take more that one byte of storage in UTF-8, MySQL didn't realize this, leading to chopped-off text, we developed a script to convert PHP-Nuke's database to UTF-8 first, without losing any text, it involved increasing the length of some fields, and using iconv, at the end, all text data was migrated successfully.

Another side of data migration was converting BBCode (used by Nuke's phpBB forums) to something Drupal could parse, in the beginning it seemed that bbcode.module could handle this, but later it turned out that phpBB adds random strings to BBCode, so we updated our scripts to strip them out, we decided to do so instead of converting BBCode to HTML because we were planning on installing bbcode.module anyway, we guessed members used to BBCode would prefer to use it over HTML after the migration.

Theme Creation

When we opened up the first Drupal beta on our site, members complained about two things, forum threaded look and the theme in general, we started with a theme based on SpreadFirefox, but the majority didn't like it, we switched to FriendsElectric later and got less complaints, but later we realized that we had to develop our own theme for several reasons:

• Using a contrib theme from Drupal.org would make our site look like many other Drupal sites.
• Our site would keep the look it had for a year and a half.
• None of the contrib themes looked particularly similar to our original theme, or suitable for our site.

With that in mind, we started to port our PHP-Nuke's theme to Drupal, the original theme is based on tables, and hardly made use of CSS if it all, we based our work on Box_grey and ported the theme to XHTML/CSS, as I said earlier, Arabic is a right-to-left language, so the theme's layout needed to be so too, this complicated things a lot, especially that drupal.css doesn't take this into consideration, and assumes that the theme's direction is always LTR, after coding and debugging, we came up with a theme almost identical to the original theme, working well with Mozilla and IE browsers, the theme works with Opera and KHTML, but with minor annoyances.

Another aspect of the theme was making the forums flat, the forums were the core of the old site, and a major change from flat to threaded look wasn't welcomed by our community, so we worked on making the new forums look like phpBB, in other words, the forums' theme had to show the member's avatar, post count, registration date, and other info, to the side of the post body, the fourms should behave like phpBB in being flat, new replies should always appear in the bottom, with some theme work, it all was possible.

We also made modifications here and there to make the site look right, comment.module assumes the layout is LTR too, we had to fix this, we also added topic icons to the homepage, and did some changes to the user page.

Localization

As I said earlier, we chose PHP-Nuke in the very beginning of the site because it had an Arabic localization, later when we were evaluating current CMS's to choose what to migrate to, Drupal had "having a recent Arabic localization" as a plus,however, we then realized we couldn't use the localization available at Drupal.org, it wasn't updated to 4.6, wasn't consistent and had many typos, instead of fixing it, we decided it would be easier to start fresh, it didn't take long, and we had a localization customized for our site, right now all the site is fully translated into Arabic (including contrib modules we use) except for some parts of the admin area.

Drupal Configuration

After the migration work, Drupal needed to be configured to cover the old site's functionality, and provide new features we had in mind, Drupal's core covered frontpage news, polls, forums, comments, and user profiles, we installed many contrib modules as well, some of them are:

• article: for the articles section of the site.
• attachment: makes it easier for members to upload files with there posts.
• flexinode + taxonomy_dhtml: for the downloads section.
• image: for the gallery.
• privatemsg: for private messages.

In addition to many other modules that add more functionality, the most welcomed one by the community is TinyMCE, we also have taxonomy_access for the site's private sections, like admin, mod, and active members forums, and quote, freelinking, and bbcode for easier posing of content, troll, mail, and adminblock for easier administration.

After the migration we enabled two new sections, blogs and wiki, both were welcomed by the community, and provided functionalities that weren't possible before, blogs provide a space for each member to talk about their personal matters, and the wiki is where the community collaborates and makes decisions.

Drupal's tight integration made the whole site more active, members used to prefer the forums over other sections, and given that the old forums (phpBB) weren't that integrated with the rest of the site (PHP-Nuke), many members didn't realize that content was posted to the other sections as well, now with Drupal, /tracker provides a list of all updates, forums feel more like a part of the whole site, comments on news, polls, as well as forum posts count towards member's post count, and many other small details that made members participate in all site sections now, thanks to Drupal design and integration.

Drupal's search engine friendliness made our site reach top 3 links in many Google searches for phrases that are major and popular in Arabic, the site is even the number one result for some generic single-word searches now!

Members are glad with the new features and speed of the site, there are some minor bugs here and there, but we worked/are working on fixing them quickly with the help of the community.

We believe that CSC-SY.net is one of the first Arabic sites on the Internet that utilize Drupal, have a right-to-left theme and full Arabic localization, and it's definitely the very first fully-Arabic community site powered by Drupal, we hope that the site will convince other Arab web admins to try out Drupal, seeing how it can handle Arabic just fine, we are sure they will be satisfied with it as we are.

A Final Word

I'd like to thank the Drupal developers and contributors for their awesome work that made our new site happen, I've done the migration process almost single-handedly, and now I'm very satisfied with the results, I'd like to give back to the community, I've already contributed two patches for contrib modules, and I have some more, I'm also willing to contribute the Arabic localization if it's possible, I'd like to thank the folks at #drupal-support, especially chx and BuddaBoy for their help, and brick for the interesting chats, since I've started with the migration, I usually stick in #drupal-support and offer help as much as my knowledge allows me to.

-Ayman (CSC-SY.net site admin)