UTF-8 issues in node_revisions

Please excuse the long post. I really don't know what I'm doing so I am trying to explain everything that I did.

I'm a volunteer for a charity in London that works with Middle Eastern women facing violence in the family (www.ikwro.org.uk). For some time we've been running a campaign site on the issue of so-called 'honour' killings (www.stophonourkillings.com). I set this up in 2004 using Dragonfly CMS because at the time I felt this was the only solution to the high level of multilingual support I needed, with being UTF-8. I became more and more dissatisfied with DF particularly since the site was hacked over the summer and once the kids were back in school I resolved to move to Drupal.

I installed Drupal with my hosts (Apache 2.2.13 Unix / PHP 5.2.10 / MySQL 5.0.81 -community-log) and everything was fine. I simply made correspondances between tables in the original DF site and the new Drupal site, dumped them out of DF, bodged around with them in WordPad, and uploaded them back up. This worked out perfectly with no problems at all until I got to work on node_revisions. I took the data into Excel and worked it out there, just because of the size of the file. But I was having problems with symbols like ' and I couldn't upload it back. The file was about 17megs. In the end I asked for help from a local web developer and he created my file as a big insert table using Open Office, which I uploaded and initially looked OK... So me and my volunteers got to work creating new nodes ready to relaunch the site. However, checking some of the nodes later I found out there were big problems: some would have the teaser from one node combined with the body text of another. This was particularly noticeable when an article started in Norwegian, for example, and then concluded in Farsi. Checking his file in Universal SQL Editor I noticed that there were still a lot of problems with ' symbols and particularly -- in the body text.

So I went back to the data and tried to work it out for myself. I used Open Office and tried to export the files as CSV. Whether it was the size of the file or whatever, but OO just crashed every time I tried to output -- except once. I took that file, Wordpadded it a bit to take out certain characters and uploaded it to a second test database and transferred across the new nodes to combine them. All the garbage characters showing up in phpMyAdmin (for non-English symbols) were similar to the ones in the file I had been given, where the languages worked out fine even if the data was not correct so I wasn't worried. I cleared node_revisions and copied the data from the table in my test database into node_revisions. Garbage characters displaying in text ARGH! I checked all tables and databases to make sure they were marked UTF-8, I ran the table alter command. Nothing doing. I took the node-revisions file into Universal SQL Editor and saved it down in all available formats. No joy. Finally I ran out of bandwidth for the month and even worse I have run out of ideas.

I am not a computer person. The only reason that I ever get anything done is through being willing to stay up all night trying stuff out until something works. Please can someone help me and don't be afraid to talk to me as if I am an idiot, because I might well be one :-(.