By FortEZ on

I have recently come up with this error in my production site and it has almost driven me mad for a full morning. Let me explain:

My company's main production site is build on EC2. We have a customized AMI that contains latest stable PHP MySQL and Apache versions and our drupal site (currently 6.14). Our update policy when it comes to adding modules, heavy theme modifications etc is the following:

- The developer that is responsible for the update, downloads the website locally, makes the changes in a virtual Ubuntu server, tests various settings, stability etc and determines whether the changes are safe to be implemented in our production site.

- Then we start a copy of our AMI, with an exact copy of the hard disk volume, apply the changes and test them once more with the AMI's configuration

- Finally, if there are no problems, we change the elastic IP to point to the new AMI.

The problem started when I tried to login to our website as admin. Here is how the issue manifested:

- Tried to login with the admin account: I was forwarded to the mysite/admin page where a message appeared: you are not authorized to access this page.

- I though that I miss-entered the pass so I tried to recover the password. Still the same message

- Then I realized that the username of the login form was not turning red like when you are entering the wrong password. So I figured that the problem was not the password but something else.

- I spent almost 4 hours trying to figure out what went wrong. I launched a clone of our live AMI and experimented on it... A LOT! I wasn't able to find the exact same problem reported anywhere, but there were some issues that were pretty close to what I was dealing with. So I tried running the update.php (had to change the settings.php file to allow me running it without being logged in), tried adding some stuff in the page.tpl.php page, and lots and lots of other things.

- Finally, I started looking around in the database through phpmyadmin pretty much blindly. In a post someone mentioned something about the "sessions" table in the database so I took a look at it. Phpmyadmin threw an error that the table was corrupted.

I truncated the specific table and the issue was resolved. I wanted to post this in the forums so that if any1 else comes up with this issue he will know what to do.

And a quick question: does anybody know how a table in the database can become corrupted suddenly? any possible reasons for the issue?

⋅ Categories: Drupal 6.x