barracuda_log.txt - http://pastebin.com/5YNi1jBM
octopus_log.txt - http://pastebin.com/jTM2puef
/root/.o1.octopus.cnf - http://pastebin.com/LL3QeEJs

I read some of the other issues in the queue relating to this issue but none of the mentioned solutions seem to resolve my issue.

The actual message I'm recieving is:
UPGRADE B: Hostmaster STATUS: upgrade completed
UPGRADE B: Simple check if Aegir upgrade is successful
UPGRADE B: FATAL ERROR: Required file /data/disk/o1/aegir/distro/006/sites/o1.server.com/settings.php does not exist
UPGRADE B: FATAL ERROR: Aborting AegirSetupB installer NOW!
UPGRADE A: FATAL ERROR: AegirSetupB installer failed
UPGRADE A: FATAL ERROR: Aborting AegirSetupA installer NOW!
FATAL ERROR: AegirSetupA installer failed
FATAL ERROR: Aborting Octopus installer NOW!

I removed the /opt/tmp/*FAIL files and the auto created additional 00x folders in /data/disk/o1/aegir/distro/ so only the actual hostmaster install directory exists and the previous one but this resulted in the same error.

Another thing I noticed was early in the installation I got error: SQLSTATE[28000] [1045] Access denied for user 'o1'@'localhost' (using password: YES) and now that Aegir isn't able to carry out normal tasks.

Any ideas?

Comments

omega8cc’s picture

Please try to run on command line, *a few times*:

randpass 32 esc

Then a few times:

randpass 32 alnum

Then a few times:

pwgen -v -s -1

And post the output, so we could get a better idea on what happens there.

Then please let us know the output of commands shown below - don't worry, all passwords are re-generated on every upgrade automatically:

cat /var/aegir/backups/system/.aegir_root.pass.txt
su -s /bin/bash - aegir -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /var/aegir/.drush/server_localhost.alias.drushrc.php
cat /data/disk/o1/.o1.pass.txt
su -s /bin/bash - o1 -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php
omega8cc’s picture

Status: Active » Postponed (maintainer needs more info)

Please enable debugging and post full logs displayed in the terminal.

If you will get different passwords displayed/stored in the database and the config files for master and/or satellite instance, you will need to use our syncpass tool, as explained below.

1. Please sync passwords first:

cd
rm -f BOA.sh.txt
wget -q -U iCab http://files.aegir.cc/BOA.sh.txt
bash BOA.sh.txt
syncpass fix aegir (only if master instance is also affected)
syncpass fix o1

2. Then run the upgrade:

barracuda up-head (only if master instance was also affected)
octopus up-head o1 both

3. Finally, post the results again:

cat /data/disk/o1/.o1.pass.txt
su -s /bin/bash - o1 -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php
snlnz’s picture

Here's the output from the first commands:
randpass / pwgen output - http://pastebin.com/b7q4D0am

cat /var/aegir/backups/system/.aegir_root.pass.txt
su -s /bin/bash - aegir -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /var/aegir/.drush/server_localhost.alias.drushrc.php

http://pastebin.com/L1v9qKWb

cat /data/disk/o1/.o1.pass.txt
su -s /bin/bash - o1 -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php

http://pastebin.com/K34c0QpQ

barracuda upgrade appeared to work fine but noticed access denied root@localhost error during upgrade.
http://pastebin.com/3iZ8Cqqm

octopus up-head failed
http://pastebin.com/4ZgUN0hn

Final output

cat /data/disk/o1/.o1.pass.txt
su -s /bin/bash - o1 -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php

http://pastebin.com/kF0HBVz2

Is there anywhere the entire install is logged? From memory it was there once upon a time but now it doesn't seem to exist unless the "by design" has moved it.

snlnz’s picture

If it's any help, I can no longer do any Aegir tasks as getting error:

SQLSTATE[28000] [1045] Access denied for user 'o1'@'localhost' (using password: YES)

This starting happening immediately after the failed octopus upgrade prior to logging this issue.

omega8cc’s picture

Did you modify the output in any way?

nz:~# grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php 
  'master_db' => 'mysql://oct:6jrj2iAb@localhost',

Is there really mysql://oct:? It should be mysql://o1:

Also, you didn't provide all requested information:

cat /data/disk/o1/.o1.pass.txt

omega8cc’s picture

Also this looks wrong;

su -s /bin/bash - aegir -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
ERROR 1146 (42S02) at line 1: Table 'fqsn.hosting_db_server' doesn't exist

The table hosting_db_server should exist.

omega8cc’s picture

Can you access mysql root by typing on command line: mysql ?

This ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES) shouldn't happen. Have you tried to change your mysql root password manually?

omega8cc’s picture

You should *not* try HEAD right now, because it doesn't include all fixes from BOA-2.0.6 and BOA-2.0.7, because we couldn't commit them to git yet.

omega8cc’s picture

Please provide *real* output of requested commands or we will not be able to assist you.

This looks obviously fake, because "oct" user can't display results for "o1" user.

su -s /bin/bash - oct -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
vid	nid	db_user	db_passwd
4	4	o1	eq8plWHy
nz:~# grep master_db /data/disk/o1/.drush/server_localhost.alias.drushrc.php
  'master_db' => 'mysql://o1:eq8plWHy@localhost',

Plus, you didn't provide *required* information, again:

cat /data/disk/o1/.o1.pass.txt

omega8cc’s picture

Anyway, everything will just fail, because your mysql root password is incorrect. I can only guess that you have changed it manually(?) before but you didn't update essential files: /root/.my.cnf and /root/.my.pass.txt

snlnz’s picture

I had modified the username output from o1 to oct but for the sake of getting this resolved here it is.

# cat /data/disk/oct/.oct.pass.txt
eq8plWHy
# grep master_db /data/disk/oct/.drush/server_localhost.alias.drushrc.php 
  'master_db' => 'mysql://oct:eq8plWHy@localhost',

I have modified the servername.domain output

# su -s /bin/bash - aegir -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
ERROR 1146 (42S02) at line 1: Table 'MYSERVERNAME.hosting_db_server' doesn't exist
Can you access mysql root by typing on command line: mysql ?

No

# mysql
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
You should *not* try HEAD right now, because it doesn't include all fixes from BOA-2.0.6 and BOA-2.0.7, because we couldn't commit them to git yet.

Sorry was just following your instructions at that time.

This looks obviously fake, because "oct" user can't display results for "o1" user.

# su -s /bin/bash - oct -c "drush @hostmaster sqlq \"SELECT * FROM hosting_db_server\""
vid	nid	db_user	db_passwd
4	4	oct	eq8plWHy
Anyway, everything will just fail, because your mysql root password is incorrect. I can only guess that you have changed it manually(?) before but you didn't update essential files: /root/.my.cnf and /root/.my.pass.txt

No I didn't, this is a result of the upgrade failing. I don't change passwords like this manually.

snlnz’s picture

Status: Postponed (maintainer needs more info) » Fixed

I resolved the upgrade issue by resetting the mysql root password.

service mysql stop
mysqld_safe --skip-grant-tables
mysql --user=root mysql
update user set Password=PASSWORD('contents_of_/root/.my.cnf_goes_here') where user='root';
flush privileges;
exit;

After that I was able to complete the upgrade successfully.

omega8cc’s picture

Status: Fixed » Postponed (maintainer needs more info)

No, wait, this shouldn't happen.

Did the /root/.my.cnf include some too weird characters or something? Barracuda should update the mysql root password once it is re-generated on upgrade.

snlnz’s picture

No it didn't and I'm not sure what happened to be honest.
It wasn't until I manually reset it and added it to the /root/.my.cnf file and /root/.my.pass.txt then only the upgrade would work and reset the passwords accordingly.

omega8cc’s picture

Status: Postponed (maintainer needs more info) » Closed (cannot reproduce)

I don't see any reason why this happened, then.