Ubuntu 9.10 install.sh fail at hostmaster export (cvs)

Posted by carlthuringer on December 1, 2009 at 5:42pm

Issue Summary

Still having trouble. I'm using hostmaster HEAD from the git.aegirproject.org/hostmaster.git repository.

The install goes through and downloads drupal, then hangs up on a CVS command. I'd try some variations on downloading it, except that I don't really know CVS well enough to write my own checkout request.

I saw a similar hangup in #646426: install.sh fails at hostmaster deployment on Ubuntu #7 and assumed it was a server error related to yesterday's downtime (for profile deployment on d.o) but it's still happening to me.

aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.localhost
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] n
aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.localhost
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] y
==> Creating basic directory structure
==> Installing drush in /var/aegir
--2009-12-01 12:32:08--  http://ftp.drupal.org/files/projects/drush-All-Versions-2.1.tar.gz
Resol... ftp.drupal.org... 64.50.236.52
Connecting to ftp.drupal.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 80209 (78K) [application/x-gzip]
Saving to: `drush-All-Versions-2.1.tar.gz'

100%[===================================================================================>] 80,209      34.6K/s   in 2.3s    

2009-12-01 12:32:12 (34.6 KB/s) - `drush-All-Versions-2.1.tar.gz' saved [80209/80209]

==> Drush seems to be functionning properly
==> Installing drush make in /var/aegir/.drush
Project drush_make (6.x-2.0-beta5) downloaded to /var/aegir/.drush/. [success]
==> Installing provision backend in /var/aegir/.drush
Project provision (6.x-0.4-alpha3) downloaded to /var/aegir/.drush/. [success]
==> Deploying hostmaster application
Project information for drupal retrieved.                            [ok]
drupal downloaded from                                               [ok]
http://ftp.drupal.org/files/projects/drupal-6.14.tar.gz.
Unable to export hostmaster from                                     [error]
pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib
contributions/profiles/hostmaster.
An error occurred at function : drush_drush_make_make                [error]
An error occurred at function : drush_provision_hostmaster_make      [error]
You have new mail in /var/mail/aegir
aegir@carl-dannegar:~/hostmaster$ cd ..
aegir@carl-dannegar:~$ ls -a
.  ..  backups	.bash_history  config  drush  .drush  hostmaster  hostmaster-6.x-0.4-alpha3  .nano_history
aegir@carl-dannegar:~$ cd hostmaster-6.x-0.4-alpha3/
aegir@carl-dannegar:~/hostmaster-6.x-0.4-alpha3$ ls
CHANGELOG.txt  includes		  INSTALL.pgsql.txt  LICENSE.txt      modules	  scripts  update.php
COPYRIGHT.txt  index.php	  install.php	     MAINTAINERS.txt  profiles	  sites    UPGRADE.txt
cron.php       INSTALL.mysql.txt  INSTALL.txt	     misc	      robots.txt  themes   xmlrpc.php
aegir@carl-dannegar:~/hostmaster-6.x-0.4-alpha3$ cd profiles
aegir@carl-dannegar:~/hostmaster-6.x-0.4-alpha3/profiles$ ls
default

Comments

#1

Posted by carlthuringer on December 1, 2009 at 5:45pm

Oh, and if I give install.sh.txt another swing at this current state, it stops at apache2.conf.txt missing

aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.localhost
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] y
==> Creating basic directory structure
/usr/bin/drush
==> Drush is in the path, good
==> Drush seems to be functionning properly
==> Drush make already seems to be installed
==> Provision already seems to be installed
sed: can't read /var/aegir/hostmaster-6.x-0.4-alpha3/profiles/hostmaster/apache2.conf.txt: No such file or directory

#2

Posted by carlthuringer on December 1, 2009 at 6:00pm

Finally, if I give install.sh.txt a third swing, it announces completion. But the profiles directory still doesn't have the hostmaster profile!

aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.localhost
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] y
==> Creating basic directory structure
/usr/bin/drush
==> Drush is in the path, good
==> Drush seems to be functionning properly
==> Drush make already seems to be installed
==> Provision already seems to be installed
==> Install process complete: follow the wizard
Now point your browser to http://aegir.localhost/install.php and proceed
with the remainder of the installation using the Hostmaster Install profile.
Some of the instructions given, you will already have completed, but carefully
read each step in turn to ensure you don't miss anything.
aegir@carl-dannegar:~/hostmaster$ cd ..
You have new mail in /var/mail/aegir
aegir@carl-dannegar:~$ cd hostmaster-6.x-0.4-alpha3/
aegir@carl-dannegar:~/hostmaster-6.x-0.4-alpha3$ cd profiles
aegir@carl-dannegar:~/hostmaster-6.x-0.4-alpha3/profiles$ ls
default

I think that the chunk that calls $DRUSH hostmaster make #HOSTMASTER_DIR should look a little deeper for its if statement. To test this, I tried editing that line to read:

if [ -e $HOSTMASTER_DIR/profiles/hostmaster ]

The script went back to attempting to deploy hostmaster in the drupal install that was downloaded, but fails with the old dash/drush2.1 error...

aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.localhost
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] Y
==> Creating basic directory structure
/usr/bin/drush
==> Drush is in the path, good
==> Drush seems to be functionning properly
==> Drush make already seems to be installed
==> Provision already seems to be installed
==> Deploying hostmaster application
The command could not be executed successfully (returned: /var/aegir/drush/drush.php: 1: cannot open ?php: No such[error]
file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: HINTS_CentOS.txt: not found
/var/aegir/drush/drush.php: 6: HINTS_CentOS.txt: not found
/var/aegir/drush/drush.php: 7: HINTS_CentOS.txt: not found
/var/aegir/drush/drush.php: 8: HINTS_CentOS.txt: not found
/var/aegir/drush/drush.php: 9: modules/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
, code: 2)
An error occurred at function : drush_provision_hostmaster_make                                                   [error]

Which before had seemed to happen whenever I tried launching install.sh.txt on a partial install of aegir/hostmaster.

#3

Posted by lucidus_neil on December 2, 2009 at 4:00am

Was having a similar problem with Centos installation. However, I suspect the problem is directly related to last night's downtime. d.o fundamentally changed the manner in which installation profiles are downloaded. If I run the install script, it pulls down an entire drupal distribution into the hostmaster directory minus the installation profile (which seems weird). I'm wondering whether it has to do with the all inclusive install mechanism implemented for installation profiles last night.

#4

Posted by mig5 on December 2, 2009 at 4:08am

The issue isn't related to the infrastructure changes at drupal.org in any way.

It is related to a change that was made to drush_make in recent times. The latest version of drush_make prevents our 0.4alpha3 release from being built, as it tries to fetch hostmaster from CVS and fails. It is a problem that needs to be fixed in drush_make since our release is already out. See #646748: old makefiles using cvs don't specify the first : colon in the cvsroot, thus breaking in new versions of drush_make

I recommend everyone switch to using our git repositories (in effect, using git's equivalent of 'HEAD'), until we make a new release. The use of the INSTALL.txt and install.sh from git, avoid this error.

INSTALL.txt
install.sh

Alternatively, stick to the 0.3 release for production systems, as this pre-dates the use of drush_make (the installation is a more manual process).

I'll leave this ticket open but there is really unfortunately nothing we can do with this, and future releases will be using git and so avoid this problem. When the Drush Make ticket is marked fixed, I'll mention it here and close the ticket.

Also: current version of drush_make also (at time of this writing) depends on php5-curl, but shouldn't (see #647546: drush_make 6.x-2.0-beta5 depends on php curl libraries, should seek alternatives) . We aren't marking it as a dependency as a result, however right now, you will need it installed to complete the execution of the install.sh script.

#5

Posted by carlthuringer on December 2, 2009 at 1:05pm

Using unedited install.sh from GIT repository. Failed with 'cannot open ?php: no such file'

aegir@carl-dannegar:~$ ls -a
.  ..
aegir@carl-dannegar:~$ git clone git://git.aegirproject.org/hostmaster.git
Initialized empty Git repository in /var/aegir/hostmaster/.git/
remote: Counting objects: 4088, done.
remote: Compressing objects: 100% (1199/1199), done.
remote: Total 4088 (delta 2915), reused 3997 (delta 2860)
Receiving objects: 100% (4088/4088), 784.04 KiB | 39 KiB/s, done.
Resolving deltas: 100% (2915/2915), done.
aegir@carl-dannegar:~$ cd hostmaster
aegir@carl-dannegar:~/hostmaster$ sh install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.example.com
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] y
==> Creating basic directory structure
==> Installing drush in /var/aegir
--2009-12-02 07:53:56--  http://ftp.drupal.org/files/projects/drush-All-Versions-2.1.tar.gz
Resol... ftp.drupal.org... 64.50.236.52
Connecting to ftp.drupal.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 80209 (78K) [application/x-gzip]
Saving to: `drush-All-Versions-2.1.tar.gz'

100%[===================================================================================>] 80,209      35.0K/s   in 2.2s    

2009-12-02 07:54:00 (35.0 KB/s) - `drush-All-Versions-2.1.tar.gz' saved [80209/80209]

==> Drush seems to be functionning properly
==> Installing drush make in /var/aegir/.drush
Project drush_make (6.x-2.0-beta5) downloaded to /var/aegir/.drush/. [success]
==> Installing provision backend in /var/aegir/.drush
Project provision (6.x-0.4-alpha3) downloaded to /var/aegir/.drush/. [success]
==> Deploying hostmaster application
The command could not be executed successfully (returned:            [error]
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: backups: not found
/var/aegir/drush/drush.php: 6: backups: not found
/var/aegir/drush/drush.php: 7: backups: not found
/var/aegir/drush/drush.php: 8: backups: not found
/var/aegir/drush/drush.php: 9: backups/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
, code: 2)
An error occurred at function : drush_provision_hostmaster_make      [error]
You have new mail in /var/mail/aegir

Using unedited install.sh with bash instead of sh. Fails at the mentioned drush_make CVS error.

aegir@carl-dannegar:~$ ls -a
.  ..
aegir@carl-dannegar:~$ git clone git://git.aegirproject.org/hostmaster.git
Initialized empty Git repository in /var/aegir/hostmaster/.git/
remote: Counting objects: 4088, done.
remote: Compressing objects: 100% (1199/1199), done.
remote: Total 4088 (delta 2915), reused 3997 (delta 2860)
Receiving objects: 100% (4088/4088), 784.04 KiB | 38 KiB/s, done.
Resolving deltas: 100% (2915/2915), done.
aegir@carl-dannegar:~$ cd hostmaster
aegir@carl-dannegar:~/hostmaster$ bash install.sh.txt
==> Aegir automated install script
==> This script makes the following assumptions: 
 * you have read INSTALL.txt and prepared the platform accordingly
 * you are running as your "aegir" user
 * the following settings are correct:
AEGIR_DOMAIN=aegir.example.com
AEGIR_VERSION=6.x-0.4-alpha3
AEGIR_HOME=/var/aegir
WEB_GROUP=www-data
HOSTMASTER_DIR=/var/aegir/hostmaster-6.x-0.4-alpha3
DRUSH=/var/aegir/drush/drush.php
DRUSH_VERSION=All-Versions-2.1

Some of those settings can be changed on the commandline, see:

 install.sh.txt -h

for more information.
Do you want to proceed with the install? [Y/n] Y
==> Creating basic directory structure
==> Installing drush in /var/aegir
--2009-12-02 07:57:10--  http://ftp.drupal.org/files/projects/drush-All-Versions-2.1.tar.gz
Resol... ftp.drupal.org... 64.50.236.52
Connecting to ftp.drupal.org|64.50.236.52|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 80209 (78K) [application/x-gzip]
Saving to: `drush-All-Versions-2.1.tar.gz'

100%[===================================================================================>] 80,209      34.1K/s   in 2.3s    

2009-12-02 07:57:14 (34.1 KB/s) - `drush-All-Versions-2.1.tar.gz' saved [80209/80209]

==> Drush seems to be functionning properly
==> Installing drush make in /var/aegir/.drush
Project drush_make (6.x-2.0-beta5) downloaded to /var/aegir/.drush/. [success]
==> Installing provision backend in /var/aegir/.drush
Project provision (6.x-0.4-alpha3) downloaded to /var/aegir/.drush/. [success]
==> Deploying hostmaster application
Project information for drupal retrieved.                            [ok]
drupal downloaded from                                               [ok]
http://ftp.drupal.org/files/projects/drupal-6.14.tar.gz.
Unable to export hostmaster from                                     [error]
pserver:anonymous:anonymous@cvs.drupal.org:/cvs/drupal-contrib
contributions/profiles/hostmaster.
An error occurred at function : drush_drush_make_make                [error]
An error occurred at function : drush_provision_hostmaster_make      [error]
You have new mail in /var/mail/aegir

So I don't see that using install.sh.txt from the GIT repository avoids this error.

#6

Posted by Mark Trapp on December 2, 2009 at 4:21pm

With a hint from Vertice on IRC, I was able to get aegir to successfully install by changing install.sh.txt to download 2.0b3 of drush_make instead of the latest version. Change:
$DRUSH dl drush_make --destination=$AEGIR_HOME/.drush
to
$DRUSH dl drush_make-6.x-2.0-beta3 --destination=$AEGIR_HOME/.drush
Remove all files from $AEGIR_HOME (make sure you kill $AEGIR_HOME/.drush as well), and re-run install.sh.txt.

#7

Posted by carlthuringer on December 2, 2009 at 6:09pm

Status:

active

» needs work

Aha, I was just spending some time earlier today poking through the files in the GIT package, trying to see if I could figure out exactly where things were choking up.

I had come across that drush_make version but didn't know what to do to force it to download that instead of the standard one, which I assume is the recommended one.

I tried it with your recommended edit, but ran into the same issue. I had to roll it back to beta2, which coincides with the beginning of my (and others') issues with CVS.

$DRUSH dl drush_make-6.x-2.0-beta2 --destination=$AEGIR_HOME/.drush

I used hostmaster HEAD from the GIT repository at git.aegirproject.org. I edited the site domain to be aegir.localhost and changed /bin/sh to /bin/bash at the top of the file. I also modified the above code as written, to get drush make 2.0 beta2. I used 'bash install.sh.txt' as my execution command.

And it worked.

I wish I could flag this as 'workaround' but for now I'll put it as 'needs work'

#8

Posted by mig5 on December 2, 2009 at 9:06pm

Well your ?php no such file is a symptom of running Dash as the default shell in Ubuntu. Causing problems due to Drush2.1, which has been fixed in HEAD and nothing we can do about it. Hence the /bin/bash workaround.

You then experience the 'Unable to export' error which is specific to CVS, you say:

Using unedited install.sh with bash instead of sh. Fails at the mentioned drush_make CVS error.

This has to be not the version in Git, but rather the 0.4alpha3 release, because it is a symptom of fetching from CVS, and nothing in our GIT repos fetches from CVS anymore.

The error causing that is a change in drush_make. We could fix the makefile for the 0.4alpha3 release but we already released it :)

So there is not really any 'work' that can be done, save:

Dash issues:

1) Wait for a new Drush release that contains a fix, or
2) Patch it per #637574: drush finds the wrong php executable when executing under dash

Drush_make 'Unable to export' issues:

1) Use the latest development version from GIT, which is immune to this issue since it doesn't use CVS, or
2) Use 0.4alpha3 but edit the script to use an older version of Drush Make, or
3) Use the 0.3 release of Aegir, which predates Drush Make.

If Drush Make makes a new release, the 'Unable to export' issue when installing 0.4alpha3 will go away. Then just the Drush/Dash workaround will need to be applied for people using Ubuntu (until a new release of Drush comes out).

#9

Posted by Mark Trapp on December 3, 2009 at 3:12am

carthuringer:

Good to know; I had used beta2, but Vertice had mentioned in IRC that beta3 should work. It looks like beta 2 was a few days before the commit mentioned in comment #5 of #646748: old makefiles using cvs don't specify the first : colon in the cvsroot, thus breaking in new versions of drush_make was made, which seems to be the root of all the issues with drush_make.

#10

Posted by mig5 on March 2, 2010 at 10:43am

Status:

needs work

» closed (fixed)

Alphas since alpha3 are fetched via wget, or git clone of the bleeding edge stuff. Plus, the format change is unlikely to be fixed retrospectively in Drush Make as it was incorrect to begin with.

Closing this as nothing else will, nor needs to be done.

Project:	Hostmaster (Aegir)
Version:	6.x-0.4-alpha3
Component:	Code
Category:	bug report
Priority:	normal
Assigned:	Unassigned
Status:	closed (fixed)

Ubuntu 9.10 install.sh fail at hostmaster export (cvs)

Jump to:

Issue Summary

Comments