cleanup jenkins build scripts

Over the past couple weeks I've re-factored puppet-drush and puppet-aegir pretty significantly to:

• use parameterized classes, and otherwise modernize the code;
• remove external dependencies;
• improve support for Ubuntu & Nginx;
• add smoke-tests to ensure manifests compile

I've also added a tests/.ci directory in these puppet modules that contain a Vagrantfile and some scripts. These are set up to run the smoke tests from a "shell provisioning" script. It's a bit of an abuse of Vagrant, but it makes configuring Jenkins jobs absolutely trivial (basically, you just symlink the Vagrantfile into the project root), and pretty much self-contained in the puppet modules themselves.

I've taken a similar approach in aegir-up to provision a vm, install aegir using puppet-aegir (either from the .debs and 'manually' from the git repos), and then run our provision tests. I'll probably add comparable tests directly to puppet-aegir to run these tests independently of the extras added by aegir-up.

Admittedly, it takes somewhat longer to run tests this way, ~14 minutes from source vs. ~9 minutes previously (though not really an apples to apples comparison). However, we could cut this down significantly by using a basebox with dependencies (apache/nginx, mysql, php, etc.) already installed.

To get more immediate feedback on changes that don't necessarily affect the installation of Aegir itself, we could cut the test time down even further by using a basebox with Aegir already installed from source, and just update it before running tests.

My reasoning for pursuing this approach was largely to consolidate the various installation and test scripts, as well as to test and use the puppet modules more broadly. I'm setting this issue to 'needs review' in order to get feedback from other maintainers.

Yes, they're under the 'Puppet' tab.

link to related issues

I believe at least some of the tasks in the summary need to happen before 2.0. Although, the crucial ones might already have individual issues. If so, we should list them here, and downgrade the priority again.

Is #1380326: Vagrant-based test harness the same test idea as in #1 or an old attempt?

The installation of Aegir 2 by puppet seems to be working (http://ci.aegirproject.org/view/Puppet/).
Both the latest stable debian package and the sources from git.

What we still need:
* install 1.x and upgrade to 2.x ( both debian and from source)
* install 2.0 and upgrade to latest HEAD ( or stable release)

That could turn out to be quite a matrix.

Variables:
* basebox (Debian 7, Ubuntu last LTS, Ubuntu last stable, ...)
* webserver (apache, nginx)
* database? (mysql, maria)

issue summary

I'm thinking of doing two matix tests to start, one from the git sources and the other from .debs.

Common variables:
* basebox name
* basebox url
* webserver

git/hostmaster-install variables:
* branches 1.x, 2.x, 3.x

.deb-specific variables:
* package: aegir, aegir2, aegir3
* distribution: stable, testing, unstable

Our first matrix test is mostly working! See: http://ci.aegirproject.org/view/Puppet/job/Matrix.deb/

I had to hardcode the box urls in the Vagrantfile, and the current Wheezy one I have isn't working. I think upgrading Vagrant to 1.2+ will fix that though. We're also only testing Aegir 2 right now, since we're dropping support for Aegir 1, and don't have .debs for Aegir 3 yet.

I'll need to get a comparable set of tests running for git, but I'd like to abstract the Vagrantfile and setup/test scripts more, so they can be shared across all tests. That way, each test will only contain the things (puppet manifest, additional test scripts, etc.) that are really specific to it.

This all needs to be properly documented, which I'll do in a couple readme's. Also, the directory structure should be shuffled around to separate smoke tests from functional tests.

Our test appear to be working quite well, but we still have quite a few that we are no longer running. We should perhaps clean them out, to avoid confusion.

Yesterday I created http://ci.aegirproject.org/view/Stable builds/job/S_aegir-debian-official-3.x as a copy of http://ci.aegirproject.org/view/Stable%20builds/job/S_aegir-debian-offic...

Both jobs failed on the last test to check that the package file made it into the repository.
I say a problem with that and committed a fix to https://github.com/aegir-project/aegir_jenkins_scripts/commits/master

The root cause was better visible in the reprepro logfile, stored in /var/log/incoming.log

Error: trying to put version 'aegir3-provision' of '3.0~beta1' in 'sid|main|source',
while there already is the stricly newer '3.0~beta1+57.bdbe461' in there.
(To ignore this error add Permit: older_version.)
Action 'reprepro' returned exitcode 255

I'v now added Permit: older_version to the /srv/reprepro/conf/incoming configfile.

The log in now much cleaner ... only:

Will call action reprepro for: aegir3-provision_3.0~beta1_amd64.changes
Will call action reprepro for: aegir3-provision_3.0~beta1+60.bdbe461_amd64.changes

But still a failure.

The new aegir3-provision_3.0~beta1_all.deb packaged now ended up in the morgue dir.

I've added a note to the release process docs:

*note from just after 7.x-3.0-beta1*: It's probably a good idea to disable the Jenkins jobs named 'D_aegir-debian*'. This could prevent later troubles in the 'Publish the Debian packages' section.

And now that I've finally bent by brain around this ... my steps:

1. reprepro@zeus:~$ reprepro remove unstable aegir3-cluster-slave aegir3 aegir3-provision aegir3-hostmaster
2. Trigger a new official build, http://ci.aegirproject.org/view/Stable%20builds/job/S_aegir-debian-offic...
3. Copy the good beta1 build to stable:
   

   reprepro@zeus:~$ reprepro copy stable unstable aegir3
   reprepro@zeus:~$ reprepro copy stable unstable aegir3-hostmaster
   reprepro@zeus:~$ reprepro copy stable unstable aegir3-provision
   reprepro@zeus:~$ reprepro copy stable unstable aegir3-cluster-slave
   

4. Trigger a new dev build to get the latest snapshot in there again.
5. http://ci.aegirproject.org/job/D_aegir-debian-build-3x

#2859549: Deprecate or move Jenkins

I added https://github.com/aegir-project/aegir_jenkins_scripts/blob/master/READM... with a link to gitlab CI