There are a few problems with our release/jenkins scripts right now.
1. they are dispersed in 4 different places (debian/rules, release.sh, aegir jenkins scripts, vagrant scripts for aegir)
2. we use sometimes fabric, sometimes puppet, we should choose one or the other ( )
3. the official release process documentation tells us to use release.sh, but there's a changelog task in jenkins that does the same
4. we have multiple branches for 1.x, 2.x, this won't scale to multiple releases and makes it hard to port changes accross versions - we should merge all those branches and the scripts should support all releases with arguments
There are probably other things i'm missing here.
We want to refactor all tests to use puppet-aegir and a common codebase.
Smoke tests are in place in http://ci.aegirproject.org/view/Puppet/ - they just validate the manifests.
So we have to re-create the tests that were destroyed in the last jenkins upgrade (see).
We'd need to create a gigantic matrix test, with the following parameters:
* basebox distro (ubuntu 12.04, debian squeeze, debian wheezy)
* git + hostmaster-install vs debian package
* debian: 1.x, 2.x, 3.x (aegir, aegir2, aegir3)
* debian.aegirproject.org: stable/testing/unstable
* git: which ref? tag or branch
we also need:
* a job to automatically build debian packages and upload them to unstable (for 1.x, 2.x, etc)
* a job to do automatic upgrade tests (another matrix?)
those tests can either be triggered or ran daily.
one test that should be triggered at every commit would be to:
* run "update" (git pull and run tests) on a running VM
User interface changes
A much simpler set of tests.
There are too many fields here.