I thought there was just a load of functionality missing, but it turns out that there was just a lot of stuff I didn't know - and wasn't documented.
I'll collate some notes here - from an installer-person perspective and maybe end up with something that could turn into a README.

The case in point is already live - http://features.sparksinteractive.co.nz/
version = "6.x-1.1"
But I'm trying to sort out some processes internally here.

I'm using a semi-private git repo as the VCS, and selectively pushing a library of features out to the (local) world. But I had some misunderstandings.

Where is the packaging magic?

Yes, there IS actually automatic packaging built into the system, but it doesn't actually happen until you do some hidden steps.
Previously, I tried making projects, linking them to a git repo ... but nothing clever happened apart from some makefile information being published.
So I've been manually creating releases (from my git 'master' or '6.x-2.x' branch) and uploading handmade (well, gitorious-made) tarballs.
There must be an easier way.
Today I opened up the code and found clues.

You MUST not just branch, but also TAG your project code

No packaging will happen until you TAG stuff.
And push it too.

git tag -m "tagging it for the feature server release. Still unstable for now" 6.x-1.1
git push origin 6.x-1.1

The -m message is optional, but I think git will like to see it in the system. Perhaps it will become a release note.

Also - I'm not allowed to number anything 0.1, 0.2 . I thought I was being clever by tagging it as pre-release ... but that doesn't fit the expected naming scheme ... So I guess I've got to start using alpha1 or something, though I never really got the hang of those.

Next ...
I sorta suspect that cron should do the rest, but no.

But I see evidence of drush support. I have to guess that I'm expected to run fserver-package?

cd $DRUPAL_ROOT/sites/features.sparksinteractive.co.nz
drush -v --uri=features.sparksinteractive.co.nz fserver-package 'nzgls_metadata'

Undefined property: FserverProject_git::$node fserver.drush.inc:398                                                                                 [notice]
Trying to get property of non-object fserver.drush.inc:398                                                                                          [notice]
Undefined index: extra fserver.drush.inc:240                                                                                                        [notice]
Undefined index: list content.module:1031                                                                                                           [notice]
Undefined index: data content.module:1031                                                                                                           [notice]
sprintf(): Too few arguments drush.inc:1321                                                                                                      [warning]
Executing: 
Undefined offset: 0 fserver.drush.inc:363                                                                                                           [notice]
Executing: git --git-dir=sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747/.git tag -l 6.x-1.1 -n 1000
sprintf(): Too few arguments drush.inc:1321                                                                                                      [warning]
Executing: 
Undefined offset: 0 fserver.drush.inc:363                                                                                                           [notice]
Executing: git --git-dir=sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747/.git tag -l 6.x-1.1 -n 1000
Executing: git --git-dir=sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747/.git checkout 6.x-1.1
Executing: tar -Pcf - --exclude='.git' --transform='s,sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747,nzgls_metadata,' sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747 | gzip > sites/features.sparksinteractive.co.nz/files/fserver/nzgls_metadata-6.x-1.1.tgz
chmod(): Operation not permitted drush.inc:1216                                                                                                  [warning]
chgrp(): Operation not permitted drush.inc:1216                                                                                                  [warning]
Undefined property: FserverProject_git::$node fserver.drush.inc:398                                                                                 [notice]
Trying to get property of non-object fserver.drush.inc:398                                                                                          [notice]
Undefined index: extra fserver.drush.inc:240                                                                                                        [notice]
Undefined index: list content.module:1031                                                                                                           [notice]
Undefined index: data content.module:1031                                                                                                           [notice]
 Project         Release  Filename                   
 NZGLS Metadata  6.x-1.0  nzgls_metadata-6.x-1.0.tgz 
 NZGLS Metadata  6.x-1.1  nzgls_metadata-6.x-1.1.tgz 

Command dispatch complete                                                                                                                           [notice]
Executing: rm -rf sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320184747

Yeah, this is not looking encouraging.
I got releases, but no actual content in their tarballs.
The user I was running drush as DID have access to the repo.
eg, running git ls-remote --tags git@git.sparksinteractive.co.nz:dmans-dev/nzgls_metadata.git as I see fserver is doing works correctly. It can detect that there are tagged versions to build ... but there are a number of possible issues. chmod permissions I can expect, but the rest?
I'll get onto them myself now, but any pointers as to what I'm missing would be helpful. Hints anyone?

Right now, it seems that fserver doesn't really want me to release -dev versions? I can live with that ... but couldn't find that documented.

CommentFileSizeAuthor
#3 fserver-escape_shell_arguments-1328774-20111102.patch3.08 KBdman

Comments

dman’s picture

dman commented

The -m message did turn up in the release notes. That is excellent.

dman’s picture

dman commented

Turns out part of my issue is because I don't run a 'master' branch on my new development, I switch directly into 6.x-1.x when initializing the git repo.
This is my own fault a bit - it leaves me in a state a bit like #1074584: warning: remote HEAD refers to nonexistent ref, unable to checkout.

Looking for the right way to deal with it (I don't want a dummy master branch getting in the way)

I discovered this by trying the verbose drush steps by hand, eg:

git clone git@git.sparksinteractive.co.nz:dmans-dev/nzgls_metadata.git sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320183748

Cloning into sites/features.sparksinteractive.co.nz/files/tmp/fserver_git_nzgls_metadata_1320183748...
remote: Counting objects: 14, done.
remote: Compressing objects: 100% (14/14), done.
remote: Total 14 (delta 2), reused 0 (delta 0)
Receiving objects: 100% (14/14), 4.15 KiB, done.
Resolving deltas: 100% (2/2), done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.

These are notes to self, just tracing some troubleshooting steps, maybe this trail will help others who have to search for it...

dman’s picture

dman commented
Title: The missing Manual » The missing Manual + plus security patch
Status: Active » Needs review
StatusFileSize
new fserver-escape_shell_arguments-1328774-20111102.patch3.08 KB

OK, I found the issues that PHP and drush were bitching about .. and they had a point.
Current code (6.x-dev and 7.x) does not take care to escape or quote the parameters that are being passed directly to the commandline. Y'know - this is dangerous. Only if someone manages to insert funny characters into our tagnames in the VCS, but still, best practice and all.

So ... sprintf format is damn ugly, but it's what drush_shell_exec() uses. so here's a set of replacements, eg

-      drush_op('drush_shell_exec', "git clone {$this->FserverProject->repository} {$this->path}");
+      drush_op('drush_shell_exec', "git clone %s %s", $this->FserverProject->repository, $this->path);

No improvement in readability, but it does stop PHP and drush from throwing warnings.

This issue wasn't just a notice, it was a full error in this line, due to the % in the input:

-      drush_op('drush_shell_exec', "git --git-dir={$this->path}/.git log -1 --pretty=format:%ct {$vc}");
+      drush_op('drush_shell_exec', "git --git-dir=%s log -1 --pretty=format:%s %s", $this->path . '/.git', '%ct', $vc);

ALSO - it looks like this has been fixed in D7, so I've backported the following: --work-tree was required.

-          drush_op('drush_shell_exec', "git --git-dir={$this->path}/.git checkout {$vc}");
+          drush_op('drush_shell_exec', "git --git-dir=%s --work-tree=%s checkout %s", $this->path . '/.git', $this->path, $vc);

Now, finally, I am getting results as they should be, with only minimal complaints about file permissions (my own issue)

Remaining ... why isn't this done on cron, or should I set up my own cronjob on the system?

dominikb1888’s picture

dominikb1888 commented

Very nice writeup! Hope the changes get committed soon.

eugenmayer’s picture

eugenmayer commented

looks good guys. I will reserve me some time to actually commit all those fixed and extensions to the project and rerelease. Good work!

eugenmayer’s picture

eugenmayer commented
Priority: Normal » Major
Status: Needs review » Needs work

we will need to reroll the patch against the current 6.x-1.x dev, as some commands got changed in the meantime.