Using Field Group in a Custom Content Type Programmatically

Yes.
If you mean programmatically add a new field group type. See field_group.api.php
If you mean having field groups in code, you can have default fieldgroups in a hook (hook_field_group_info) or through features.
If you mean, having one at runtime, then no, but that would be a theming concept.

I can see hook_field_group_info() on field_group.api.php but it does not explain what the hook must return.

How do I describe/create the groups and assign them to entities?
How do I nest groups ?
How do I describe which fields go in each group?

Thanks.

Sorry to repeat, forgot to set the right version.

I can see hook_field_group_info() on field_group.api.php but it does not explain what the hook must return.

How do I describe/create the groups and assign them to entities?
How do I nest groups ?
How do I describe which fields go in each group?

Thanks.

Subscribing, some examples for hook_field_group_info() would be really nice.

Demo for groups of "webform" content type

/**
* Implements hook_field_group_info().
*/
function custom_module_field_group_info() {
$export = array();

$field_group = new stdClass();
$field_group->disabled = FALSE; /* Edit this to true to make a default field_group disabled initially */
$field_group->api_version = 1;
$field_group->identifier = 'group_NAME_OF_GROUP|node|webform|form';
$field_group->group_name = 'group_NAME_OF_GROUP';
$field_group->entity_type = 'node';
$field_group->bundle = 'webform';
$field_group->mode = 'form';
$field_group->parent_name = '';
$field_group->data = array(
'label' => 'NAME OF GROUP',
'weight' => '1',
'children' => array(
0 => 'field_YOUR_FIELD',
),
'format_type' => 'fieldset',
'format_settings' => array(
'formatter' => 'collapsible',
'instance_settings' => array(
'description' => '',
'classes' => '',
'required_fields' => 1,
),
),
);
$export['group_NAME_OF_GROUP|node|webform|form'] = $field_group;

return $export;
}

I used the code from #7 (in mymodule.module) but the hook is not called.
The groups are not created and there is no entry in the database table field_group.
Do I have to invoke the hook manually?
Thanks

You also need to implement hook_ctools_plugin_api():

/**
 * Implements hook_ctools_plugin_api().
 */
function YOUR_MODULE_ctools_plugin_api($owner, $api) {
  if ($owner == 'field_group' && $api == 'field_group') {
    return array('version' => 1);
  }
}

Latest comment should idd fix the problem.