Can't add or revoke rule in Security Group

Here is the nova-api log trace too. Will have to investigate further.

2012-12-20 00:03:19 DEBUG nova.utils [-] Reloading cached file /etc/nova/policy.json from (pid=30303) read_cached_file /opt/stack/nova/nova/utils.py:1000
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] action: AuthorizeSecurityGroupIngress from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:339
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] arg: IpPermissions.ToPort         val: 0 from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:341
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] arg: GroupName            val: default from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:341
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] arg: IpPermissions.IpRanges.CidrIp val: 0.0.0.0/0 from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:341
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] arg: IpPermissions.FromPort        val: 0 from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:341
2012-12-20 00:03:19 DEBUG nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] arg: IpPermissions.IpProtocol      val: tcp from (pid=30303) __call__ /opt/stack/nova/nova/api/ec2/__init__.py:341
2012-12-20 00:03:19 ERROR nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] Unexpected error raised: _rule_dict_last_step() argument after ** must be a mapping, not str
2012-12-20 00:03:19 TRACE nova.api.ec2 Traceback (most recent call last):
2012-12-20 00:03:19 TRACE nova.api.ec2   File "/opt/stack/nova/nova/api/ec2/__init__.py", line 486, in __call__
2012-12-20 00:03:19 TRACE nova.api.ec2     result = api_request.invoke(context)
2012-12-20 00:03:19 TRACE nova.api.ec2   File "/opt/stack/nova/nova/api/ec2/apirequest.py", line 79, in invoke
2012-12-20 00:03:19 TRACE nova.api.ec2     result = method(context, **args)
2012-12-20 00:03:19 TRACE nova.api.ec2   File "/opt/stack/nova/nova/api/ec2/cloud.py", line 666, in authorize_security_group_ingress
2012-12-20 00:03:19 TRACE nova.api.ec2     rulesvalues = self._rule_args_to_dict(context, values)
2012-12-20 00:03:19 TRACE nova.api.ec2   File "/opt/stack/nova/nova/api/ec2/cloud.py", line 535, in _rule_args_to_dict
2012-12-20 00:03:19 TRACE nova.api.ec2     rule = self._rule_dict_last_step(context, **kwargs)
2012-12-20 00:03:19 TRACE nova.api.ec2 TypeError: _rule_dict_last_step() argument after ** must be a mapping, not str
2012-12-20 00:03:19 TRACE nova.api.ec2 
2012-12-20 00:03:19 ERROR nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] Environment: {"HTTP_REFERER": "http://ec2-50-18-11-126.us-west-1.compute.amazonaws.com:8773/services/Cloud", "CONTENT_TYPE": "application/x-www-form-urlencoded; charset=utf-8", "SCRIPT_NAME": "/services/Cloud", "REQUEST_METHOD": "POST", "HTTP_HOST": "ec2-50-18-11-126.us-west-1.compute.amazonaws.com:8773", "PATH_INFO": "", "SERVER_PROTOCOL": "HTTP/1.0", "HTTP_ACCEPT": "*/*", "HTTP_USER_AGENT": "aws-sdk-php/Panther PHP/5.3.6 Darwin/11.4.0 Arch/x86_64 SAPI/apache2handler Integer/9223372036854775807 Build/20120926163000 simplexml/0.1 json/1.2.1 pcre/8.11 spl/0.2 curl/7.21.3 openssl/0.9.8l apc/3.1.7 memcached/2.0.0-dev pdo/1.0.4dev pdo_sqlite/1.0.1 sqlite/2.0-dev sqlite3/0.7-dev zlib/1.1 xdebug/2.1.0 memory_limit/100M date.timezone/America.Los_Angeles open_basedir/off safe_mode/off zend.enable_gc/on", "RAW_PATH_INFO": "/services/Cloud", "REMOTE_ADDR": "50.143.166.131", "wsgi.url_scheme": "http", "SERVER_NAME": "10.196.16.216", "SERVER_PORT": "8773", "GATEWAY_INTERFACE": "CGI/1.1", "HTTP_ACCEPT_ENCODING": "gzip, deflate"}
2012-12-20 00:03:19 ERROR nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] UnknownError: An unknown error has occurred. Please try your request again.
2012-12-20 00:03:19 INFO nova.api.ec2 [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] 0.447617s 50.143.166.131 POST /services/Cloud CloudController:AuthorizeSecurityGroupIngress 400 [aws-sdk-php/Panther PHP/5.3.6 Darwin/11.4.0 Arch/x86_64 SAPI/apache2handler Integer/9223372036854775807 Build/20120926163000 simplexml/0.1 json/1.2.1 pcre/8.11 spl/0.2 curl/7.21.3 openssl/0.9.8l apc/3.1.7 memcached/2.0.0-dev pdo/1.0.4dev pdo_sqlite/1.0.1 sqlite/2.0-dev sqlite3/0.7-dev zlib/1.1 xdebug/2.1.0 memory_limit/100M date.timezone/America.Los_Angeles open_basedir/off safe_mode/off zend.enable_gc/on] application/x-www-form-urlencoded text/xml
2012-12-20 00:03:19 INFO nova.ec2.wsgi.server [req-8d283b73-9c75-4158-a28c-a0e1c4f44bc5 admin admin] 50.143.166.131 "POST /services/Cloud HTTP/1.1" status: 400 len: 353 time: 0.4487212

I looked through the python code of the nova api and came across this:

 # TODO(soren): This has only been tested with Boto as the client.
    #              Unfortunately, it seems Boto is using an old API
    #              for these operations, so support for newer API versions
    #              is sketchy.
    def authorize_security_group_ingress(self, context, group_name=None,
                                         group_id=None, **kwargs):
        self._validate_group_identifier(group_name, group_id)

        security_group = self.security_group_api.get(context, group_name,
                                                     group_id)

        prevalues = kwargs.get('ip_permissions', [kwargs])
        postvalues = []
        for values in prevalues:
            self._validate_security_group_protocol(values)
            rulesvalues = self._rule_args_to_dict(context, values)
            self._validate_rulevalues(rulesvalues)
            for values_for_rule in rulesvalues:
                values_for_rule['parent_group_id'] = security_group['id']
                if self.security_group_api.rule_exists(security_group,
                                                       values_for_rule):
                    err = _('%s - This rule already exists in group')
                    raise exception.EC2APIError(err % values_for_rule)
                postvalues.append(values_for_rule)

        if postvalues:
            self.security_group_api.add_rules(context, security_group['id'],
                                           security_group['name'], postvalues)
            return True

        raise exception.EC2APIError(_("No rule for the specified parameters."))

It looks like support for the latest PHP SDK might not supported yet. Any thoughts?

Will investigate a bit more.

Also filed a question to Nova Launchpad: https://answers.launchpad.net/nova/+question/217216

There was a problem when Adding & Revoking (Deleting) IPs such as "TCP/UDP", "ICMP", and "Add Group" after creating a Security Group with OpenStack Cloud.

There was also a problem when adding Security Groups, because there was no source to read the IPs & Port Range from. So I added a code to read the IP & Ports from the fields in the "Add TCP/UDP" Form.

Issues fixed in the below patches.

I've tested it and it is working fine with OpenStack Cloud, please perform the testing with EC2.

Thanks for the patch! I'm excited that you are helping me with this issue!

I applied the patch and it works well for adding security rules in OpenStack. But when trying to revoke Security Groups, it would revoke all permission that is associated with the group.

When I tried to use the code for EC2, I got errors because SecurityGroupName and SecurityGroupOwnerId cannot be passed in the IpPermissions array. I found out that within IpPermissions, you can pass GroupName and OwnerId in a nested array() and the Security Group is also added without needing SecurityGroupName and SecurityGroupOwnerId.

Regarding the patch: Sec-Groups-Modifications-1870284.patch, I like the idea! But I also think that you can add ICMP ip addresses as well. I've decided to add the TCP/UDP/ICMP fields directly into the "Add Group" fieldset. The form will use Ajax to reload the fields between TCP/UDP & ICMP. I'm attaching a screenshot for you so you can see what I mean.

Finally, I'm attaching two patches for you to take a look it. I took the basis of your work, enhanced it, cleaned it up a bit with what I described in this issue. Try it out and see if this works for you.

You might want to apply the patches against a clean checkout. There were some changes made to aws_cloud_get_security_action()

Comments and thoughts are appreciated!

Had to reroll the Sec-Groups-Modifications-1870284-6.patch to clean up some commented out code. Here it is again.

Thank you very much and I'm glad that it helped you, but unfortunately Fridays & Saturdays are off here so I won't be able to connect with the Openstack Cloud and test it until Sunday from Work.

But I do like the idea of putting it all together in a single form and the user can just switch between TCP/UDP/ICMP.

I will take a look at the code until I perform the testing on Sunday and update you with the news, and thank you for your support.

I've tested the code and its working fine, thank you very much.

However I had to apply it manually because I keep getting this error message when I try to apply the patch:-

Sec-Groups-Modifications-1870284-7.patch:24: trailing whitespace.

Sec-Groups-Modifications-1870284-7.patch:150: trailing whitespace.

error: patch failed: modules/iaas/modules/aws_cloud/includes/aws_cloud_security_groups_ui.inc:364
error: modules/iaas/modules/aws_cloud/includes/aws_cloud_security_groups_ui.inc: patch does not apply

I tried the following command:-

git apply --reject --whitespace=fix Sec-Groups-Modifications-1870284-7.patch

But it still did not work, anyways ....

I believe everything is working fine from our part or the Openstack Cloud in general, except for the Failover issue.

Merry Christmas and happy new years! I didn't get to commit this last week. But here it is:

http://drupalcode.org/project/cloud.git/commit/28d9cd7