convert output to list(and nested dictionary)

Wed Jul 22 12:01:35 EDT 2015

Hi Pablo,

While playing around with the function you gave me(get_data)...i was
thinking to do something like below.  For each line create a dictionary
then append that dictionary to a list...but before i even get to that part
i get the below error and while researching it i am unable to figure out
what is going on... could you point me in the right direction ?

>>> for sg in sgs:
    for rule in sg.rules:
        pt = sg, sg.id, "inbound:", rule, " source:", rule.grants
        print pt #this is just for my own purpose
        get_data(pt)

(SecurityGroup:wordpress-app-SG, u'sg-99c4befc', 'inbound:',
IPPermissions:-1(None-None), ' source:', [sg-e632d982-995635159130])

Traceback (most recent call last):
  File "<pyshell#157>", line 5, in <module>
    get_data(pt)
  File "<pyshell#152>", line 5, in get_data
    _, proto_port, cidr = data.rsplit(":", 2)
AttributeError: 'tuple' object has no attribute 'rsplit'
>>> get_data("SecurityGroup:wordpress-app-SG, u'sg-99c4befc', 'inbound:',
IPPermissions:-1(None-None), ' source:', [sg-e632d982-995635159130]")
{'cidr': 'sg-e632d982-995635159130', 'port': 'None', 'proto': '1'}
>>>

On Tue, Jul 21, 2015 at 8:03 PM, Pablo Lucena <plucena24 at gmail.com> wrote:

> str.split and re are a nice quick way to do it:
>
> >>> def get_data(data):
> import re
> port_re = re.compile(r'(\w+)\((\S+-\S+)\)')
> cidr_re = re.compile(r'\[(.*?)\]')
> _, proto_port, cidr = data.rsplit(":", 2)
> port_match = port_re.search(proto_port)
> proto, port = port_match.group(1), port_match.group(2)
> port = port.split("-")[0]
> cidr_match = cidr_re.search(cidr)
> cidr = cidr_match.group(1)
> return dict(port=port, proto=proto, cidr=cidr)
>
> >>> get_data("SecurityGroup:default sg-e1304484 inbound:
> IPPermissions:tcp(80-80)  source: [67.184.225.222/32]")
> {'cidr': '67.184.225.222/32', 'proto': 'tcp', 'port': '80'}
> >>> get_data("SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
> IPPermissions:-1(None-None)  source: [sg-e632d982-995635159130]")
> {'cidr': 'sg-e632d982-995635159130', 'proto': '1', 'port': 'None'}
>
>
> You can alter this and add whatever extra checks you need as Chris A
> mentioned (when proto is -1 and port is None-None, or the icmp case). This
> is just a very crude example, but hopefully you get the drift.
>
> Most text parsing problems can easily be solved with these simple tools.
> Fire up your shell and test it - this is really the best way to learn how
> to do something like this.
>
>
> On Tue, Jul 21, 2015 at 5:12 PM, max scalf <oracle.blog3 at gmail.com> wrote:
>
>> Hello all,
>>
>> For Each SecurityGroup, how can i convert that into a List that in turn
>> will have a dictionary of the cidr block, protocol type and the port...so
>> from output below, the SecurityGroup called "default" had 2
>> rules...allowing TCP port from 80 and 5500 to the source IP and then
>> SecurityGroup called "Pub_HDP_SG" had only one rule...so on and so
>> forth....here is the output that i am trying to get out in the form of a
>> list....
>>
>> what I am planning to do is, take the list(and nested dictionary) and
>> pass that to a function that will in turn spitout a cloudformation template
>> using troposphere (something like "
>> http://imil.net/wp/2015/06/04/rock-your-cloudformation-with-troposphere-and-boto/
>> ")
>>
>>
>> For Better Readablity (http://pastebin.com/rT6Aswwz)
>>
>> import boto.ec2
>>
>> sgs = boto.ec2.connect_to_region('us-east-1').get_all_security_groups()
>>
>> for sg in sgs:
>>
>>     for rule in sg.rules:
>>
>>         print sg, sg.id, "inbound:", rule, " source:", rule.grants
>>
>>
>> SecurityGroup:default sg-e1304484 inbound: IPPermissions:tcp(80-80)
>>  source: [67.184.225.222/32]
>>
>> SecurityGroup:default sg-e1304484 inbound: IPPermissions:tcp(5500-5500)
>>  source: [67.184.225.222/32]
>>
>> SecurityGroup:Pub_HDP_SG sg-e632d982 inbound: IPPermissions:tcp(80-80)
>>  source: [0.0.0.0/0]
>>
>> SecurityGroup:sg3-MySecurityGroup-LB0QF9UQAOEF sg-4fe73728 inbound:
>> IPPermissions:tcp(22-22)  source: [0.0.0.0/0]
>>
>> SecurityGroup:sg3-MySecurityGroup-LB0QF9UQAOEF sg-4fe73728 inbound:
>> IPPermissions:tcp(80-80)  source: [0.0.0.0/0]
>>
>> SecurityGroup:RDP Rule - open everyone  sg-42d58d27 inbound:
>> IPPermissions:-1(None-None)  source: [0.0.0.0/0]
>>
>> SecurityGroup:us-east-open-all sg-97ffa7f2 inbound:
>> IPPermissions:tcp(22-22)  source: [10.0.20.100/32]
>>
>> SecurityGroup:us-east-open-all sg-97ffa7f2 inbound:
>> IPPermissions:tcp(53-53)  source: [10.0.20.100/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:-1(None-None)  source: [sg-e632d982-995635159130]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:tcp(22-22)  source: [67.184.225.222/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:tcp(1024-65535)  source: [10.0.20.100/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:tcp(80-80)  source: [24.12.30.198/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:udp(138-138)  source: [10.0.20.100/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:udp(53-53)  source: [24.12.30.198/32]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:tcp(30015-30015)  source: [0.0.0.0/0]
>>
>> SecurityGroup:wordpress-app-SG sg-99c4befc inbound:
>> IPPermissions:icmp(-1--1)  source: [10.0.20.100/32]
>>
>> SecurityGroup:default sg-c65a20a3 inbound: IPPermissions:-1(None-None)
>>  source: [sg-c65a20a3-995635159130]
>>
>> SecurityGroup:default sg-c65a20a3 inbound: IPPermissions:-1(None-None)
>>  source: [sg-99c4befc-995635159130]
>>
>> SecurityGroup:sg3-MySecurityGroup2-1HGPN4UF57XN6 sg-4ee73729 inbound:
>> IPPermissions:tcp(22-22)  source: [192.168.1.12/32]
>>
>> SecurityGroup:AWS-AMI-SG sg-35568d51 inbound: IPPermissions:tcp(22-22)
>>  source: [0.0.0.0/0]
>>
>> SecurityGroup:launch-wizard-2 sg-932255f6 inbound:
>> IPPermissions:tcp(22-22)  source: [10.0.20.100/32]
>>
>> SecurityGroup:launch-wizard-2 sg-932255f6 inbound:
>> IPPermissions:tcp(443-443)  source: [0.0.0.0/0]
>>
>> >>>
>>
>>
>> Here is the output i am looking for....
>>
>>
>> rule1 = [{
>>
>>     'cidr': '67.184.225.222/32',
>>
>>     'proto': 'tcp',
>>
>>     'port': 80
>>
>> },{
>>
>>     'cidr': '67.184.225.222/32',
>>
>>     'proto': 'tcp',
>>
>>     'port': 5500
>>
>> }]
>>
>>
>> rule2 = [{
>>
>>     'cidr': '[0.0.0.0/0',
>>
>>     'proto': 'tcp',
>>
>>     'port': 80
>>
>> }]
>>
>>
>> rule3 = [{
>>
>>     'cidr': '0.0.0.0/0',
>>
>>     'proto': 'tcp',
>>
>>     'port': 22
>>
>> },{
>>
>>     'cidr': '0.0.0.0/0',
>>
>>     'proto': 'tcp',
>>
>>     'port': 80
>>
>> }]
>>
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>
>
> --
> *Pablo Lucena*
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20150722/723ce93a/attachment.html>