Consolidate several lines of a CSV file with firewall rules

[Joel Goldstick]

On Fri, Oct 11, 2013 at 11:01 AM, Starriol <juanscopp at gmail.com> wrote:
> Hi guys.
> I have a CSV file, which I created using an HTML export from a Check Point firewall policy.
> Each rule is represented as several lines, in some cases. That occurs when a rule has several address sources, destinations or services.
> I need the output to have each rule described in only one line.
> It's easy to distinguish when each rule begins. In the first column, there's the rule ID, which is a number.
>
> Let me show you an example:
>
> NO.;NAME;SOURCE;DESTINATION;VPN  ;SERVICE;ACTION;TRACK;INSTALL ON;TIME;COMMENT
> 1;;fwxcluster;mcast_vrrp;;vrrp;accept;Log;fwxcluster;Any;"VRRP;;*Comment suppressed*
> ;;;;;igmp**;;;;;
> 2;;fwxcluster;fwxcluster;;FireWall;accept;Log;fwxcluster;Any;"Management FWg;*Comment suppressed*
> ;;fwmgmpe**;fwmgmpe**;;ssh**;;;;;
> ;;fwmgm**;fwmgm**;;;;;;;
> 3;NTP;G_NTP_Clients;cmm_ntpserver_pe01;;ntp;accept;None;fwxcluster;Any;*Comment suppressed*
> ;;;cmm_ntpserver_pe02**;;;;;;;
>
> What I need ,explained in pseudo code, is this:
>
> Read the first column of the next line. If there's a number:
>         Evaluate the first column of the next line. If there's no number there, concatenate (separating with a comma) \
>         the strings in the columns of this line with the last one and eliminate the text in the current one
>
> The output should be something like this:
>
> NO.;NAME;SOURCE;DESTINATION;VPN  ;SERVICE;ACTION;TRACK;INSTALL ON;TIME;COMMENT
> 1;;fwxcluster,fwmgmpe**,fwmgm**;mcast_vrrp,fwmgmpe**,fwmgm**;;vrrp,ssh**;accept;Log;fwxcluster;Any;*Comment suppressed*
> ;;;;;;;;;;
> ;;;;;;;;;;
> 3;NTP;G_NTP_Clients;cmm_ntpserver_pe01,cmm_ntpserver_pe02**;;ntp;accept;None;fwxcluster;Any;*Comment suppressed*
> ;;;;;;;;;;
>
> The empty lines are there only to be more clear, I don't actually need them.
>
> Thanks!
> --
> https://mail.python.org/mailman/listinfo/python-list

I think you posted twice, and perhaps in html?  Its hard to read.

At any rate, there is a csv module in python that will let you gather
your data in a list of lists.  With that you can iterate through the
csv rows, saving rows with a number in the first position.  Iterate
and append the rows below that until you run into another row with a
number in the first position.

Why don't you write some code, see how it goes, copy and paste the
code back here with full traceback if you get an error or with your
results if you have some.  Do it for a subset of a couple of rows of
input data.

-- 
Joel Goldstick
http://joelgoldstick.com