[Tutor] reading output from a c executable.

[Ravi Kondamuru]

This was a great info exchange for me.
I am going to try out the all-python solution and see how it works out.
thank you all,
Ravi.

On Thu, Dec 11, 2008 at 10:59 AM, Ravi Kondamuru <ravikondamuru at gmail.com>wrote:

> Reasons for C:
> 1. The log files I am working are 60-100MB files. I *assumed* using C will
> reduce the parse time.
> 2. The records themselves are variable length and hence was concerned about
> the complexity for implementation in python.
> 3. Since I am not using a database, each request to refresh the graph
> (changing time duration for display) will have to re-parse the file all over
> again. And hence speed in parsing the log is important.
>
> Hope that explains
> thanks,
> Ravi.
>
>
> On Thu, Dec 11, 2008 at 10:45 AM, Steve Willoughby <steve at alchemy.com>wrote:
>
>> Ravi Kondamuru wrote:
>>
>>> I am trying to read a binary log file to extract system counters. These
>>> counters will then be used to generate web-based graphs using the
>>> chart-director api in python. For extracting the counters I planning to
>>> write a program in C to read and give the output as lists for use by
>>> chart-director. If possible i would like to do the parsing of data only once
>>> in C on the log file and pass the processed output for direct use by python.
>>>
>>>
>>
>> Wait..
>> Why not have Python read the binary log file itself, parse it and make the
>> API calls right there?  Why do you even need the C program?
>>
>>
>>
>>> I have thought about having to populate a database first but will prefer
>>> to avoid having another intermediate datastore (apart from the log file).
>>>
>>> thanks,
>>> Ravi.
>>>
>>> On Thu, Dec 11, 2008 at 10:27 AM, Steve Willoughby <steve at alchemy.com<mailto:
>>> steve at alchemy.com>> wrote:
>>>
>>>    Ravi Kondamuru wrote:
>>>
>>>        Hi,
>>>        I am writing a script to read list output from a C executable.
>>>        How should c program be written so that python can read the
>>>        output as a list?
>>>        Any pointers to info on this appreciated.
>>>
>>>
>>>    The possibilities are truly wide open on this.  Python can read a
>>>    variety of standard formats (and of course can have custom code to
>>>    read anything.  Depending on what the data involved actually are,
>>>    you need to decide what format works best.
>>>
>>>    A simple approach is to have the C program write simple CSV output,
>>>    and use Python's csv module to read it.  A more complex solution
>>>    might be to use XML.  There are approximately 52,495,102 other
>>>    possibilities available too, so you have lots of room to work out
>>>    what's best for your  application.  More specific information on
>>>    what you're trying to accomplish would help narrow it down as well.
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20081211/e5855d6d/attachment-0001.htm>