[Tutor] reading output from a c executable.

[Ravi Kondamuru]

Reasons for C:
1. The log files I am working are 60-100MB files. I *assumed* using C will
reduce the parse time.
2. The records themselves are variable length and hence was concerned about
the complexity for implementation in python.
3. Since I am not using a database, each request to refresh the graph
(changing time duration for display) will have to re-parse the file all over
again. And hence speed in parsing the log is important.

Hope that explains
thanks,
Ravi.

On Thu, Dec 11, 2008 at 10:45 AM, Steve Willoughby <steve at alchemy.com>wrote:

> Ravi Kondamuru wrote:
>
>> I am trying to read a binary log file to extract system counters. These
>> counters will then be used to generate web-based graphs using the
>> chart-director api in python. For extracting the counters I planning to
>> write a program in C to read and give the output as lists for use by
>> chart-director. If possible i would like to do the parsing of data only once
>> in C on the log file and pass the processed output for direct use by python.
>>
>>
>
> Wait..
> Why not have Python read the binary log file itself, parse it and make the
> API calls right there?  Why do you even need the C program?
>
>
>
>> I have thought about having to populate a database first but will prefer
>> to avoid having another intermediate datastore (apart from the log file).
>>
>> thanks,
>> Ravi.
>>
>> On Thu, Dec 11, 2008 at 10:27 AM, Steve Willoughby <steve at alchemy.com<mailto:
>> steve at alchemy.com>> wrote:
>>
>>    Ravi Kondamuru wrote:
>>
>>        Hi,
>>        I am writing a script to read list output from a C executable.
>>        How should c program be written so that python can read the
>>        output as a list?
>>        Any pointers to info on this appreciated.
>>
>>
>>    The possibilities are truly wide open on this.  Python can read a
>>    variety of standard formats (and of course can have custom code to
>>    read anything.  Depending on what the data involved actually are,
>>    you need to decide what format works best.
>>
>>    A simple approach is to have the C program write simple CSV output,
>>    and use Python's csv module to read it.  A more complex solution
>>    might be to use XML.  There are approximately 52,495,102 other
>>    possibilities available too, so you have lots of room to work out
>>    what's best for your  application.  More specific information on
>>    what you're trying to accomplish would help narrow it down as well.
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20081211/fe311191/attachment-0001.htm>