[Tutor] Compute data usage from log

[Eduardo Vieira]

On Tue, Oct 27, 2009 at 5:33 AM, bibi midi <bibsmendez at gmail.com> wrote:
> #!/usr/bin/env python
> # -*- coding: utf-8 -*-
>
> '''
> Calculate internet data consumption
> of service provider
> Code as per help of python tutor mailing list
> Created: 26-Oct-2009
>
> '''
>
> intro = raw_input('press enter to view your internet data consumption: ')
> log_file = '/home/bboymen/mobily.data.plan'
> total_consumed = 0
> for line in open(log_file):
>     total_consumed += int(line.split(' ')[9])
>
>
> total_consumed = total_consumed/1000000.0
> print "total data consumed is: %0.3f MB\n" % total_consumed
>
>
> #TODO:
> #1) show the list comprehension alternative method
> #2) add exception handling e.g. when log_file cant be found or when key
> interrupt is pressed
>
> #      or when index[9] is not a number, etc
> #3) print latest date of calculated data
>
> I'm working on TODO no. 3 e.g. I want to show the latest date when wvdial
> generated the ppp data. This is normally the date of last line of the ppd:
>
> Oct 14 11:03:45 cc000002695 pppd[3092]: Sent 3489538 bytes, received
> 43317854 bytes.
> ^^^^^^^^^
>
> For the exception handling i *think* i just use the general exception method
> e.g. will catch all kinds of error. I really dont know what other errors
> will show up aside from the ones i listed in the TODO. Advise is
> appreciated.
>
>
>
>
>
> On Mon, Oct 26, 2009 at 2:12 PM, Luke Paireepinart <rabidpoobear at gmail.com>
> wrote:
>>
>>
>> On Mon, Oct 26, 2009 at 3:20 AM, Christian Witts <cwitts at compuscan.co.za>
>> wrote:
>>>
>>> fInput = open('/path/to/log.file', 'rb')
>>> total_usage = 0
>>> for line in fInput:
>>>   total_usage += int(line.split(' ')[9].strip())
>>> print total_usage
>>
>> It's actually bad to assign a variable to the file object in this case
>> (flinput = ....) because Python will automatically close a file after you're
>> done with it if you iterate over it directly, but if you include a reference
>> it will stay open until the python program ends or you explicitly call
>> flinput.close().  It doesn't matter much in this example but in general it
>> is good practice to either
>> 1) call foo.close() immediately after you're done using a file object, or
>> 2) don't alias the file object and just over it directly so Python will
>> auto-close it.
>>
>> Therefore a better (and simpler) way to do the above would be:
>>
>> total_usage = 0
>> for line in open('/path/to/log.file'):
>>     total_usage += int(line.split(' ')[9])
>>
>> Also note you don't need to strip the input because int() coersion ignores
>> whitespace anyway. And additionally you shouldn't be opening this in binary
>> mode unless you're sure you want to, and I'm guessing the log file is ascii
>> so there's no need for the 'rb'.  (reading is default so we don't specify an
>> 'r'.)
>>
>>
>> And since I like list comprehensions a lot, I'd probably do it like this
>> instead:
>>
>> total_usage = sum([int(line.split(' ')[9]) for line in
>> open('/path/to/log.file')])
>>
>> Which incidentally is even shorter, but may be less readable if you don't
>> use list comprehensions often.
>>
>> Also, the list comprehension version is likely to be more efficient, both
>> because of the use of sum rather than repeated addition (sum is implemented
>> in C) and because list comprehensions in general are a tad faster than
>> explicit iteration, if i recall correctly (don't hold me to that though, I
>> may be wrong.)
>>>
>>> Of course this has no error checking and or niceties, but I will leave
>>> that up to you.
>>
>> The same applies to my modifications.
>>
>> Good luck, and let us know if you need anything else!
>>
>> -Luke
>
>
>
> --
> Best Regards,
> bibimidi
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
>
>

I think you could benefit a lot from the code examples given by Dave
Beazley in his presentation about generators. There are scripts that
deal with just about exactly what you're looking for. Very informative
material: http://www.dabeaz.com/generators/
See the presentation slides

HTH

Eduardo