Finding size of Variable

[Ayushi Dalmia]

On Tuesday, February 4, 2014 6:39:00 PM UTC+5:30, Dave Angel wrote:
> Ayushi Dalmia <ayushidalmia2604 at gmail.com> Wrote in message:
> 
>  
> 
> >> getsizeof() gives you the size of the list only; to complete the picture you 
> 
> >> 
> 
> >> have to add the sizes of the lines.
> 
> >> 
> 
> >> 
> 
> >> 
> 
> >> However, why do you want to keep track of the actual memory used by 
> 
> >> 
> 
> >> variables in your script? You should instead concentrate on the algorithm, 
> 
> >> 
> 
> >> and as long as either the size of the dataset is manageable or you can limit 
> 
> >> 
> 
> >> the amount of data accessed at a given time you are golden.
> 
> > 
> 
> > As I said, I need to merge large files and I cannot afford more I/O operations. So in order to minimise the I/O operation I am writing in chunks. Also, I need to use the merged files as indexes later which should be loaded in the memory for fast access. Hence the concern.
> 
> > 
> 
> > Can you please elaborate on the point of taking lines into consideration?
> 
> > 
> 
> 
> 
> Please don't doublespace your quotes.  If you must use
> 
>  googlegroups,  fix its bugs before posting. 
> 
> 
> 
> There's usually no net gain in trying to 'chunk' your output to a
> 
>  text file. The python file system already knows how to do that
> 
>  for a sequential file.
> 
> 
> 
> For list of strings just add the getsizeof for the list to the sum
> 
>  of the getsizeof of all the list items. 
> 
> 
> 
> -- 
> 
> DaveA

Hey! 

I need to chunk out the outputs otherwise it will give Memory Error. I need to do some postprocessing on the data read from the file too. If I donot stop before memory error, I won't be able to perform any more operations on it.