Problems with sys.stout.flush()

[Dave Angel]

Joel Ross wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">Carl 
> Banks wrote:
>> On May 23, 2:20 am, Joel Ross <jo... at cognyx.com> wrote:
>>> Carl Banks wrote:
>>>> On May 22, 10:33 pm, Joel Ross <jo... at cognyx.com> wrote:
>>>>> Hi all,
>>>>> I'm using python 2.5 and trying to flush the sys.stout buffer with
>>>>> sys.stout.flush(), but doesn't seem to work. Each time a line is 
>>>>> printed
>>>>>    it appends the one before it I need to clear the output and 
>>>>> write a
>>>>> new output without appending the previous one.
>>>> That's not how streams work, chief.  Once you output (and flush)
>>>> something you can't un-output it.
>>>> What you probably want to do is to write a carriage return ("\r")
>>>> which usually causes the cursor to return to the beginning of the
>>>> line, so that any new text you write overwrites the old text.
>>>> This has nothing to do with flushing; flushing doesn't erase or clear
>>>> the old input.  Flushing is usually needed for a different reason,
>>>> however, namely standard output doesn't actually get sent to the
>>>> console until it sees a newline ("\n") unless you flush the buffer.
>>>> Try to adapt this example to your problem:
>>>> for i in xrange(11):
>>>>     sys.stdout.write('*'*(10-i) + ' '*i + '\r')
>>>>     sys.stdout.flush()
>>>>     time.sleep(2)
>>> Your example prints a new line each time,
>>
>> Did you run it and observe what happened, or did you just guess what
>> it actually did?  On my system it doesn't print new lines.
>>
>>
>>> Doesn't help me with a
>>> progress bar, do you know of anyway doing it with the print command? My
>>> progress bars works fine this is the only problem im having with it at
>>> the moment. any help would be appreciated.
>>
>> Well, based on the code you posted it doesn't look like your progress
>> bar is fine.
>>
>> Anyway, you haven't described the problem you are having very well,
>> and we can guess what might be wrong but if you want better help
>> you're going to have to describe your problem more coherently.  Give
>> us the
>>
>> 1. What code are you actually running (cut-and-paste, please, don't
>> retype).
>> 2. What do you expect the code to output (type in actual output you
>> expect to see, not merely a description).
>> 3. What does the code actually output when you run it (cut-and paste,
>> please, don't just describe).
>> 4. Include any tracebacks if there are any.
>>
>> In the meantime, I repeat my suggestion that you take my example and
>> adapt it to what you are doing.
>>
>> One other note: the print statement is unsuitable for this task
>> because it always prints a trailing whitespace.  Use sys.stdout.write
>> for it.
>>
>>
>> Carl Banks
> Hey,
>
> The only reason I didn't supply full details is because I thought it 
> may just be something simple I was missing. Sorry for the confusing. 
> Well here's the full code.
>
> import sys, os
> from time import sleep
>
>
> class progress:
>
>     def progressbar(self, number, total,  char):
>
>         percentage = float(number*100)/total
>         percentage = int(round(percentage))
>         percentage = int(100 - percentage)
>         self.f=sys.stdout
>         if percentage > 0:
>             char = char * percentage
>             self.f.write(char)
>             self.f.flush()         
>             sleep(0.2)
>
>
>     def countlines(self, file):
>
>         lineCount = 0
>         f = open(file)
>         it = iter(f)
>         try:
>             while it.next():
>                 lineCount += 1
>         except StopIteration:
>                     pass
>         return lineCount
>
>
> def main():
>
>     p = progress()
>     lines = 5
>     #lines = p.countlines("/tmp/junk")
>     count = lines
>     for i in xrange(lines):
>         p.progressbar(count, lines, "*")
>         count -=1
>     print "Finished"
>
> if __name__ == "__main__":
>         main()
>
> For now we can ignore the countlines() function. I need a nice 
> progress display e.g. ********************** continuing on til it 
> reaches 100% I need this to print out on one line only. At the moment 
> it does do this, only problem is its appends the print out from the 
> last print out. I need it to clear the last print out and then print 
> the new print out without appending the last.
>
> So when it prints a progress at 50% it will print 50 (*) characters 
> and if the next progress is 51% it will print 51 (*) characters 
> including the last 50 (*) characters, so instead on ending up with 100 
> (*) characters I end up with a shit load of them depending on how many 
> lines I pass to the progressbar() function.
>
> Needed Output:
> ***************************************************************************************************Finished 
>
>
>
> Current Output:
> ********************************************************************************************************************************************************************************************************Finished 
>
>
> Note: As you can see I'm only sending 5 lines to the progressbar() 
> function and It prints this many characters, so you could imagine if I 
> have 10000 lines I would end up with a heap of * characters.
>
> Regards
> jross
>
>
You don't tell your environment.  Like, what version of Python, and more 
importantly what operating system?  You also don't say how you're 
testing the system, nor where it's intended to finally run.

There is no portable way to say "erase those charcters I sent earlier, I 
didn't mean it."  flush() does nothing even close to that.

In a standard console (at least on Windows), adding a simple \r   (*NOT* 
\n) would work easily, as Carl Banks already suggested.  However, in 
some shells (I test with Komodo), it does not.

So, you have a couple of choices, depending on your environment.  The 
simplest is to do a cursor-position operation, specific to your 
particular output "device."  Nobody can reliably help you with that 
without knowing what the environment is.

More complex for a forward-moving progress bar is to simply keep track 
of how many stars have already been output, and only add the additional 
ones you want.  in your current incarnation, the percentage value shows 
20, then 40, then 60, then 80  (it never reaches 100, but maybe that's 
okay).  Currently you're printing all those stars over again.  This 
solution would be to print only the difference, or 20 more stars each time.

A way to accomplish that would be to create a counter in the progress 
class, that knows how many stars have already been output.  Something like:

class progress:

    def __init__(self):
        self.already = 0              #number of stars already displayed

    def progressbar(self, number, total,  char):

        percentage = float(number*100)/total
        percentage = int(round(percentage))
        percentage = int(100 - percentage)
        self.f=sys.stdout
        if percentage > 0:
            char = char * (percentage-self.already)
            self.already = percentage
            self.f.write(char + "X")    #note:  remove the "X" when 
you're sure it's working
            self.f.flush()
            sleep(0.2)

Clearly the self.write() function should change to simply    
self.write(char)  when you're confident it's working.  For now, it gives 
you a marker to see how it's being pieced together.

Incidentally, Carl's other suggestion, that you use a new variable 
instead of overwriting 'char', is a red herring.  It's good advice to 
never change your parameters in place, and for mutable data types, could 
be a problem.  But because char is a string, this is not your problem.