Frustrating circular bytes issue

[Robert Kern]

On 6/26/12 5:30 PM, J wrote:
> This is driving me batty... more enjoyment with the Python3
> "Everything must be bytes" thing... sigh...
> I have a file that contains a class used by other scripts.  The class
> is fed either a file, or a stream of output from another command, then
> interprets that output and returns a set that the main program can
> use...  confusing, perhaps, but not necessarily important.
>
> The class is created and then called with the load_filename method:
>
>
>   def load_filename(self, filename):
>          logging.info("Loading elements from filename: %s", filename)
>
>          file = open(filename, "rb", encoding="utf-8")
>          return self.load_file(file, filename)

I get this with Python 3.2:

Traceback (most recent call last):
   File "bytes_unicode.py", line 32, in <module>
     d.load_filename(__file__)
   File "bytes_unicode.py", line 6, in load_filename
     file = open(filename, "rb", encoding="utf-8")
ValueError: binary mode doesn't take an encoding argument

Are you sure you are copy-pasting the code that is actually running? Can you 
reduce your test case down to a small self-contained example that runs and 
demonstrates the problem? I suspect that there is some other detail that is 
causing things to fail. The following code works fine for me:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

import re

class Dummy(object):
     """ This is a dummy file.

     éµ∫é∂∂é∂ üñîçø∂é
     """

     def load_filename(self, filename):
         file = open(filename, "r", encoding="utf-8")
         return self.load_file(file, filename)

     def load_file(self, file, filename="<stream>"):
         for string in self._reader(file):
             print(string)
             if not string:
                 break

     def _reader(self, file, size=4096, delimiter=r"\n{2,}"):
         buffer_old = ""
         while True:
             buffer_new = file.read()
             print(type(buffer_new))
             if not buffer_new:
                 break
             lines = re.split(delimiter, buffer_old + buffer_new)
             buffer_old = lines.pop(-1)

             for line in lines:
                 yield line

         yield buffer_old

if __name__ == '__main__':
     d = Dummy()
     d.load_filename(__file__)


-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco