ValueError: I/O operation on closed file. with python3

[Peter Otten]

Adam Mercer wrote:

> Hi
> 
> I'm trying to update one of my scripts so that it runs under python2
> and python3, but I'm running into an issue that the following example
> illustrates:
> 
> $ cat test.py
> try:
>   # python-2.x
>   from urllib2 import urlopen
>   from ConfigParser import ConfigParser
> except ImportError:
>   # python-3.x
>   from urllib.request import urlopen
>   from configparser import ConfigParser
> 
> server='http://www.lsc-group.phys.uwm.edu/~ram/files'
> 
> fp = urlopen('%s/latest.ini' % server).fp
> cp = ConfigParser()
> cp.readfp(fp)
> print(cp.get('version', '10.8'))
> $
> 
> This works as expected when using python2:
> 
> $ python2.7 test.py
> 5.2.10
> $
> 
> but when using python3 I receive the following error:
> 
> $ python3.3 test.py
> Traceback (most recent call last):
>   File "test.py", line 14, in <module>
>     cp.readfp(fp)
>   File
>   
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
> line 753, in readfp
>     self.read_file(fp, source=filename)
>   File
>   
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
> line 708, in read_file
>     self._read(f, source)
>   File
>   
"/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py",
> line 1010, in _read
>     for lineno, line in enumerate(fp, start=1):
> ValueError: I/O operation on closed file.
> $
> 
> Is there a way to get this working in both python2 and python3?
> 
> This is a small script and I'm starting to have some users wanting to
> use python3 and others sticking to python2 so I'd like to accommodate
> them both if possible.

Python 3.3.0 (v3.3.0:bd8afb90ebf2, Sep 30 2012, 14:49:00) 
[GCC 4.6.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.request import urlopen
>>> url = "http://www.lsc-group.phys.uwm.edu/~ram/files/latest.ini"
>>> fp = urlopen(url).fp
>>> fp.read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: read of closed file
>>> resp = urlopen(url)
>>> resp.fp.read()
b'[version]\n10.5 = 5.2.4\n10.6 = 5.2.10\n10.7 = 5.2.10\n10.8 = 5.2.10\n'

I don't know whether this behaviour is intentional or accidental (I assume 
the latter).

I then ran into another problem:

>>> from configparser import ConfigParser
>>> p = ConfigParser()
>>> resp = urlopen(url)
>>> p.readfp(resp.fp)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/configparser.py", line 754, in readfp
    self.read_file(fp, source=filename)
  File "/usr/local/lib/python3.3/configparser.py", line 709, in read_file
    self._read(f, source)
  File "/usr/local/lib/python3.3/configparser.py", line 1011, in _read
    if line.strip().startswith(prefix):
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

So ConfigParser.readfp() (which has been deprecated in favour of 
read_file(), btw.) expects a text file:

>>> from io import TextIOWrapper
>>> resp = urlopen(url)
>>> p.readfp(TextIOWrapper(resp.fp))
>>> p.get("version", "10.8")
'5.2.10'

Applying these findings to your script:

from contextlib import contextmanager
try:
  # python-2.x
  from urllib2 import urlopen
  from ConfigParser import ConfigParser

  @contextmanager
  def my_urlopen(url):
      yield urlopen(url).fp

except ImportError:
  # python-3.x
  from urllib.request import urlopen
  from configparser import ConfigParser
  import io

  @contextmanager
  def my_urlopen(url):
      resp = urlopen(url)
      yield io.TextIOWrapper(resp.fp)

server='http://www.lsc-group.phys.uwm.edu/~ram/files'

cp = ConfigParser()
with my_urlopen('%s/latest.ini' % server) as fp:
    cp.readfp(fp)

print(cp.get('version', '10.8'))

I've run it with 2.6, 2.7, 3.2, and 3.3.