[Python-Dev] Decoding incomplete unicode

[M.-A. Lemburg]

Walter Dörwald wrote:
> Martin v. Löwis wrote:
> 
>> Walter Dörwald wrote:
>>
>>> They will not, because StreamReader.decode() already is a feed
>>> style API (but with state amnesia).
>>>
>>> Any stream decoder that I can think of can be (and most are)
>>> implemented by overwriting decode().
>>
>>
>> I consider that an unfortunate implementation artefact. You
>> either use the stateless encode/decode that you get from
>> codecs.get(encoder/decoder) or you use the file API on
>> the streams. You never ever use encode/decode on streams.
> 
> 
> That is exactly the problem with the current API.
> StreamReader mixes two concepts:
> 
> 1) The stateful API, which allows decoding a byte input
>    in chunk and the state of the decoder is kept between
>    calls.
> 2) A file API where the chunks to be decoded are read
>    from a byte stream.
> 
>> I would have preferred if the default .write implementation
>> would have called self._internal_encode, and the Writer
>> would *contain* a Codec, rather than inheriting from Codec.
> 
> This would separate the two concepts from above.

Note that StreamCodec only inherits from Codec for
convenience reasons (you can define a StreamCodec
using the stateless .encode() and .decode() methods
you get from Codec) and for logical reasons: a StreamCodec
happens to be a  Codec as well, so isinstance(obj, Codec) should
be true for a StreamCodec as well.

There's nothing preventing you from overriding .encode()
and .decode() in a StreamReader or adding new methods that
implement a different approach to encode and decode.

Users should always use the file API of StreamReader
et al., not the .encode() and .decode() methods.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Aug 19 2004)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::