[Python-3000] iostack and sock2

[Josiah Carlson]

"tomer filiba" <tomerfiliba at gmail.com> wrote:
[snip]
> > - interaction with (replacement of?) the select module
> 
> well, it's too hard to design for a nonexisting module. select is all there
> is that's platform independent.

It is /relatively/ platform independent.

> random idea:
> * select is virtually platform independent
> * improved polling is inconsistent
>     * kqueue is BSD-only
>     * epoll is linux-only
>     * windows has none of those

Windows doesn't currently have a module designed to do this kind of
thing, but it is possible to have a higher-performance method for
Windows using various bits from the win32file module from pywin32 (I
have been contemplating writing one, but I haven't had the time).

[snip]

> - - - - -
> 
> > e.g an alternative approach would be to
> > define InputStream and OutputStream, and then have an IOStream that inherited
> > from both of them).
> 
> hrrm... i need to think about this more. one problem i already see:
> 
> class InputStream:
>    def close(self):....
>    def read(self, count): ...
> 
> class OutputStream:
>    def close(self):....
>    def write(self, data)...
> 
> class NetworkStream(InputStream, OutputStream):
>    ...
> 
> which version of close() gets called?

Both, you use super().

> - - - - -
> 
> > e.g. the 'position' property is
> > probably a bad idea, because x.position may then raise an IOError
> 
> i guess it's reasonable approach, but i'm a "usability beats purity" guy.
> f.position = 0
> or
> f.position += 10
> 
> is so much more convenient than seek()ing and tell()ing. we can also
> optimize += by defining a Position type where __iadd__(n) uses
> seek(n, "curr") instead of seek(n + tell(), "start")
> 
> btw, you can first test the "seakable" attribute, to see if positioning
> would work.
> 
> and in the worst case, i'd vote for converting IOErrors to ValueErrors...
> 
> def _set_pos(self, n)
>     try:
>        self.seek(n)
>     except IOError:
>        raise ValueError("invalid position value", n)
> 
> so that
> f.position = -10
> raises a ValueError, which is logical

Raising a ValueError on an unseekable stream would be confusing.

[snip]
> - - - - -
> 
> random idea:
> when compiled with universal line support, python unicode should
> equate "\n" to any of the forementioned characters.
> i.e.
> 
> u"\n" == u"\u2028" # True

I'm glad that you later decided for yourself that such a thing would be
utterly and completely foolish.

> - - - - -
> 
> > I can see that behaviour being seriously annoying when you get to the end of
> > the stream. I'd far prefer for the stream to just give me the last bit when I
> > ask for it and then tell me *next* time that there isn't anything left.
> 
> well, today it's done like so:
> 
> while True:
>    x = f.read(100)
>    if not x:
>       break
> 
> in iostack, that would be done like so:
> 
> try:
>     while True:
>         x = f.read(100)
> except EOFError:
>     last_x = f.readall() # read all the leftovers (0 <= leftovers < 100)
> 
> a little longer, but not illogical
> 
> > If you want a method with the other behaviour, add a "readexact" API, rather
> > than changing the semantics of "read" (although I'd be really curious to hear
> > the use case for the other behaviour).
> 
> well, when i work with files/sockets, i tend to send data structures over them,
> like records, frames, protocols, etc. if a record is said to be x bytes long,
> and read(x) returns less than x bytes, my code has to loop until it gets
> enough bytes.

Rather than changing what people expect with the current .read() method,
why not offer a different method called .readexact(n), which will read
exactly n bytes, performing buffering as necessary.  You can then
optimize by using cStringIOs, lists of strings, resizable bytes, or
whatever other method you want (but be careful never to .read(bignum)
unless you change the underlying .read() implementation; right now it
allocates a buffer of size bignum, which can cause huge amounts of
malloc/realloc thrashing, and generally causes MemoryErrors).

[snip]

 - Josiah