scripting language newbie - compatibility

[Alex Martelli]

"Paul Duffin" <pduffin at hursley.ibm.com> wrote in message
news:398EA451.CBFB72 at hursley.ibm.com...
    [snip]
> I had a feeling that the Windows version of Python while being part of the
> core was supported seperately in some way.

No such thing, that I know of.  I've just downloaded the latest beta of
Python's new version (1.6b1), and the source tarball only comes in one
flavour; I've built it under Windows, but it uses just the same sources as
all the rest, and I've posted the bug reports about the few minor glitches
I've found (in a couple of the tests, actually, not in the Python modules
they were testing; I've never seen such a _solid_ "beta 1" in 25 years of
involvement with software!-) to just the same sourceforge group where
I would have posted bug-reports about any other platform.

There are *extensions*, for access to functionality that only exists in
Windows (the Win32 API, the MFC, COM, ...), that are released and
supported separately.  But the part of Python that is used to run any
cross-platform program, "the core", is supported via sourceforge in
just one way, AFAIK.


> Actually you can generate byte compiled versions of Tcl which you can
> distribute but you need to use a commercial package called TclPro to
> do that. It is certainly not as easy as Python but pre byte compiling Tcl
> code is not as useful as pre compiled Python code because of Tcl's dynamic
> nature.

Oh, Python is pretty dynamic too, if you wish it to be, so there's not
much difference on this score.  Normally, one would distribute sources
anyway.  But over and again you hear about somebody who'd rather
not distribute sources, not so much for "secrecy", as to reduce risk of
internal-users innocently messing things up (there was a question on
this just today, in fact), so it's nice to know that Tcl now also has this
facility, albeit for-$$$ only (I believe Perl has something similar, again
for-$$$...?  Not sure).  I'd still rather have the option for-free just in
case I ever feel a need for it (never have in my life, but who knows:-).


> > But really, I'm most curious to hear about what makes Tcl 'better' for
> > cross-platform work in your view.
>
> You asked for it. Please note that I am not flaming Python in anyway, just
> highlighting some of the ways in which I think it could be improved.

Absolutely!  And thanks.  Most of the issues you raise seem to me to
boil down to 'marketing': we haven't done a good job showing how to
assess Python -- as evidenced by the fact that it does what you want
as well, or better than, the solution you've chosen over it (I think) and
yet, despite a reasonably thorough examination, you have not perceived
this.  I don't know what to do about it, but it's surely a point to ponder.

And the 'dependence on OS linker functionality' issue, aka 'stubs', is
IMHO an excellent point you make -- one that we could fix in the
implementation (2.0 or later, I guess:-) without changing half a comma
in Python-the-language, but still might help (needing to rebuild N
modules for python15.dll -> python16.dll, with the prospect of
doing it again soon for -> python20.dll, is sensitizing me to this very
issue these days!-).


> I like the abstractions that Tcl places on top of things like I/O.

Yep.  But how are Python's abstractions any poorer?  You can basically
use any object that supports a write() method wherever you can use a
builtin file-object for output, etc, etc.  Indeed, the deeply O-O nature of
Python seems to me to afford much more natural ways to provide such
abstractions.

> I think that event driven programming is much cleaner than threads for
many

Full agreement on this point.  I multithread when I really have to, but I do
that just as little as possible, and that turns out to be (for most kinds of
problems I'm handling currently) very little indeed; basically, I sometimes
have to make certain COM servers/controls be freely-threaded (or both-
threaded) for *performance* reasons -- that's the kind of thing I code in
C++, of course, often after _prototyping_ it in Python -- but that's about
it.

In other words, I fully subscribe to Ousterhout's "Why threads are a bad
idea (for most purposes)" article -- it really showed me the way back when
I first read it, and I re-read it periodically to renew the faith:-).

> problems and Tcl excels at that even on Windows which doesn't really like
> event driven programming. WaitForMultipleObjects is a poor substitute for
> select, especially when it comes to sockets.

What makes Tcl better than Python for event-driven processing?  I just
can't see it (after doing a LOT of Tcl, but that was a while ago).  By the
way, I also find Windows quite acceptable for event-driven processing,
but kernel-objects are not the "mainstream" `events' level on Windows;
rather, window-messages are.  One can use asynchronous sockets and
get I/O socket events translated into window-messages, etc.  Asyc I/O
tends to perform much better than multi-threaded blocking I/O on a busy
NT server (it's been a while since I measured that, so I don't know if it's
still true on Win/2000, but it surely used to be!).  Etc, etc.

One nice thing about events-vs-threading on Windows is COM's SCM
(Service Control Manager): one thing it does is mediate between a
multi-threaded world (e.g. requests coming in from several clients on
different threads/processes/machines) and a single-threaded server --
it does that by turning the requests into window-events, which are
serialized in normal event processing (aka message-looping).  Except
for performance, you just don't have to worry about threading; it all
comes in as a serialized stream of events which request something.  Bliss.

But that's Windows -- let's get back to Python.  I do lots of event-driven
processing in Python, and haven't ever felt the need to multi-thread there;
what am I missing?

For example, one outstanding example of the utter power of event
processing versus threads is Sam Rushing's "Medusa", a wonderful
server-kernel which is currently distributed under a very free license
by "Nightmare", his software-house.  Based on an excellent "async
socket" implementation, it provides a single/process, single/thread
server multiplexing requests in event-driven fashion.  Performance
and scalability -- well, you have to SEE it to believe it.

And it's all written in Python.  The asyncore.py core part of it is part
of the Python standard distribution, in fact.


> Tcl's [socket] command is a very powerful abstraction which makes it
> trivially easy to write simple client server code.
>
> Python and Perl on the other hand both provide access to the low level
> socket/bind/accept/connect calls which are harder to use. This low level
> access would also make it harder to implement on a system which provides
> support for sockets in a different way. This may not be particularly
likely
> for sockets but applies to other things as well.

But you don't _have_ to use the low-level access; it's there, sure, but
so are wrappers already built on top of it by exceedingly-clever people,
that provide all the degree of abstraction one could want (IMHO).

E.g., asyncore and asynchat let you work at any level of abstraction
you desire -- from the raw poll call (which can wrap either the select
or poll syscalls), to a dispatcher class to use it, a slightly
richer/simpler
subclass dispatcher_with_send, moving on to what asynchat provides
for typical command/response protocols (smtp, nntp, ftp, ...).  All
event-driven, lightweight, fast, and with the chance to choose whatever
level of abstraction is _suitable_ for a given problem-space.

Maybe herein lies the problem: Python gives you *too much* choice,
and therefore freedom, and therefore architectural responsibility.  There
is nothing in Python to _stop_ you from choosing an approach that is
far too low-level for your needs, architecturally inappropriate.  It's all
based on a conception of *empowering* the programmer rather than
*constraining* him/her, IMHO.  Maybe that's why I love it so much --
if I was looking for constraints, I wouldn't be using scripting, but rather
Eiffel, Sather, or some other good very-strongly-typed language.  But
I guess it may easily cause many people to look at Python, fail to see
the parts of it that are just right for their specific needs (because there
is
SO much in its standard library -- and so little in its
language-core-itself,
but that's "minimalist perfection", also maybe hard to appreciate at first
blush: it seems to have in the language-proper very close to a minimal
core of things that have to be there, with full combination abilities, to
let
all the rest of the edifice be built on solid, elegant foundations) -- and
turn
elsewhere.

If so, then we have a 'marketing' problem rather than a 'technical' one,
but that doesn't make it any less real.  Indeed, it makes it worse, since
this small community appears to have a surfeit of insanely great techies,
but I'll bet we're as short of insanely-great marketeers as of lawyers:-).

Stuff to ponder about... thanks for providing us with it!


And now, coming to a technical point...:

> One of the other things that I think that Tcl does better than Python in
> terms of cross platform support has to do with linking.

    [pretty-sensible-looking analysis snipped]

> whatsoever so in order to load extensions Python has to be built as a DLL
> and every extension has a dependency on that particular version of Python
> so they have to be recompiled, or the Python dll has to have a generic
> name which does not contain the version number.

True.  Moving to python15.dll to python16.dll to python20.dll invalidates
all the applications/extensions/modules/etc which were linked against the
python14.dll, and so on.  I suspect the generic-name idea was not followed
in order to let multiple Python versions coexist (but I don't really
_know_).

> Tcl on the other hand does not rely on the operating systems linking
> mechanism to expose its API to its extensions, rather it explicitly
> gives the extension a pointer to tables of function pointers to all
> the exported functions.

Which is what Python requires its extension-modules to do, but it
doesn't follow that path itself.


> Macros are provided which invisibly map function calls to calls through
> the table. This means that when the extension is built it does not
> have any unresolved references to the Tcl C API so the operating systems
> loader does not get involved at all.
>
> This means that Tcl behaves the same way on all platforms irrespective
> of the behaviour or limitations of the platforms loader.

Yes, a potential cross-platform plus.


> This "Stubs" mechanism is very simple but solves a lot of problems.
> e.g. say that you want to rationalise the Python C API by removing
> old functions but still provide a backwardly compatible API to
> existing extensions. This can easily be done by having two sets of
> tables and giving new extensions a pointer to the new table and old
> extensions a pointer to the old table.

Yes, the potential advantages are clear, I think.

> If anyone is interested in adding Stubs to Python then feel free to
> contact me as I am more than willing to help.

I don't have the Unix machines around at present to help with the
cross-platform part, but I could at least help with the Windows part
if we got a group together on Sourceforge.  It seems to me that we
would have no need to give up on any functionality that Python now
supports; a stubbed-function-extension could be provided _as an
addition_ to the current export-all-functions-through-the-OS approach.

Maybe with a choice of header files or a #if to let either approach be
used.  Hmmm... of course, there will be performance trade-offs (the
native-OS linker surely does name-resolution faster than we can do
portably through Stubs) so both approaches are going to have to be
supported.  But I can surely see a niche where the Stubs approach
may be preferable -- paying some extra startup cost for the benefit
of running optimally/identically even on machines with inferior OS
loaders.  Re-hmmm....


Alex