scripting language newbie - compatibility

[Paul Duffin]

Alex Martelli wrote:
> 
> "Paul Duffin" <pduffin at hursley.ibm.com> wrote in message
> news:398EA451.CBFB72 at hursley.ibm.com...
>     [snip]
> > I had a feeling that the Windows version of Python while being part of the
> > core was supported seperately in some way.
> 
> No such thing, that I know of.

See Cameron's reply for a clarification.
> 

> > > But really, I'm most curious to hear about what makes Tcl 'better' for
> > > cross-platform work in your view.
> >
> > You asked for it. Please note that I am not flaming Python in anyway, just
> > highlighting some of the ways in which I think it could be improved.
> 
> Absolutely!  And thanks.  Most of the issues you raise seem to me to
> boil down to 'marketing': we haven't done a good job showing how to
> assess Python -- as evidenced by the fact that it does what you want
> as well, or better than, the solution you've chosen over it (I think) and
> yet, despite a reasonably thorough examination, you have not perceived
> this.  I don't know what to do about it, but it's surely a point to ponder.
> 
> And the 'dependence on OS linker functionality' issue, aka 'stubs', is
> IMHO an excellent point you make -- one that we could fix in the
> implementation (2.0 or later, I guess:-) without changing half a comma
> in Python-the-language, but still might help (needing to rebuild N
> modules for python15.dll -> python16.dll, with the prospect of
> doing it again soon for -> python20.dll, is sensitizing me to this very
> issue these days!-).
> 

Adding support for 'stubs' is a little fiddly but it is not complicated
and shouldn't reduce the functionality available through the C API
at all.

> > I like the abstractions that Tcl places on top of things like I/O.
> 
> Yep.  But how are Python's abstractions any poorer?  You can basically
> use any object that supports a write() method wherever you can use a
> builtin file-object for output, etc, etc.  Indeed, the deeply O-O nature of
> Python seems to me to afford much more natural ways to provide such
> abstractions.
> 

True. I was thinking more about the event driven abstractions.

> > I think that event driven programming is much cleaner than threads for
> many
> 
> Full agreement on this point.

[snipped details]

> > problems and Tcl excels at that even on Windows which doesn't really like
> > event driven programming. WaitForMultipleObjects is a poor substitute for
> > select, especially when it comes to sockets.
> 
> What makes Tcl better than Python for event-driven processing?  I just
> can't see it (after doing a LOT of Tcl, but that was a while ago). 

[snipped a lot of interesting Windows specific details]

> But that's Windows -- let's get back to Python.  I do lots of event-driven
> processing in Python, and haven't ever felt the need to multi-thread there;
> what am I missing?
> 
> For example, one outstanding example of the utter power of event
> processing versus threads is Sam Rushing's "Medusa", a wonderful
> server-kernel which is currently distributed under a very free license
> by "Nightmare", his software-house.  Based on an excellent "async
> socket" implementation, it provides a single/process, single/thread
> server multiplexing requests in event-driven fashion.  Performance
> and scalability -- well, you have to SEE it to believe it.
> 
> And it's all written in Python.  The asyncore.py core part of it is part
> of the Python standard distribution, in fact.
> 

You are correct that the problem is probably 'marketing' I have not seen
many mentions of asyncore.py but have seen a lot of examples of how
to use the lower level socket calls.

> > Tcl's [socket] command is a very powerful abstraction which makes it
> > trivially easy to write simple client server code.
> >
> > Python and Perl on the other hand both provide access to the low level
> > socket/bind/accept/connect calls which are harder to use. This low level
> > access would also make it harder to implement on a system which provides
> > support for sockets in a different way. This may not be particularly
> likely
> > for sockets but applies to other things as well.
> 
> But you don't _have_ to use the low-level access; it's there, sure, but
> so are wrappers already built on top of it by exceedingly-clever people,
> that provide all the degree of abstraction one could want (IMHO).
> 
> E.g., asyncore and asynchat let you work at any level of abstraction
> you desire -- from the raw poll call (which can wrap either the select
> or poll syscalls), to a dispatcher class to use it, a slightly
> richer/simpler
> subclass dispatcher_with_send, moving on to what asynchat provides
> for typical command/response protocols (smtp, nntp, ftp, ...).  All
> event-driven, lightweight, fast, and with the chance to choose whatever
> level of abstraction is _suitable_ for a given problem-space.
> 
> Maybe herein lies the problem: Python gives you *too much* choice,
> and therefore freedom, and therefore architectural responsibility.  There
> is nothing in Python to _stop_ you from choosing an approach that is
> far too low-level for your needs, architecturally inappropriate.  It's all
> based on a conception of *empowering* the programmer rather than
> *constraining* him/her, IMHO.  Maybe that's why I love it so much --
> if I was looking for constraints, I wouldn't be using scripting, but rather
> Eiffel, Sather, or some other good very-strongly-typed language.  But
> I guess it may easily cause many people to look at Python, fail to see
> the parts of it that are just right for their specific needs (because there
> is
> SO much in its standard library -- and so little in its
> language-core-itself,
> but that's "minimalist perfection", also maybe hard to appreciate at first
> blush: it seems to have in the language-proper very close to a minimal
> core of things that have to be there, with full combination abilities, to
> let
> all the rest of the edifice be built on solid, elegant foundations) -- and
> turn
> elsewhere.
> 
> If so, then we have a 'marketing' problem rather than a 'technical' one,
> but that doesn't make it any less real.  Indeed, it makes it worse, since
> this small community appears to have a surfeit of insanely great techies,
> but I'll bet we're as short of insanely-great marketeers as of lawyers:-).
> 

Marketing is a real pain isn't it.

> Stuff to ponder about... thanks for providing us with it!
> 
> And now, coming to a technical point...:
> 
> > One of the other things that I think that Tcl does better than Python in
> > terms of cross platform support has to do with linking.
> 
>     [pretty-sensible-looking analysis snipped]
> 
> > whatsoever so in order to load extensions Python has to be built as a DLL
> > and every extension has a dependency on that particular version of Python
> > so they have to be recompiled, or the Python dll has to have a generic
> > name which does not contain the version number.
> 
> True.  Moving to python15.dll to python16.dll to python20.dll invalidates
> all the applications/extensions/modules/etc which were linked against the
> python14.dll, and so on.  I suspect the generic-name idea was not followed
> in order to let multiple Python versions coexist (but I don't really
> _know_).
> 

That is probably correct.

> > Tcl on the other hand does not rely on the operating systems linking
> > mechanism to expose its API to its extensions, rather it explicitly
> > gives the extension a pointer to tables of function pointers to all
> > the exported functions.
> 
> Which is what Python requires its extension-modules to do, but it
> doesn't follow that path itself.
> 

I know, I saw that and found it very strange indeed that Python didn't
do it itself.

> > Macros are provided which invisibly map function calls to calls through
> > the table. This means that when the extension is built it does not
> > have any unresolved references to the Tcl C API so the operating systems
> > loader does not get involved at all.
> >
> > This means that Tcl behaves the same way on all platforms irrespective
> > of the behaviour or limitations of the platforms loader.
> 
> Yes, a potential cross-platform plus.
> 
> > This "Stubs" mechanism is very simple but solves a lot of problems.
> > e.g. say that you want to rationalise the Python C API by removing
> > old functions but still provide a backwardly compatible API to
> > existing extensions. This can easily be done by having two sets of
> > tables and giving new extensions a pointer to the new table and old
> > extensions a pointer to the old table.
> 
> Yes, the potential advantages are clear, I think.
> 
> > If anyone is interested in adding Stubs to Python then feel free to
> > contact me as I am more than willing to help.
> 
> I don't have the Unix machines around at present to help with the
> cross-platform part, but I could at least help with the Windows part
> if we got a group together on Sourceforge.  It seems to me that we
> would have no need to give up on any functionality that Python now
> supports; a stubbed-function-extension could be provided _as an
> addition_ to the current export-all-functions-through-the-OS approach.
> 

You will find that if you do a reasonably good job with it that it will
very quickly become the standard way of doing it, especially as extension
writers are used to doing it themselves.

As an example in order to build an extension to use the Tcl Stubs
interface an extension writer has to do three things.

	Build the extension with -DUSE_TCL_STUBS.

	Add a call to Tcl_InitStubs in the entrypoint.

	Link the extension to the Tcl Stubs library; this is a static 
	library which implements Tcl_InitStubs. It simply extracts the
	pointer to the Stubs tables out of the Tcl_Interp parameter
	which Tcl passes into the entrypoint and stores it in a global
	variable which all of the macros use.

> Maybe with a choice of header files or a #if to let either approach be
> used. 

That is the way Tcl does it.

> Hmmm... of course, there will be performance trade-offs (the
> native-OS linker surely does name-resolution faster than we can do
> portably through Stubs) so both approaches are going to have to be
> supported. 

The point with Stubs is that you don't do name resolution at all. The
supplier of the interface, Python in this case, populates the table of
function pointers itself, no name resolution to do.

> But I can surely see a niche where the Stubs approach
> may be preferable -- paying some extra startup cost for the benefit
> of running optimally/identically even on machines with inferior OS
> loaders.  Re-hmmm....

I think that you may find that the startup costs of Python may be
marginally more as it has to initialise the table but the time taken
to load an extension should be reduced as there is no name resolution
to do. I think that some platforms store symbols in linear lists and 
not hashes which means that name resolution can take a long time.

One other thing which differentiates Python and Tcl cross platform
support is that Tcl tends to provide more 'complete' cross platform
solutions. A recent thread on strptime illustrates the fact very
clearly. I did think that this was the case but did not have any
specific examples.

Python only provides a strptime function if it is supported by the
platform it is running on. This means that it is unusable for writing
cross platform code. 

Tcl on the other hand implements its own version of the function to
ensure that it will work on all platforms.

Unfortunately because of this focus on completeness it does take a
lot longer to add functionality to Tcl because it has to work on
all platforms. This means that Python probably has access to more
of an operating systems functionality than Tcl.

For example Tcl does not have fork, or signal, or ... although they
are available in extensions.