intentional tease ( was: New Python User Question)

[Grant Griffin]

In article <mailman.998816920.4899.python-list at python.org>, tanzer at swing.co.at
says...
>
>
>Grant Griffin <not.this at seebelow.org> wrote:
>
>[delightful musings skipped]

Thanks. :-)

>> In Python, this function works equally well for lists, tuples, arrays,
>> or Numeric arrays full of integers, floats, or longs (except that it
>> always returns a float--darn!) That's beautiful. But in all honesty,
>> how often do you really _need_ to do generic programming? I'm
>> guessing, only a very small fraction of the time--1%, maybe? Most of
>> the time, you (the programmer) really do know what types you're
>> working with--even if Python doesn't, not because Python's stupid, but
>> because it's not as smart as you, and you haven't told it (which, if
>> you think about it, isn't really the smartest thing you ever did
>> <wink>.)
>
>Your statement about the rarity of generic programming might apply to
>you (although I doubt it) but in general its just plain wrong.

Of course, this varies depending on what you're trying to do--in both general
and in specific cases.

In all honesty, most of what _I_ use Python for could be categorized as "text
processing".  Basically, I read in a file and turn it into a different file, or
maybe make summaries of it.  (I'd like to use Python for more.)

I also once did a fairly complicated DSP simulation in Python, using NumPy. 
(Most DSP folks would use Matlab for ath, but I like Python because it's much
more beautiful.  And it's much cheaper <wink>.)

Python has become an essential tool for me, but, unfortunately, due to the
speed/size issue, I'm able to use it only for "offline" PC tasks like this.  My
own production code generally runs in embedded systems.  In those cases, due to
the inherent requirement for maximum speed and minimum size, one has little
choice but to use a compiled language.  (Historically, due to compiler
availability, one has had little choice in DSP but to use C, though C++
compilers have recently become available for DSP microprocessors.)

In embedded systems (and many other application domains), using an interpreter
is simply not an option.  Other posts in this thread have suggested that the
lesser speed of Python is worth what you gain in development time.  I
wholeheartedly agree--at least in those cases where you can trade the two.  For
the offline data processing I do, it's a no-brainer: use Python.

But depending on how many embedded boxes one plans to sell, the ease of
development of Python can _truly_ be false economy, if it forces you to put $10
of extra computer inside the box.  (Python still makes a good prototyping
language, though.)  More imporatantly, you sometimes simply couldn't get there
from here if you used an interpreter.

Although there are might be some folks who can spend 100.0% of their programming
lives in Python (except when they have to write extension modules <wink>), I
suspect that _most_ folks who use Python do part of their work in Python and
part in a compiled language--probably C/C++.

*KEY POINT*: Anything that greatly speeds Python (and a compiler--in whatever
form--is the only thing we can reasonably expect would, after 10 years of
careful tweaking by lots of talented folks!) increases the percentage of
programs we can write in Python.  That is undeniably A Good Thing.

However, in working on my little project, I've realized that even leaving
dynamic variables aside, certain things in the design of Python are inherently
"slow".  A simple example is the fact that Python allows negative array indices.
When implementing that, one can only check for a negative index, then add the
length.  And even if it wasn't negative, you have to bounds-check the result, to
possibly raise an IndexError.  (An "assert" won't do: the Python code might have
been written to rely on an IndexError.)

Therefore, compiled Python will always be slower than something written from
scratch in C++ (which doesn't do bounds-checking: you write it where you need it
rather than it happening automatically in every case).

>
>Using generics in Ada or templates in C++ might be difficult for the
>average programmer but doing generic programming in Python is so
>trivial that just about everybody does it.

True.  Like I said, I was doing it in Python and I didn't even know it.  I guess
that means it must be easy!

But for the vast majority of offline text-processing tasks I do in Python, I'm
not sure generic programming is of much use.  In these cases, the data consists
almost entirely of strings, lists of strings, dictionaries of strings, etc.  And
I, the programmer, know all that in advance.  In such cases, I'm writing code
that wants to be statically typed, and that makes it a good candidate for
compilation.

Although Python has a few methods common among strings, lists, and tuples, any
code that uses a special string method (e.g. split) wants the string to be
statically typed: you're not going to be using that code for lists, tuples,
ints, floats, files, etc.

I bet there's nobody who has used Python for more than a month that hasn't used
a string method.  Moral of the story: even those who claim to do a lot of
generic programming don't do it 100.0% of the time, so even they can benefit
from a little static typing.

>Every time somebody
>reassigns sys.stdout to something which is not a file, they are are
>enjoying the benefits of generic programming.

But we can do that sort of thing in C and C++, even without templates.  C
pointers are good for that (Python uses 'em!), and C++ virtual functions are
even better.  Dynamic typing is just one technique among many to do generic
programming.

>
>> OK, so what if a system already existed that provided the benefits of
>> generic programming (when you _do_ need it), without the overhead?
>> What if that system wasn't Python, but, with a little work (a _lot_ of
>> work actually--that's why I need your help!), could be made to _seem_
>> like Python?
>> =
>
>> I'm seeing a speed factor that ranges from about 1:1 to 70:1,
>> depending on the algorithm. Also, as compiled programs, my executables
>> are a lot smaller than equivalent "frozen" Python programs, and are
>> *much* harder to de-compyle.
>
>Now that certainly sounds interesting. 

Thanks.  (Maybe my little dishwasher story worked <wink>.)

>Will your system also allow the
>use of generic programming when the need for it was not foreseen by
>the original author of the code (for instance, the passing of an
>object of a compatible but completely different type as an argument to
>a function)?

Yes.  It's built on top of another system of generic programming, so that
capability comes for free.  Suppose you invent a new type XYZ and you want to
make a list of XYZs.  No problem.  Suppose you want to reassign sys.stdout. 
Enjoy.

Also, it comes with its own system of dynamic types.  I'm not sure this part is
really mature yet, but it does work.  For example, an averaging function similar
to what I posted previously accepts lists or tuples populated with integers or
floats.

BTW, I learned something important in this thread.  I originally created my
system of dynamic types mainly just to emulate Python--and also because it was a
fun design problem.  In all honesty, though, I didn't really think anybody would
use it much if a system of static types was available.  But I just learned that
it _is_ important, if for no other reason than user acceptance.  (When you
create something like this, you have to eliminate as many excuses not to use it
as you can <wink>.)

But I bet the static types will be much more popular in the end.  I'm not sure
that my system of dynamic types will run a whole lot faster than Python's.  (I
haven't benchmarked it yet.)  Instead, I think the real "bang-for-the-buck" of
my system (in terms of speed) is due its option of using static types.

>If not I'd rather stay with Python.

I'm not trying to replace Python, just expand it's universe.  When the benefits
of speedier, smaller-than-"frozen", more obscured executables are not of
interest, Interpreted Python will *always* be a better choice than my system. 
Interpreted Python will always have more features, and will always be more
mature.  I don't have any plans to stop using Interpreted Python myself.

However, I suspect that one or more of the benefits of Compyled Python will be
of interest to nearly everybody at some time or another.  (Just ask yourself: If
such a thing existed, worked well, and were free, would I use it?  Honest answer
for most folks: yes, at least every now-and-then.)

Remember, folks, I come not to bury* Ceasar, but to praise him.

filching-is-the-sincerest-form-of-flattery-ly y'rs,

=g2
*or depose

_____________________________________________________________________

Grant R. Griffin                                       g2 at dspguru.com
Publisher of dspGuru                           http://www.dspguru.com
Iowegian International Corporation            http://www.iowegian.com