Python questions -> compiler and datatypes etc

[Richard Jones]

On Wednesday 10 October 2001 17:28, Billy wrote:
> Yeah, java can do the job, but so can C++... see what
> I mean? Just cause it can do it doesn't mean its
> practical. Java is ok for webdev (i have done a bit)
> it's just tediously slow in implementing sometimes.

I agree, it is. I was mostly pointing out that your statement "java is not 
suited for specific web development" may be construed as flamage :)



> Zend Encoder (www.zend.com) takes text php code and
> puts it into an intermedary format not readable by
> humans. each time a script is called the interpreter
> calls the encoded script. This means you can
> distribute your applications without letting people
> see the code.

Python's bytecode is not human-readable (unless you're Tim Peters, I think). 
In the absence of original source, python is perfectly happy to import just 
the bytecode. That is, if you

1. create "foo.py" (empty file is OK)
2. python -c 'import foo'
3. rm foo.py
4. python -c 'import foo'

... will work because #4 is able to import the bytecode from foo.pyc.


> I've seen various reviews, the scientific basis of
> them was rather dubious but you could definately
> gather a general feeling that python was slower than
> java, to a reasonable extent.

In general, yes. The devil is in the detail here. There are several instances 
where Java runs like a dog (hello, Swing) and there are similar instances 
where Python does (complex mathematics, not using Numeric). There are ways to 
get them close - using C modules in Python (eg. Numeric) or avoiding Swing 
like the plague :)


> Compiling python is at the impossible end of the
>
> > scale of ease of
> > implementation. There has been a lot of discussion
> > about this issue on this
> > list in the past. Search for "python compiler" and
> > enjoy :)
> >
> > Python compiles to bytecode. Python's runtimes
> > execute compiled bytecode.
> > There are several runtimes - one of which is Java.
> > The predominant one is
> > written in C.
>
> -------------------------------------------
>
> I'm unclear on your explanation here. Would I be
> correct in saying that there is no compiler (source
> code to machine code) for python. And that this is due
> to the very complex nature of implementing such a
> device.

There is no compiler that takes python code an produces machine code. And you 
are correct, it's because of the "complex nature of implementing such a 
device". There have been efforts (one of them quite recent). I can't find any 
refrences though, sorry.


> *Off topic:
> I would be interested in finding out more information
> on this, why can't you do it with python when people
> are doing it with java? Any pointers to web resources
> etc??

Do a search in the python mailing list for "python compiler". The answer is 
invariably "it's bloody hard". This stems from the fundamental dynamic typing 
of python. It's very very very hard to write a compiler when you don't know 
the type of the objects you're dealing with beforehand. Java knows these 
types, because it's statically typed. Therefore it's relatively trivial 
(compared to python) to write a compiler for Java.


> And you say that the python interpreter takes bytecode
> and executes that. Does this mean that you can use
> python to compile your source code into a format which
> is not readable by humans (bytecode or whatever..
> which is as reasonably reverse-engineering proof) and
> distribute that bytecode (not source code) to stop
> tampering and protect your commercial interests?

As I mentioned above, the bytecode is not human-readable. It is possible to 
decode the bytecode back into something that's almost exactly the same as the 
original source, using a program called decompyle.

This also has been discussed extensively. The end result of the discussions 
is that it's possible to obscure your bytecode, but it's almost impossible to 
do it perfectly. This is also the case with any other programming - even C or 
assembler is eventually still decodable (oh, the days of disassembling 68k 
Amiga demos to figure how people optimised their 3d routines... *ahem* I 
digress.)

Again, search for "hiding bytecode" (or similar) in the python mailing lists.

Anyone who believes that their application is safe from prying eyes and 
fingers obviously ignores the widespread warez distribution of cracked 
applications and application keys and so on.


> I did do an extensive search for information on a
> python compiler... all accross the web, all i found
> were a few "discontinued" sites that dated back to
> June 5th 1957. I never ask questions without checking
> for myself first.

The python mailing list is a much better place to start for searching - it's 
up-to-date for one thing (web searching relies on web crawlers which are 
notorious for being slow to update.)


> I would want to use strict datatypes and generic types
> to interchange speed and efficency with flexability.

Then you'll be wanting to develop that speedy code in a language that allows 
it - use C or C++ if you're develping in C Python or Java if you're 
developing in Jython.


> Java lets you define
> int foo()
> double foo() and
> object foo()
>
> very nice for building generic classes.
>
> keyword args/ (*args thing <- can't quite understand
> that one) can't do the same as say:
>
> excuse the syntax... not familiar enough with python
> yet
>
> class foo:
> #load foo from file
> define foo(filename, directory)
> filesystem calls etc etc...
>
> #load foo from database
> define foo(datbasename)
> connect to database etc etc......
>
> or even better
> #load foo from file
> define foo(FileHandler fhFile)
> bla bla bla
>
> #load foo from database
> define foo(DatabaseObject database)
> bla bla bla
>
> So what your saying is that python doesn't have the
> ability to switch between this?

Your mailer stuffed up your indentation there, but I think I get the gist of 
it. In short, the dynamically-typed way of doing the above is:

class foo:
  def __init__(self, database=None, file=None):
    if database is not None:
       # init from database
    elif file is not None:
       # init from file
    else:
       raise ValueError, "I need a database or file to work with"

_or_ the less flexible

class foo:
  def __init__(self, data):
    if isinstance(data, DatabaseObject):
       # init from database
    elif isinstance(data, FileHandler):
       # init from file
    else:
       raise ValueError, "I need a database or file to work with"


Having said that, I'd ask questions about how the code is actually structured 
there. The "foo" class obviously does something with some data source, be it 
a file or a database. It therefore has to have some handling of the 
interactions with the data source (above, you have read operations - does it 
do writing too?). Something like the following is significantly more 
"pythonic":


class FooBase:
  ''' defines all the methods that operate on Foo data '''

class FooDatabase(FooBase):
  ''' defines interactions between a Foo and a DatabaseObject '''

class FooFile(FooBase):
  ''' defines interactions between a Foo and a FileHandler object '''


And then when you want to use one or the other, you'd instantiate the 
appropriate one - either FooDatabase(database_object) or FooFile(file_object).

Another pattern to use is to go with a proxy class. Choose the interface that 
is most convenient and write a wrapper class for the other one. That is:


class Foo:
   ''' perform operations on a DatabaseObject '''

class FileAsDatabaseObject:
   ''' accept a FileHandler and make it look like a Database Object '''


Lots of different ways you can do things :)

And once you truly grok the idea of dynamic typing, you'll see there a lots 
more - and they all make soooo much more sense than defining

  foo(FileHandler fhFile)
  foo(DatabaseObject database)
  ... etc


> Thanks heaps for your time,

No problem, hope I've helped!


    Richard