bytecode non-backcompatibility

[Maurice LING]

> 
>>2. The current compilation scheme (compiling to bytecode as and when
>>it is needed) works well for scripting purposes but is less desirable
>>in commercial settings. Less distribution happens when it is used
>>purely for scripting purposes, such as system maintenance or tuning.
> 
> 
> The solution with the current situation depends on what you mean by
> "commercial settings".
> 

Commerically, I will mean distributing the program without the codes. 
The best way under the current method is to freeze the application. In 
Java, you can just distribute the JAR files or CLASS files. I can 
imagine there will be some level of touchiness when source codes are 
distributed...

> 
>>3. Using Python in commercial settings will usually require
>>distribution of resulting software and it is may or may not be
>>desirable to distribute source codes as well. Unless the application
>>is frozen, distributing source code is a must.
> 
> 
> People have recommended that you distribute the Python interpreter
> with commercial applications. This makes installing them much easier
> for the end user. It also means the byte code and the interpreter will
> always match.
> 
> 
>>4. One advantage that Java platform has is that it does not require
>>the release of source files and still promotes platform-independence.
> 
> 
> Java is only "platform-independent" to the degree that the jvm and
> libraries you need for your application are already available.  A
> quick google finds applications that require jvm versions - so I
> suspect that you can't always run Java code built for a new version of
> the jvm on older jvms. So you have to make sure the target platform
> has a recent enough implementation of the jvm installed. You may have
> problems if you try using a different groups jvm as well. I haven't
> looked into that in a number of years.
> 
> So you can't just ship java bytecodes to someone expecting they'll be
> able to run it out of the box. They may well need to update their java
> environment, or install a second one (I recall one time having four
> Java products that took in total three different jvms to run. Bleah.)
> 
> This isn't really different from Python. In both cases, the end user
> has to have a suitable version of the bytecode interpreter and
> libraries installed. Java is a little better in that they provide
> backwards compatability.
> 

I think backward compatibility is good enough. A Java JAR file compiled 
under for JVM 1.2 can still run in JVM 1.4. I don't think anyone will or 
can expect forward compatibility, that will require godly powers...

According to Steve Holden's reply, Python C API is only backward 
compatible in minor versions but not major versions. The issue is, why 
not major versions as well unless there's something exceedingly prohibitive?

> 
>>5. Unstable bytecodes makes updating to a newer version of Python very
>>tedious and risk breaking old scripts, if they uses C modules.
> 
> 
> Unstable bytecodes have nothing to do with these problems. 
> 
> The current CPython installation process puts the python command and
> the libraries for different versions in different directories. This
> allows you to have multiple versions installed so you can keep old
> scripts working with the old version until you've had time to test
> them. It also makes life much easier on the developers, as they can
> have a development version installed on the machine at the same time
> as they have a production version without breaking the old scripts. It
> also means you have to reinstall all your modules on the new
> installation - which is what makes the update process tedious for me.
> 
> Now, this could be mitigated by having Python libraries installed in
> the same location for all versions. You could fix all the bytecode
> files by running compileall.py as a script. Sometimes, a module won't
> work properly on a new version of Python, and will have to be
> updated. You'll have to find those by trial and error and fix them as
> you find them. You'll also have to recompile all the C modules, which
> will break running scripts on the old interpreter.
> 
> Under the currewnt system, old scripts that are run by the new
> interpreter will break if all the modules they need aren't installed
> yet. Once they're installed, the scripts should work.
> 
> If you have scripts that use C modules that aren't installed in the
> standard Python search path, they may well break on the new
> interpreter until you recompile them. Having both versions of the
> interpreter available makes this tolerable.
> 
> The cure I proposed seems worse than disease. If you've got a better
> solution, I'm sure we'd be interested in hearing it.
> 

Perhaps this is another call for Python version of CPAN (CPyAN or PYAN). 
  It can be modelled after Fink or Darwinports.

If I remembered correctly, Fink uses apt-get and curl.

What can be done in PYAN is to encourage all 3rd party library 
developers to centralize their libraries in it, which I think all will 
gladly respond. All that is needed to be  deposited in PYAN is a 
description file, like the simplest setup.py file. All that is needed in 
this description file is
1. where to download the source codes (e.g. CVS)?
2. version number?
3. standard stuffs (optional) like authors, descriptions, copyright?

Python can then have a built-in mechanism to read the description file 
and download the source codes and do the standard "sudo python setup.py 
install" to install the library into site-package. The main thing this 
mechamisn does is to maintain what had been installed in site-package 
through it and what versions. So when newer versions of Python is 
installed, there can be a script to take the old site-package log, 
download the same set of libraries and install them for the new version 
of Python.

Darwinports uses a very simple means to achieve this. All the 
description files are in a CVS repository. The actual source codes for 
the libraries may or may not reside in the CVS repository. Darwinports 
system installation is little more than checking out the entire 
repository of description files. When user wants to install a library, 
it then downloads the library's source codes and performs the standard 
"configure, make, make install" operations. Of course, the root user 
will have to do a update to update to the latest version of description 
files.

I think something like this can be set up for Python quite easily. I 
recall some time back, there's a long discussion about setting up 
Python's version of CPAN but I can't recall the contents of the 
discussions.

> 
> Fixing the byte-code problem won't help with the dependency
> problem. If anything, it'll make it worse, because you'll have old
> libraries that were installed with previous versions of Python to
> contend with. If those are acceptable, that's all well and good. But
> for complex packages like eric - which depends on sip and qt and a
> number of other things - the latest versions tend to rely on having
> up-to-date libraries. So you install eric, and watch it fail because
> some library is out of date. Update that library and repeat. The
> easiest way to deal with this is to find all the libraries it needs,
> and just update them all.
> 
> The dependencies problem is actually pretty easy to solve. In fact, in
> many Python environments, it's already solved. On my system, if I
> install Eric3 using the provided installation package, the
> dependencies will be picked up automatically. Not very helpfull if
> you're not on such a system, I know. The long-term solution is for
> PyPI to grow to include this functionality.
> 

I must say that I do not quite understand your system. Please enlighten me.

Thanks and cheers
Maurice