[Python-Dev] Generally boared by installation (Re: Setting project home path the best way)

[Kristján Valur Jónsson]

Yes!
For many years I have been very frustrated by the install-centric nature of python.  I am biased, of course, by the fact that I am developing an application where python is embedded, an application that needs to run out of the box.  A developer may have many many versions (branches) of the application on his drive, and each needs to work separately.
We have managed to isolate things, by patching python (and contributing that patch) to override the default library seach path (and ignore environment paths) when python is started up thogh the api.  All well and good.
But recently we have started in increasing amount to use external libraries and packages and we have been introduced to the dependency hell that is public python packages.  In this install-centric world, developers reference huge external packages without a second thought, which cause large dependency trees.  Using a simple tool may require whole HTTP frameworks to be downloaded.
What is worse is when there are versioning conflicts between those dependencies.

I don't have a well formed solution in mind, but I would see it desirable to have a way for someone to release his package with all its dependencies as a self-contained and isolated unit.  E.g. if package foo.py relies on functionality from version 1.7 of bar.py, then that functionality could be bottled up for foo´s exclusive usage.
Another package, baz.py, could then also make use of bar, but version 1.8.  The two bar versions would be isolated.

Perhaps this is just a pipedream.  Even unpossible.  But it doesn't harm to try to think about better ways to do things.
K


-----Original Message-----
From: Christian Tismer [mailto:tismer at stackless.com] 
Sent: 15. nóvember 2012 23:10
To: Kristján Valur Jónsson
Cc: python-dev at python.org
Subject: Generally boared by installation (Re: [Python-Dev] Setting project home path the best way)

Hi guys,

I am bored of installing things. 
Bored of things that happen to not work for some minor reasons. 
Reasons that are not immediately obvious. 
Things that don't work because some special case was not handled. 
Things that compile for half an hour and then complain that something is not as expected. 
May it be a compiler, a library, a command like pip or easy-install, a system like macports or homebrew, virtualenv, whatsoever. 

These things are all great if they work. 

When they do not work, the user is in some real trouble. And he reads hundreds Of blogs and sites and emails, which all answer a bit of slightly related questions, but all in all - 

This is not how Python should work !!

I am really bored and exhausted and annoyed by those packages which Pretend to make my life eadier, but they don't really. 

Something is really missing. I want something that is easy to use in all cases, also if it fails. 

Son't get me wrong, I like things like pip or virtualenv or homebrew. 
I just think they have to be rewritten completely. They have the wrong assumption that things work!

The opposite should be the truth: by default, things go wrong. Correctness is very fragile. 

I am thinking of a better concept that is harder to break. I thin to design a setup tool that is much more checking itself and does not trust in any assumption. 

Scenario:
After hours and hours, I find how to modify setup.py to function almost correctly for PySide. 

This was ridiculously hard to do! Settings for certain directories, included and stuff are not checked when they could be, but after compiling a lot of things!

After a lot of tries and headaches, I find out that virtualenv barfs on a simple link like ./.Python, the executable, when switching from stock Python to a different (homebrew) version!!

This was obviously never tested well, so it frustrates me quite a lot.  

I could fill a huge list full of complaints like that if I had time. But I don't. 

Instead, I think installation scripts are generally still wrong by concept today and need to be written in a different way. 

I would like to start a task force and some sprints about improving this situation. 
My goal is some unbreakable system of building blocks that are self-contained with no dependencies, that have a defined interface to talk to, and that know themselves completely by introspection. 

They should not work because they happen to work around all known defects, but by design and control. 

Whoever is interested to work with me on this is hereby honestly welcomed!

Cheers - chris

Sent from my Ei4Steve

On Nov 15, 2012, at 10:17, Kristján Valur Jónsson <kristjan at ccpgames.com> wrote:

> When python is being run from a compile environment, it detects this by looking for "Lib" folders in directories above the one containing the executable. 
> (I always thought that this "special" execution mode, hardwired in, 
> was a bit odd, and suggested that this could be made a function of pep405) Anyway, keeping your executable as part of the tree is the trick I use, and to make things nice I put  right next to it:
> site.py
> sitecustomize.py
> 
> sitecustomize.py is where you would put the logic to set sys.path by walking up the hierarchy and finding the proper root.
> site.py is there to merely import sitecustomize.py, in case a site.py is not found in all the default places python looks.
> 
> K
> 
> 
>> -----Original Message-----
>> From: Python-Dev [mailto:python-dev-
>> bounces+kristjan=ccpgames.com at python.org] On Behalf Of Christian 
>> bounces+Tismer
>> Sent: 11. nóvember 2012 20:31
>> To: python-dev at python.org
>> Subject: [Python-Dev] Setting project home path the best way
>> 
>> Hi friends,
>> 
>> I have a project that has its root somewhere on my machine.
>> This project has many folders and contains quite some modules.
>> 
>> There is a common root of the module tree, and I want to use
>> - either absolute imports
>> - relative imports with '.'
>> 
>> Problem:
>> 
>> - I want to run any module inside the heirarchy from the command-line
>> 
>> - this should work, regardless what my 'cwd' is
>> 
>> - this should work with or without virtualenv.
>> 
>> So far, things work fine with virtualenv, because sys.executable is 
>> in the project module tree.
>> 
>> Without virtualenv, this is not so. But I hate to make settings like 
>> PYTHONPATH, because these are not permanent. .
>> 
>> Question:
>> 
>> How should I define my project root dir in a unique way, without 
>> setting an environment variable?
>> What is the lest intrusive way to spell that?
>> 
>> Reason:
>> 
>> I'd like to make things work correctly and unambigously when I call a 
>> script inside the module heirarchy. Things are not fixed: there exist 
>> many checkouts In the file system, and each should know where to 
>> search its home/root in the tree.
>> 
>> Is this elegantly possible to deduce from the actually executed script file?
>> 
>> Cheers - chris
>> 
>> Sent from my Ei4Steve
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-
>> dev/kristjan%40ccpgames.com
> 
> <winmail.dat>