organizing many python scripts, in a large corporate environment.

[Dan Stromberg]

If you just want to be unhappy about the current situation, I hereby
formally bestow upon you permission to be so.  :)

The problem seems to be that you want to hook into the python process,
without hooking into the python process.  Were this possible, it seems we
might have a serious security issue to face.

Most sites don't have these difficulties, because they either allow changing
an environment variable globally, or allow changing site-packages globally.
Or more likely, they allow both.

virtualenv might be a good way for you to go, but I'm not confident about
that.  You'd still need to hook into that somehow.

You could perhaps push through python running /usr/local/mumble if it has
the same owner as the python interpreter or something, but that may have
security implications that'd need to be carefully considered.  Similarly,
you could perhaps push through python running ~/.python if it has
appropriate ownership, but it sounds like this wouldn't be viable in your
environment either.

The easiest thing to do, given your unusual constraints, is probably to put
the following at the top of each script:

#!/usr/bin/python

# start initializer boilerplate
import sys
sys.path.insert('/my/magic/directory')
import initializer_mod
# end initializer boilerplate

The vast majority of your site specifics would go in
/my/magic/directory/initializer_mod.py.  Then if you someday need to change
/my/magic/directory to /your/mundane/directory, sed -i is your friend, so
long as you keep these 3 (or 5) magic lines nice and consistent.  If you
really, truly, honest-to-whatever despise boilerplate, you could even set up
a script that automatically adds these lines at the beginning of each of
your scripts, if they aren't already there - in fact, it could replace
everything in between those two magic comments, so you can expand on that as
"needed".

But seriously - is it really so bad talking to your IT group about one small
change that could be used by multiple departments?  If the constraints
you're under outside of your IT group are truly as ossified as you make them
sound, shouldn't that add weight to your case when you present it to your
IT?

If you just want to get a handle on what is importing what in a modestly
sized directory (even a few hundred files isn't really that much), you could
probably try a graphviz graph generated using something like snakefood.

BTW, foo.py isn't generally a good thing to put on your $PATH:
http://lintian.debian.org/tags/script-with-language-extension.html
I see it as language vanity that needlessly complicates rewriting a script
from one language to another - it seems to come mostly from Windows which
has fits sometimes without file extensions, and the perl world where
language extensions on CGI scripts were seen as a means of advocacy.  What
goes on your $PATH is part of an interface though - there's little point in
clunking that up interfaces with implementation detail.  Yes, you might
actually want to rewrite your Python or Perl or Ruby into C or Go or Haskell
or OCaML someday.

Despite a degree of snarkiness in this post, I do hope this message helps
you somehow.

On Fri, Mar 11, 2011 at 9:25 PM, bukzor <workitharder at gmail.com> wrote:

> We've been doing a fair amount of Python scripting, and now we have a
> directory with almost a hundred loosely related scripts. It's
> obviously time to organize this, but there's a problem. These scripts
> import freely from each other and although code reuse is  generally a
> good thing it makes it quite complicated to organize them into
> directories.
>
> There's a few things that you should know about our corporate
> environment:
>
> 1) I don't have access to the users' environment. Editing the
> PYTHONPATH is out, unless it happens in the script itself.
> 2) Users don't install things. Systems are expected to be *already*
> installed and working, so setup.py is not a solution.
>
> I'm quite willing to edit my import statements and do some minor
> refactoring, but the solutions I see currently require me to divide
> all the code strictly between "user runnable scripts" and "libraries",
> which isn't feasible, considering the amount of code.
>
> Has anyone out there solved a similar problem? Are you happy with it?
> --Buck
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110316/f326db5d/attachment-0001.html>