[py-dev] background/future plans

[holger krekel]

Hi everbody, 

here is another mail to give and discuss some more background, 
part of which some people already know about, other's don't.  

First of all, there is a pending renaming from 'std' to 'py' 
accompanied by some subsequent small renamings.  The rationale is 

- speaking of the 'std' lib is confusing for python-dev and
  other pythonistas

- the name 'py' is pretty indicative and 'py' does support - like
  stated at EuroPython 2004 - the notion of beeing a 
  "complementary second standard library". 

- command line tools provided by 'py' will have a prefix of 
  'py.*' like e.g. 'py.test' so that <tab>-completion works
  nicely with 'py.<tab>' both inside bash/readline as well 
  as on the interactive python prompt. (i plan to enhance 
  and integrate rlcompleter2 at some point, see 
  http://codespeak.net/rlcompleter2)

I'd like to do the renaming this weekend but am not sure
yet how to do it.  "Bicycle Repair Man" seems to be completly
lacking any documentation and i know of know of no other
tool to support such a grand scale renaming except writing
my own little script.  Also 'utest' is probably to become
simply 'py.test' being a python namespace and the name 
of the central script. 

std-soon-to-be-py uses a strict mechanism to control the
export of its names. In std/__init__.py you'll currently find
all the namespaces and names listed which are exposed via the
import mechanism. This allows the library to be easily
refactorable even across major releases while retaining
automatically tested backward compatibility.  

The idea is that - unless something gets deprecated - such 
"import APIs" are guaranteed across major releases.  If it 
works in X.Y it will work in X.(Y+1). If it is deprecated in 
X.Y then it may go away in X.(Y+1).  The behaviour and the 
names _indirectly_ reachable through classes are only 
guaranteed through the tests.  So if you wan't to make sure 
that a certain behaviour lasts across releases you'd better 
write a test for it and check it in/submit it. Any untested 
API part could go away. 

OK, then on with some short introduction to the current 
py components - already using the proposed naming scheme: 

    py    the 'py lib hook' and root namespace. It's always enough
          to just "import py" and work your way with e.g.
          'py.path.svnurl(...)'.  Packages and modules will be 
          lazily imported. All namespaces on the way should 
          be clean and should have no garbage. (do a 

    .path.local  the local filesystem class providing an  
                 object oriented OO API for filesystem accesses 

    .path.svnurl the remote subversion URL class providing 
                 listdir(), update(), remove(), ... 
                 methods for accesses to the transactional 
                 subversion filesystem 

    .path.svnwc  a local working copy API offering most of what
                 path.local offers plus commit(), revert(), add() 
                 etc.pp. 

    .test.main()         the commandline API for starting tests 

    .test.assert_raises  the one method besides the plain 'assert' 
                         statement that you usually want from a  
                         testing module 

    .test.compat.*       some unittest.py compatibility stuff like
                         TestCase 

    .test.*              various stuff that shouldn't maybe be exposed :-) 

    .magic.dyncode       a module which helps with dynamic code generation
                         allowing for better tracebacks and some more 
                         convenience 
    .magic.invoke/revoke the mechanism to invoke magic which usually 
                         involves (careful) monkey-patching of system 
                         libraries like inspect or the builtin namespace

    .execnet   offering an innovative approach to networked applications 
               by allowing them to execute code in each other's process 
               space.  Currently, this works for sockets and pipes/child
               processes, offering a 'SSHGateway' should be no problem 
               as it is basically the same as the "PopenGateway". 

               The small code in 'execnet' is an extract from a hacking
               week with Armin on a <inhale>multiuser multiplatform multi-line 
               concurrent pygame-based interactive python shell named shpy</inhale>. 
               (also in the codespeak repo). 

As you probably have noticed there are all kinds of little open ends and 
food for thought.  More than anything, 'the py lib' should allow for
a much faster pace and refactoring than the current python lib.  Thus I 
find it most important to agree on mechanisms for making this possible 
mainly "export" control and tests-are-the-only-behaviour-guaranties 
at the moment.  Again, the testing mechanism plays a central role. 

cheers, 

    holger