[py-dev] xdist sending info between master and slave - support for client code

[meme dough]

Hi,

> Is it possible to modify xdist so that client code on master can send info
>
> > to the slave.  Will need access to rsync roots before it goes ahead and
> > installs the slave though.
>
> I think we could add a hook called at the master-side which could return/update
> an extra dictionary to be transfered to the client (in theory even a
> pickled-object but i'd like to avoid guaranteeing pickleability for the core
> test distribution mechanism).  A slave-side hook (sessionstart?) would
> get this information.

I created a new hook that gets called just before each test node is installed:

def pytest_testnodeinstall(nodemanager, gateway, slavedata):

Client code on the master can set anything it needs to send to the
slave on the slavedata object.

The slavedata is passed over the channel in the same way as config
object and is stored on the slave node such that any client code on
the slave can access the slavedata.  It can do this in any hook that
the slave client code needs to implement (provided that it can get
access to the session object which is the slave node object which has
the slavedata).


> > Is it possible to modify xdist so that client code on the slave can send
> > info back to the master.  This looks like must modify xdist somehow since it
> > doesn't seem to send any report back at finish.
>
> We can add a hook or re-use an existing one for allowing a client to send
> back (e.g. report) data but the question is: which hook sees this information
> on the master side?
>
> Maybe sessionfinish?  So the slave's sessionfinish hook could add whatever
> extra information and the final sessionfinish sees it.

When the slave node is created it sets a slavereport to an empty
slavereport object.

Client code on the slave can add to the slavereport in any hook that
it implements.

When the slave finishes it already sends a slavefinished event which
now also sends back the slavereport object.

The master TXNode stores the slavereport on itself.

Client code on the master can implement the pytest_testnodedown hook
and access the slavereport.


> Maybe it's better to design and call specific xdist-prefied hooks and not re-use
> existing ones?  As you have hacked on this topic and for your purposes
> already - what do you think?

xdist hooks seem better than the existing ones.  Also many slaves send
reports back on pytest_testnodedown hook and seems better to deal with
just the one immediately rather than build up and pass to
pytest_sessionfinish.

I implemented and if you think it is good enough would you please be
so kind as to pull the changes in and release pytest and xdist.  You
can change of course if you have a better idea!

http://bitbucket.org/memedough/py-trunk/overview

http://bitbucket.org/memedough/pytest-xdist/overview

:)