psphere: how to make thread safe

[jonathan.kinred at gmail.com]

On Sunday, 26 August 2012 22:45:25 UTC+10, jonatha... at gmail.com  wrote:
> On Wednesday, 22 August 2012 22:03:48 UTC+10, sajuptpm  wrote:
> 
> > Hi,
> 
> > 
> 
> > 
> 
> > 
> 
> > psphere: Python interface for the VMware vSphere Web Services SDK
> 
> > 
> 
> > 
> 
> > 
> 
> > I already developed an app using https://bitbucket.org/jkinred/psphere. But getting lot of errors since psphere is not thread safe (I think). So i wrote couple of scripts to test it (See attached files) and found that caching mechanism used by psphere is not thread safe. Could someone please give me some suggestion to make it thread safe.
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > =======Test Code ========
> 
> > 
> 
> > 
> 
> > 
> 
> > import psphere
> 
> > 
> 
> > from psphere.client import Client
> 
> > 
> 
> > from psphere.managedobjects import HostSystem, VirtualMachine, ComputeResource
> 
> > 
> 
> > client = Client("192.168.0.114", "root", "vmware1") ##vCenter
> 
> > 
> 
> > print "\nSucessfully connected to vCenter.\n"
> 
> > 
> 
> > 
> 
> > 
> 
> > from threading import Thread
> 
> > 
> 
> > 
> 
> > 
> 
> > def myfunc(i):
> 
> > 
> 
> > 	host1 = HostSystem.get(client, name="192.168.0.134")
> 
> > 
> 
> > 	host2 = HostSystem.get(client, name="192.168.0.113")
> 
> > 
> 
> > 	print "----i------",i
> 
> > 
> 
> > 	while True:
> 
> > 
> 
> > 		#host1.update(properties=["config", "vm"])
> 
> > 
> 
> > 		#host2.update(properties=["config", "vm"])
> 
> > 
> 
> > 		c = type(host1.config.network)
> 
> > 
> 
> > 		v = type(host2.config.network)
> 
> > 
> 
> > 		for vm in host1.vm:
> 
> > 
> 
> > 			k = vm.config.template
> 
> > 
> 
> > 		for vm in host2.vm:
> 
> > 
> 
> > 			p = vm.config.template
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > for i in range(10):	
> 
> > 
> 
> >     t = Thread(target=myfunc, args=(i,))
> 
> > 
> 
> >     t.start()
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > """
> 
> > 
> 
> > OUTPUT 
> 
> > 
> 
> > =======
> 
> > 
> 
> > Sucessfully connected to vCenter.
> 
> > 
> 
> > 
> 
> > 
> 
> > ----i------ 1
> 
> > 
> 
> > ----i------ 3
> 
> > 
> 
> > ----i------ 5
> 
> > 
> 
> > ----i------ 0
> 
> > 
> 
> >  ----i------ 4
> 
> > 
> 
> > ----i------ 2----i------ 7
> 
> > 
> 
> > ----i------ 
> 
> > 
> 
> > 9
> 
> > 
> 
> > ----i------ 6
> 
> > 
> 
> > ----i------ 8
> 
> > 
> 
> > Exception in thread Thread-4:
> 
> > 
> 
> > Traceback (most recent call last):
> 
> > 
> 
> >   File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
> 
> > 
> 
> >     self.run()
> 
> > 
> 
> >   File "/usr/lib/python2.7/threading.py", line 504, in run
> 
> > 
> 
> >     self.__target(*self.__args, **self.__kwargs)
> 
> > 
> 
> >   File "vcenter_test1.py", line 19, in myfunc
> 
> > 
> 
> >     k = vm.config.template
> 
> > 
> 
> >   File "/home/saju/cvt/trunk/src/cvt/web/cvt/psphere/__init__.py", line 79, in __get__
> 
> > 
> 
> >     value = self.fget(inst)
> 
> > 
> 
> >   File "/home/saju/cvt/trunk/src/cvt/web/cvt/psphere/managedobjects.py", line 1236, in config
> 
> > 
> 
> >     return self._get_dataobject("config", False)
> 
> > 
> 
> >   File "/home/saju/cvt/trunk/src/cvt/web/cvt/psphere/__init__.py", line 116, in _get_dataobject
> 
> > 
> 
> >     return self._cache[name][0]
> 
> > 
> 
> > KeyError: 'config'
> 
> > 
> 
> > 
> 
> > 
> 
> > Exception in thread Thread-6:
> 
> > 
> 
> > Traceback (most recent call last):
> 
> > 
> 
> >   File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
> 
> > 
> 
> >     self.run()
> 
> > 
> 
> >   File "/usr/lib/python2.7/threading.py", line 504, in run
> 
> > 
> 
> >     self.__target(*self.__args, **self.__kwargs)
> 
> > 
> 
> >   File "vcenter_test1.py", line 17, in myfunc
> 
> > 
> 
> >     v = type(host2.config.network)
> 
> > 
> 
> > AttributeError: VirtualMachineConfigInfo instance has no attribute 'network'
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > 
> 
> > """
> 
> 
> 
> I'm the author of psphere. I'd be really interested to get some experienced insight into this as I really don't know the first thing about thread safety.
> 
> 
> 
> psphere does use a cache, but I'm confused as to how the cache would be thread unsafe as the cache is just a property of an object.
> 
> 
> 
> What I'm suspecting (and this only came to me while writing this post) is that I have a "client" object which is passed into each object and used to retrieve data from the server. See this gist:
> 
> https://gist.github.com/3478641
> 
> 
> 
> When used in a threaded scenario could the client be used by multiple objects simultaneously and the replies would come in out of order?
> 
> 
> 
> A couple of years ago I started psphere to learn Python and I always hoped that I would have an experienced developer join me. I did learn Python but I never got that experienced developer so when it comes to issues like this I'm definitely out of my depth! Please if someone could spend the time with me I would really appreciate it!

Just to clarify, the real Client is here:
https://github.com/jkinred/psphere/blob/master/psphere/client.py#L45