[Neuroimaging] Thread-safe ArrayProxy/fileslice?
Nathaniel Smith
njs at pobox.com
Wed Feb 8 15:23:00 EST 2017
The best solution would be to use a pread-style interface that lets you
read from a given offset without seeking at all, and thus without needing
any locks. Unfortunately, Python hasn't historically had the best support
for this, but it's available if you're using Python 3 on unix:
https://docs.python.org/3/library/os.html#os.pread
-n
On Feb 8, 2017 5:55 AM, "paul mccarthy" <pauldmccarthy at gmail.com> wrote:
> Howdy all,
>
> Does anybody have experience accessing image data through the ArrayProxy
> class (or functions in the fileslice module) in a multi-threaded
> environment? I am visualising large 4D images which are kept on disk (via
> my indexed_gzip module), and am having trouble when accessing the data
> from multiple threads, as the seek/read pairs from different threads will
> occasionally become intertwined with each other.
>
>
> My hacky workaround is to patch the ArrayProxy.__getitem__ method, and
> add a threading.Lock to each instance, as follows:
>
>
> import threading
>
> import nibabel as nib
> import nibabel.arrayproxy as ap
>
> def ArrayProxy__getitem__(self, slc):
>
> if not hasattr(self, '_thread_lock'):
> return self.__real_getitem__(slc)
>
> self._thread_lock.acquire()
>
> try:
> return ap.ArrayProxy.__real_getitem__(self, slc)
>
> finally:
> self._thread_lock.release()
>
> # Patch ArrayProxy.__getitem__
> ap.ArrayProxy.__real_getitem__ = ap.ArrayProxy.__getitem__
> ap.ArrayProxy.__getitem__ = ArrayProxy__getitem__
>
>
> # Then add a lock to instances
> # which need to be thread-safe
> img = nib.load('MNI152_T1_2mm.nii.gz')
> img.dataobj._thread_lock = threading.Lock()
>
>
> This is the first thing I came up with, although I will probably end up
> adding the lock to the fileobj, and patching the fileslice.fileslice
> function instead. Unless there are any better ideas?
>
> Cheers,
>
> Paul
>
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20170208/6d031bc0/attachment.html>
More information about the Neuroimaging
mailing list