[Neuroimaging] CZI grant - what would you like to see in Nibabel?

[Emanuele Olivetti]

Hi Matthew,

My colleagues and I would like to have a (much) faster loader of
tractography data with a smaller memory footprint, especially for TRK
(Trackvis) files. Just to give you an idea - which, yes, it's an extreme
case but definitely not so infrequent for us - when loading a very large
tractogram of 10 million streamlines it takes:
- 15 minutes and approximately 10Gb of RAM with Nibabel
- 1 minute and approximately 5Gb of RAM with our custom TRK loader (it
works but code style is meh,
https://github.com/FBK-NILab/nilab/blob/master/load_trk.py)
- 6 seconds and 3.5Gb of RAM if we previously save the tractogram as numpy
array (numpy.savez()) and then load it with numpy.load(). But in this case,
the streamlines are resampled to a fixed number of points.

Thank you for all the work done!

Emanuele


On Thu, Jul 30, 2020 at 5:01 PM Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> Although this is pushing things close the wire, I am currently trying
> to work out whether to apply for a Chan Zuckerberg grant for Nibabel:
>
>
> https://chanzuckerberg.com/wp-content/uploads/2020/06/EOSS-3-RFA-Announcement-and-Instructions-CYCLE-3-COMBINED-1.pdf
>
> Before we do that, I wanted to ask - if y'all could get anything into
> Nibabel - what
> would it be?
>
> So far, in various conversations, Chris Markiewicz and I came up with:
>
> * More comprehensive DICOM to NIfTI etc support
> * Labelled axes for images, maybe via xarray
> * Integrated API for surface formats [1]
> * HDF5 serialization for many image formats
> * Ability to read images directly from URLs, read / write images to /
> from AWS, GCS, Azure data storage addresses [2]
> * Optimized visualization for web and desktop
>
> Is there anything else you'd like to see?
>
> See you,
>
> Matthew
>
> [1] https://github.com/nipy/nibabel/issues/936
> [2] c.f.
> https://pandas.pydata.org/pandas-docs/version/1.1.0/whatsnew/v1.1.0.html#fsspec-now-used-for-filesystem-handling
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>


-- 
Emanuele Olivetti, Ph.D.
Senior Research Scientist at Bruno Kessler Foundation (FBK-ICT)
NeuroInformatics Laboratory (NILab) http://nilab.fbk.eu
Center for Mind and Brain Sciences (CIMeC), University of Trento
Via delle Regole 101 - 38123 Mattarello (Trento), ITALY
olivetti at fbk.eu - +39 0461314179
emanuele.olivetti at unitn.it - +39 0461282760

-- 
_FBK vi invita a leggere il suo Piano di rientro 
<https://trasparenza.fbk.eu/COVID-19-comunicazioni-del-Datore-di-Lavoro-raccomandazioni-e-altro/Piano-di-rientro-FBK/Piano-di-rientro-FBK> 
| FBK invites you to read its Premises Reopening Plan 
<https://trasparenza.fbk.eu/COVID-19-comunicazioni-del-Datore-di-Lavoro-raccomandazioni-e-altro/Piano-di-rientro-FBK/English-version_FBK-Reopening-Plan>._



--
Le informazioni contenute nella presente comunicazione sono di natura 
privata e come tali sono da considerarsi riservate ed indirizzate 
esclusivamente ai destinatari indicati e per le finalità strettamente 
legate al relativo contenuto. Se avete ricevuto questo messaggio per 
errore, vi preghiamo di eliminarlo e di inviare una comunicazione 
all’indirizzo e-mail del mittente.

--
The information transmitted is 
intended only for the person or entity to which it is addressed and may 
contain confidential and/or privileged material. If you received this in 
error, please contact the sender and delete the material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20200730/b1b7cef9/attachment.html>