From christine.s.w.law at gmail.com Mon Nov 1 21:22:51 2021 From: christine.s.w.law at gmail.com (Christine Law) Date: Mon, 1 Nov 2021 18:22:51 -0700 Subject: [Neuroimaging] Stanford Postdoc opening Message-ID: *Postdoctoral Fellow Position in Neuroimaging based Biomarkers of Chronic Pain* *DESCRIPTION* The Systems Neuroscience & Pain Laboratory at Stanford University (SNAPL) is actively recruiting a postdoctoral fellow who will join our research project on chronic pain. Funded by the National Institute of Health and directed by Professor Sean Mackey, in collaboration with an internationally renowned group of collaborators from the Departments of Computer Sciences, Sleep Medicine, Genetics, Neurosciences and others, our goal is to develop and validate objective diagnostic and prognostic biomarkers of chronic pain. The focus of work for this postdoc will be on the neuroimaging of people with chronic pain using structural and functional MRI techniques. Our plan is to characterize neurobiological, multiomic, sensory testing, sleep/function actigraphy and other mechanisms underlying chronic pain and use this information to develop objective biomarkers of pain, ultimately, to derive new personalized interventions. When you join, you will have opportunity to collaborate with top neuroscientists, psychologists, machine learning experts, sleep medicine researchers, genetics experts and pain specialists at Stanford. Additionally, you will have access to cutting-edge fMRI technology; to invent and develop your own related research. Multidisciplinary aspects of this project will allow you to explore diverse chronic pain topics ranging from biomarker design for degenerative conditions to heart rate variability as a simple biomarker for psychological disorders. We have a strong track record for successfully transitioning postdoctoral fellows to independent grant funding and faculty positions. We can offer NIH T32 training to select candidates. The ideal candidate is a motivated problem solver and innovator who has a neuroscience background, enjoys challenging the paradigms of contemporary research and experimentation, is proficient with computer-aided analysis, and is enthusiastic and passionate about fMRI acquisition. *RESEARCH AREA* ? fMRI and behavioral data from Healthy Control and patients with musculoskeletal pain, ? Multivariate pattern analysis of spinal cord/brain data for corticospinal biomarker, ? Integration of multimodal biomarkers into composite predictive signatures, *QUALIFICATION* Applicants must have received, as of the beginning date of the appointment, an MD/PhD, MD or PhD or comparable doctoral degree from an accredited domestic or foreign institution. Experience in neuroimaging analysis is a must. Additional experience in any of these is a plus: 1. Cognitive / Affective Neuroscience 2. Computational modeling / Machine Learning / Neural Networks / ICA 3. Chronic pain 4. MATLAB / Python / R / Linux / C 5. SPM / AFNI / FSL / fMRIPrep *APPLICATION MATERIALS* Submit (1) CV, (2) NIH Biosketch, (3) Letter of Research Intent to our website: https://med.stanford.edu/pain/snapl/research-positions/postdoc-openings.html Find instructions, blank format pages, and sample biosketches here: biosketch formats . Applicants follow non-fellowship templates. Letter of Intent template is available here . For more information, visit http://snapl.stanford.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.hoechenberger at gmail.com Thu Nov 4 08:28:49 2021 From: richard.hoechenberger at gmail.com (=?UTF-8?Q?Richard_H=C3=B6chenberger?=) Date: Thu, 4 Nov 2021 13:28:49 +0100 Subject: [Neuroimaging] [ANN] MNE-Python 0.24 Message-ID: Hello everyone, We are very pleased to announce the new v0.24 release of MNE-Python! [image: :balloon:][image: :balloon:][image: :balloon:] This release is the culmination of more than 6 months of work by 45 contributors, 13 of whom contributed for the very first time ? meaning almost one third of contributors were first-timers! Thanks so much to all of you! [image: :pray:] We are extremely excited to share all the amazing things we?ve been working on. Google Summer of Code The MNE-Python team supervised three students in this year?s Google Summer of Code (GSoC) . We are thrilled to announce that all of our students successfully finished their projects, greatly enhancing the MNE-Python user experience: - Adam Li produced the MNE-Connectivity package for advanced connectivity analysis with MNE-Python. - Alex Rockhill created a new GUI for the localization of intracranial electrodes on CT and MRI scans, available via mne.gui.locate_ieeg() . - Martin Schulz reimagined the time-series data browser and created the mne-qt-browser package that will redefine your experience of visual data inspection: faster, smoother, better. Adam, Alex, and Martin: We had a blast working with you! Thank you for your effort, endurance, and persistence! Summary of the most notable changes - The coregistration GUI has been rebuilt from the ground up using a state-of-the-art technology stack. It was tedious and difficult work, believe us, but it was worth it! What changes for you? The GUI should be more responsive, easier to use, and work more reliably on all platforms! [image: :raised_hands:] - Working with intracranial recordings? A new GUI, mne.gui.locate_ieeg(), helps to localize intracranial electrodes with CT and MR scans! [image: :brain:] - Ever heard of MNE reports? These single-file HTML documents can help you track an entire analysis from beginning to end, and share the results with your coworkers. With this release, reports are easier to use and looking better than ever ! [image: :star_struck:] - Often find yourself creating separate evokeds for the different event types in your epochs? Well, we just made your life a bit easier! Epochs.average() now has a by_event_type parameter to automatically produce evokeds for each event type ? no need to write a loop anymore! [image: :relieved:] - A new, modern, good-looking and fast way to visualize time-series data has been added by Martin Schulz via the new (optional) mne-qt-browser package. We are certain it will redefine your experience of visual data inspection in MNE-Python! Our selection of highlights Preprocessing, epochs, and evokeds - Acquired some data with breaks [image: :coffee:] between experimental blocks, but the recording kept on running? These breaks can now be automatically annotated via mne.preprocessing.annotate_break(). - The figures produced by ICA.plot_sources() for evoked data now use different colors and line styles to make it easier to tell components apart. [image: :bar_chart:] - We have added cortical signal suppression (CSS) via mne.preprocessing.cortical_signal_suppression(). [image: :no_bell:] - Epochs.average() now has a by_event_type parameter to generate evokeds for each event type. [image: :sunglasses:] - We know you missed it: Evoked objects now have a get_data() method, just like Raw and Epochs! You?re welcome. [image: :pray:] Source estimation - The coregistration GUI has been rebuilt from the ground up and no longer requires Mayavi. [image: :hammer_and_wrench:] - We?ve added a new function, mne.bem.make_scalp_surfaces(), to compute the scalp surfaces required for coregistration. Previously, this could only be done on the command line. [image: :skull:] - mne.inverse_sparse.mixed_norm now supports the SURE method to automate setting of the regularization parameter. [image: :chart_with_upwards_trend:] iEEG - There is a new GUI, mne.gui.locate_ieeg(), to locate intracranial electrode contacts from a CT and anatomical MRI scans. This is described in a new tutorial . [image: :brain:] fNIRS - Oxy- and deoxyhemoglobin data types can now be interpolated. [image: :drop_of_blood:] - NIRSport, NIRSport2, and data acquired with NIRx devices using Aurora software version 2021.9 can now be read. [image: :open_book:] Reports - We completely revamped the report experience: new HTML layout, many new methods, more flexibility! The functionality is demonstrated in the respective tutorial . [image: :closed_book:] File I/O - We?ve added support for exporting evoked MFF and raw EDF+ files. [image: :articulated_lorry:] - All functions for reading and writing files should now automatically handle ~ (the tilde character) and expand it to the user?s home directory. Saves you Linux and macOS folks some typing! [image: :keyboard:] Dependencies - pooch is now required to download the included datasets. [image: :open_file_folder:] API enhancements, changes, and deprecations - All get_data() methods now have tmin and tmax parameters to specify which time period to get. No more manual fiddling with those pesky indices! [image: :stopwatch:] - You should not modify the Info object manually as this can have unexpected consequences. We?re now starting to enforce this and produce warnings if you don?t follow our advice ? And in a future release of MNE-Python, these warnings will change into exceptions. So please pay attention to them! And do let us know if you need help avoiding them. - We will remove Mayavi and PySurfer 3D plotting backends in the next version. [image: :wastebasket:] - The mne kit2fiff GUI has been split off into the separate mne-kit-gui package. [image: :package:] - The mne.gui.fiducials() GUI has been deprecated, use mne coreg or mne_kit_gui.fiducials() to set MRI fiducials instead. [image: :wastebasket:] Full changelog For a full list of improvements and API changes, see the changelog . Installing the new release Since quite a few things ? including dependencies ? have changed, we recommend creating a new environment with a ?fresh? installation. Please follow the installation instructions on our website. Feedback As usual, we welcome your bug reports, feature requests, critiques, and contributions. Development takes place on GitHub. If you would like to contribute, star ? the project, or just take a peek at the code, visit mne-tools/mne-python . Staying up to date You may follow us on Twitter at @mne_news Getting help The MNE-Python forum is an excellent way to ask your questions and to share your knowledge with others. We also announce our bi-weekly online office hours there. And of course, we provide a large body of documentation, tutorials, and examples on our website, mne.tools. Closing remarks We hope you will enjoy the new features and the many, many small improvements we have added, and are looking forward to receiving your feedback. We will also start highlighting and demonstrating some of the new functionality in brief tutorials and video clips shortly, so be sure to follow us on Twitter to not miss any of those! Stay safe and take care! The MNE-Python developers Contributors MNE-Python is a community-driven project. We are always very happy to welcome new contributors of code and documentation! 45 people contributed to this release, and 13 of them (29%) contributed for the very first time ? thank you so much for your time and effort, we truly appreciate it! This is the list of contributors, in alphabetical order. First-time contributors - Darin Erat Sleiter - Dominique Makowski - Evgeny Goldstein - Jan Sosulski - John Samuelsson - Joshua Teves - Marian Dovgialo - Mathieu Scheltienne - Pierre-Antoine Bannier - Reza Shoorangiz - Romain Derollepot - Timothy Gates - Xiaokai Xia All other contributors - Adam Li - Alex Rockhill - Alexandre Gramfort - Britta Westner - Clemens Brunner - Daniel McCloy - David Julien - Denis A. Engemann - Eric Larson - Erica Peterson - Evan Hathaway - Fede Raimondo - Gansheng Tan - Guillaume Favelier - Hubert Banville - Jean-Remi King - Jeff Stout - Johann Benerradi - Josh Koen - Lukas Hecker - Luke Bloy - Marijn van Vliet - Martin Schulz - Miko?aj Magnuski - Richard H?chenberger - Riessarius Stargardsky - Robert Luke - Ross Maddox - Ryan Law - Stefan Appelhoff - Steve Matindi - Valerii Chirkov [image: :heart:] [image: :heart:] [image: :heart:] -------------- next part -------------- An HTML attachment was scrubbed... URL: From bhg-donostia at bcbl.eu Mon Nov 8 07:05:08 2021 From: bhg-donostia at bcbl.eu (bhg-donostia at bcbl.eu) Date: Mon, 8 Nov 2021 13:05:08 +0100 (CET) Subject: [Neuroimaging] Brainhack Donostia 2021 Message-ID: <1798611441.3935066.1636373108352.JavaMail.zimbra@bcbl.eu> To all students, researchers, and science fans, At the Basque Center on Cognition, Brain and Language (BCBL), we are organizing the fourth edition of Brainhack Donostia ( [ https://brainhack-donostia.github.io/ | https://brainhack-donostia.github.io/ ] ), an event focused on neuroscience and the promotion of open-source resources in an accessible way across disciplines and experience levels. This year?s edition will take place from November 22 to November 24, 2021 online. During the three-day online workshop, there will be a hackathon in which you can propose, develop and collaborate on neuroscience-related projects (e.g., data acquisition, visualization, etc.). Being the hackathon the core of the event, all participants are expected to take part in at least one project. Check the project page for more info: [ https://brainhack-donostia.github.io/projects/ | https://brainhack-donostia.github.io/projects/ ] . Also, you will be able to attend the keynote lectures of our eminent invited speakers. They will share their experience on state-of-the-art methods, good practices in science and current hot topics in research. To know more about the talks, check the schedule here: [ https://brainhack-donostia.github.io/schedule/ | https://brainhack-donostia.github.io/schedule/ ] . Follow us on Twitter ( [ https://twitter.com/bhdonostia?lang=en | @BHDonostia ] ) to stay updated. Please visit [ https://brainhack-donostia.github.io/registration/ | https://brainhack-donostia.github.io/registration/ ] to register. This year?s registration is free! Feel free to spread the wor d among those who could be potentially interested in this event. Sharing is caring! For any inquiries, please email us at info.bhg-donostia at bcbl.eu . Looking forward to meeting you in November! -- Brainhack Donostia Organising Team https://brainhack-donostia.github.io/ https://www.bcbl.eu -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Tue Nov 9 06:32:04 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 11:32:04 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club Message-ID: Hi, Summary: we are planning to do a journal club to discuss new neuroimaging data formats. Please let me (Matthew) know if you want to be included in the scheduling for the journal club. Of course, once we've got a time, anyone can join the journal club Zoom. Detail: As part of the Chan-Zuckerberg grant work, we are thinking about new formats for using and sharing neuroimaging data. As you can see in the grant documents: https://github.com/matthew-brett/czi-nibabel we've been thinking about HDF5, but further research (e.g. [1]) has given us some pause for reflection. Meanwhile, the astronomy community has done some serious thinking about this, and written this up in a paper proposing a new astronomy format: Greenfield, M. Droettboom, E. Bray, ASDF: A new data format for astronomy, Astronomy and Computing, Volume 12, 2015, Pages 240-251, ISSN 2213-1337, https://doi.org/10.1016/j.ascom.2015.06.004. (https://www.sciencedirect.com/science/article/pii/S2213133715000645) Both the discussion and the format seem like a very good starting point for our own thinking about data formats, so we'd like to have a journal club to discuss it. I (Matthew) am going to organize this via Calendly, and I've made a Caldendly account for Nipy-related stuff, but, if you want to have a say in when we do the journal club, please let me know so I can add you to that account. I'll publish the Calendly link in a day or so. Cheers, Matthew [1] https://cyrille.rossant.net/moving-away-hdf5/ From keveen.rodriguez at udea.edu.co Tue Nov 9 08:23:50 2021 From: keveen.rodriguez at udea.edu.co (KEVEEN RODRIGUEZ ZAPATA) Date: Tue, 9 Nov 2021 08:23:50 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hello, thanks for the initiative Matthew, I would like to be part of the schedule. On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett wrote: > Hi, > > Summary: we are planning to do a journal club to discuss new > neuroimaging data formats. Please let me (Matthew) know if you want > to be included in the scheduling for the journal club. Of course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > we've been thinking about HDF5, but further research (e.g. [1]) has > given us some pause for reflection. Meanwhile, the astronomy > community has done some serious thinking about this, and written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004. > (https://www.sciencedirect.com/science/article/pii/S2213133715000645) > > Both the discussion and the format seem like a very good starting > point for our own thinking about data formats, so we'd like to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've made a > Caldendly account for Nipy-related stuff, but, if you want to have a > say in when we do the journal club, please let me know so I can add > you to that account. I'll publish the Calendly link in a day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -- Keveen Rodr?guez Zapata B.Eng. Bioengineer Msc. Student in Biomedical Sciences: Neuroscience and Bsc. Computer Science GRUNECO Universidad de Antioquia -- "La informaci?n aqu? contenida es para uso exclusivo de la persona o entidad de destino. Est? estrictamente prohibida su utilizaci?n, copia, descarga, distribuci?n, modificaci?n y/o reproducci?n total o parcial, sin el permiso expreso de Universidad de Antioquia, pues su contenido puede ser de car?cter confidencial y/o contener material privilegiado. Si usted recibi? esta informaci?n por error, por favor contacte en forma inmediata a quien la envi? y borre este material de su computador.?Universidad de Antioquia?no es?responsable por la informaci?n contenida en esta comunicaci?n, el directo responsable es quien la firma o el autor de la misma." -- UdeA -------------- next part -------------- An HTML attachment was scrubbed... URL: From andersonwinkler at gmail.com Tue Nov 9 08:47:02 2021 From: andersonwinkler at gmail.com (Anderson M. Winkler) Date: Tue, 9 Nov 2021 08:47:02 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi Matthew, hi all, I put some thinking on this a few years ago and concluded that HDF5 would be ideal. The blog post you link was already up and I didn't feel deterred by it, despite the interesting insights. At the time I put online a document for discussion ( https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing) and as a starting point, made a mapping to how we'd store NIFTI as HDF5 ( https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing ). At the end, I had to focus on other things, including career (and country) moves, and couldn't spend time and effort on this, despite the fact that the needs from then continue very much the same as today (if not perhaps even more needed now than then...). I'd be happy to participate in any effort you or others may lead towards this. Cheers, Anderson On Tue, Nov 9, 2021 at 8:24 AM KEVEEN RODRIGUEZ ZAPATA < keveen.rodriguez at udea.edu.co> wrote: > Hello, thanks for the initiative Matthew, I would like to be part of the > schedule. > > On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett > wrote: > >> Hi, >> >> Summary: we are planning to do a journal club to discuss new >> neuroimaging data formats. Please let me (Matthew) know if you want >> to be included in the scheduling for the journal club. Of course, >> once we've got a time, anyone can join the journal club Zoom. >> >> Detail: >> >> As part of the Chan-Zuckerberg grant work, we are thinking about new >> formats for using and sharing neuroimaging data. >> >> As you can see in the grant documents: >> >> https://github.com/matthew-brett/czi-nibabel >> >> we've been thinking about HDF5, but further research (e.g. [1]) has >> given us some pause for reflection. Meanwhile, the astronomy >> community has done some serious thinking about this, and written this >> up in a paper proposing a new astronomy format: >> >> Greenfield, M. Droettboom, E. Bray, >> ASDF: A new data format for astronomy, >> Astronomy and Computing, >> Volume 12, >> 2015, >> Pages 240-251, >> ISSN 2213-1337, >> https://doi.org/10.1016/j.ascom.2015.06.004. >> (https://www.sciencedirect.com/science/article/pii/S2213133715000645) >> >> Both the discussion and the format seem like a very good starting >> point for our own thinking about data formats, so we'd like to have a >> journal club to discuss it. >> >> I (Matthew) am going to organize this via Calendly, and I've made a >> Caldendly account for Nipy-related stuff, but, if you want to have a >> say in when we do the journal club, please let me know so I can add >> you to that account. I'll publish the Calendly link in a day or so. >> >> Cheers, >> >> Matthew >> >> [1] https://cyrille.rossant.net/moving-away-hdf5/ >> _______________________________________________ >> Neuroimaging mailing list >> Neuroimaging at python.org >> https://mail.python.org/mailman/listinfo/neuroimaging >> > -- > > Keveen Rodr?guez Zapata > B.Eng. Bioengineer > Msc. Student in Biomedical Sciences: Neuroscience and Bsc. Computer Science > GRUNECO > Universidad de Antioquia > > > > "La informaci?n aqu? contenida es para uso exclusivo de la persona o > entidad de destino. Est? estrictamente prohibida su utilizaci?n, copia, > descarga, distribuci?n, modificaci?n y/o reproducci?n total o parcial, sin > el permiso expreso de Universidad de Antioquia, pues su contenido puede ser > de car?cter confidencial y/o contener material privilegiado. Si usted > recibi? esta informaci?n por error, por favor contacte en forma inmediata a > quien la envi? y borre este material de su computador. Universidad de > Antioquia no es responsable por la informaci?n contenida en esta > comunicaci?n, el directo responsable es quien la firma o el autor de la > misma." > UdeA_______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbpoline at gmail.com Tue Nov 9 09:16:04 2021 From: jbpoline at gmail.com (JB Poline) Date: Tue, 9 Nov 2021 09:16:04 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, Another thing that could be useful would be to have a look at the NWB folks experience - they have gone through quite a few pain points - we ask one of the developer to give a little talk / summary of their experience I'd be interested as well Cheers JB On Tue, Nov 9, 2021 at 8:48 AM Anderson M. Winkler < andersonwinkler at gmail.com> wrote: > Hi Matthew, hi all, > > I put some thinking on this a few years ago and concluded that HDF5 would > be ideal. The blog post you link was already up and I didn't feel deterred > by it, despite the interesting insights. > > At the time I put online a document for discussion ( > https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing) > and as a starting point, made a mapping to how we'd store NIFTI as HDF5 ( > https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing > ). > > At the end, I had to focus on other things, including career (and country) > moves, and couldn't spend time and effort on this, despite the fact that > the needs from then continue very much the same as today (if not perhaps > even more needed now than then...). > > I'd be happy to participate in any effort you or others may lead towards > this. > > Cheers, > > Anderson > > > On Tue, Nov 9, 2021 at 8:24 AM KEVEEN RODRIGUEZ ZAPATA < > keveen.rodriguez at udea.edu.co> wrote: > >> Hello, thanks for the initiative Matthew, I would like to be part of the >> schedule. >> >> On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett >> wrote: >> >>> Hi, >>> >>> Summary: we are planning to do a journal club to discuss new >>> neuroimaging data formats. Please let me (Matthew) know if you want >>> to be included in the scheduling for the journal club. Of course, >>> once we've got a time, anyone can join the journal club Zoom. >>> >>> Detail: >>> >>> As part of the Chan-Zuckerberg grant work, we are thinking about new >>> formats for using and sharing neuroimaging data. >>> >>> As you can see in the grant documents: >>> >>> https://github.com/matthew-brett/czi-nibabel >>> >>> we've been thinking about HDF5, but further research (e.g. [1]) has >>> given us some pause for reflection. Meanwhile, the astronomy >>> community has done some serious thinking about this, and written this >>> up in a paper proposing a new astronomy format: >>> >>> Greenfield, M. Droettboom, E. Bray, >>> ASDF: A new data format for astronomy, >>> Astronomy and Computing, >>> Volume 12, >>> 2015, >>> Pages 240-251, >>> ISSN 2213-1337, >>> https://doi.org/10.1016/j.ascom.2015.06.004. >>> (https://www.sciencedirect.com/science/article/pii/S2213133715000645) >>> >>> Both the discussion and the format seem like a very good starting >>> point for our own thinking about data formats, so we'd like to have a >>> journal club to discuss it. >>> >>> I (Matthew) am going to organize this via Calendly, and I've made a >>> Caldendly account for Nipy-related stuff, but, if you want to have a >>> say in when we do the journal club, please let me know so I can add >>> you to that account. I'll publish the Calendly link in a day or so. >>> >>> Cheers, >>> >>> Matthew >>> >>> [1] https://cyrille.rossant.net/moving-away-hdf5/ >>> _______________________________________________ >>> Neuroimaging mailing list >>> Neuroimaging at python.org >>> https://mail.python.org/mailman/listinfo/neuroimaging >>> >> -- >> >> Keveen Rodr?guez Zapata >> B.Eng. Bioengineer >> Msc. Student in Biomedical Sciences: Neuroscience and Bsc. Computer >> Science >> GRUNECO >> Universidad de Antioquia >> >> >> >> "La informaci?n aqu? contenida es para uso exclusivo de la persona o >> entidad de destino. Est? estrictamente prohibida su utilizaci?n, copia, >> descarga, distribuci?n, modificaci?n y/o reproducci?n total o parcial, sin >> el permiso expreso de Universidad de Antioquia, pues su contenido puede ser >> de car?cter confidencial y/o contener material privilegiado. Si usted >> recibi? esta informaci?n por error, por favor contacte en forma inmediata a >> quien la envi? y borre este material de su computador. Universidad de >> Antioquia no es responsable por la informaci?n contenida en esta >> comunicaci?n, el directo responsable es quien la firma o el autor de la >> misma." >> UdeA_______________________________________________ >> Neuroimaging mailing list >> Neuroimaging at python.org >> https://mail.python.org/mailman/listinfo/neuroimaging >> > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Tue Nov 9 09:32:24 2021 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 9 Nov 2021 09:32:24 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: hi, i can help provide a data archiving, metadata, and cloud/hpc access perspective. we are using hdf5 quite a bit both through nwb and otherwise at present, and are also using zarr, a cloud native format. i would not write off hdf5 and instead ask which use cases in what physical environments would run you into trouble. while interested in attending, i would suggest scheduling without me, and will join if i can or comment on a doc as necessary. cheers, satra On Tue, Nov 9, 2021 at 9:16 AM JB Poline wrote: > Hi, > > Another thing that could be useful would be to have a look at the NWB > folks experience - they have gone through quite a few pain points - we ask > one of the developer to give a little talk / summary of their experience > > I'd be interested as well > Cheers > JB > > On Tue, Nov 9, 2021 at 8:48 AM Anderson M. Winkler < > andersonwinkler at gmail.com> wrote: > >> Hi Matthew, hi all, >> >> I put some thinking on this a few years ago and concluded that HDF5 would >> be ideal. The blog post you link was already up and I didn't feel deterred >> by it, despite the interesting insights. >> >> At the time I put online a document for discussion ( >> https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing) >> and as a starting point, made a mapping to how we'd store NIFTI as HDF5 ( >> https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing >> ). >> >> At the end, I had to focus on other things, including career (and >> country) moves, and couldn't spend time and effort on this, despite the >> fact that the needs from then continue very much the same as today (if not >> perhaps even more needed now than then...). >> >> I'd be happy to participate in any effort you or others may lead towards >> this. >> >> Cheers, >> >> Anderson >> >> >> On Tue, Nov 9, 2021 at 8:24 AM KEVEEN RODRIGUEZ ZAPATA < >> keveen.rodriguez at udea.edu.co> wrote: >> >>> Hello, thanks for the initiative Matthew, I would like to be part of the >>> schedule. >>> >>> On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett >>> wrote: >>> >>>> Hi, >>>> >>>> Summary: we are planning to do a journal club to discuss new >>>> neuroimaging data formats. Please let me (Matthew) know if you want >>>> to be included in the scheduling for the journal club. Of course, >>>> once we've got a time, anyone can join the journal club Zoom. >>>> >>>> Detail: >>>> >>>> As part of the Chan-Zuckerberg grant work, we are thinking about new >>>> formats for using and sharing neuroimaging data. >>>> >>>> As you can see in the grant documents: >>>> >>>> https://github.com/matthew-brett/czi-nibabel >>>> >>>> we've been thinking about HDF5, but further research (e.g. [1]) has >>>> given us some pause for reflection. Meanwhile, the astronomy >>>> community has done some serious thinking about this, and written this >>>> up in a paper proposing a new astronomy format: >>>> >>>> Greenfield, M. Droettboom, E. Bray, >>>> ASDF: A new data format for astronomy, >>>> Astronomy and Computing, >>>> Volume 12, >>>> 2015, >>>> Pages 240-251, >>>> ISSN 2213-1337, >>>> https://doi.org/10.1016/j.ascom.2015.06.004. >>>> (https://www.sciencedirect.com/science/article/pii/S2213133715000645) >>>> >>>> Both the discussion and the format seem like a very good starting >>>> point for our own thinking about data formats, so we'd like to have a >>>> journal club to discuss it. >>>> >>>> I (Matthew) am going to organize this via Calendly, and I've made a >>>> Caldendly account for Nipy-related stuff, but, if you want to have a >>>> say in when we do the journal club, please let me know so I can add >>>> you to that account. I'll publish the Calendly link in a day or so. >>>> >>>> Cheers, >>>> >>>> Matthew >>>> >>>> [1] https://cyrille.rossant.net/moving-away-hdf5/ >>>> _______________________________________________ >>>> Neuroimaging mailing list >>>> Neuroimaging at python.org >>>> https://mail.python.org/mailman/listinfo/neuroimaging >>>> >>> -- >>> >>> Keveen Rodr?guez Zapata >>> B.Eng. Bioengineer >>> Msc. Student in Biomedical Sciences: Neuroscience and Bsc. Computer >>> Science >>> GRUNECO >>> Universidad de Antioquia >>> >>> >>> >>> "La informaci?n aqu? contenida es para uso exclusivo de la persona o >>> entidad de destino. Est? estrictamente prohibida su utilizaci?n, copia, >>> descarga, distribuci?n, modificaci?n y/o reproducci?n total o parcial, sin >>> el permiso expreso de Universidad de Antioquia, pues su contenido puede ser >>> de car?cter confidencial y/o contener material privilegiado. Si usted >>> recibi? esta informaci?n por error, por favor contacte en forma inmediata a >>> quien la envi? y borre este material de su computador. Universidad de >>> Antioquia no es responsable por la informaci?n contenida en esta >>> comunicaci?n, el directo responsable es quien la firma o el autor de la >>> misma." >>> UdeA_______________________________________________ >>> Neuroimaging mailing list >>> Neuroimaging at python.org >>> https://mail.python.org/mailman/listinfo/neuroimaging >>> >> _______________________________________________ >> Neuroimaging mailing list >> Neuroimaging at python.org >> https://mail.python.org/mailman/listinfo/neuroimaging >> > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Tue Nov 9 10:17:26 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 15:17:26 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Tue, Nov 9, 2021 at 2:16 PM JB Poline wrote: > > Hi, > > Another thing that could be useful would be to have a look at the NWB folks experience - they have gone through quite a few pain points - we ask one of the developer to give a little talk / summary of their experience I'm sure that would be interesting - there are so many issues here - I'm hoping that the ASDF paper will be a good introduction to what's involved. Later, we should indeed start to think harder about what we want and specific formats like HDF5 ... > I'd be interested as well I've added you ... Cheers, Matthew From matthew.brett at gmail.com Tue Nov 9 10:20:45 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 15:20:45 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Tue, Nov 9, 2021 at 1:48 PM Anderson M. Winkler wrote: > > Hi Matthew, hi all, > > I put some thinking on this a few years ago and concluded that HDF5 would be ideal. The blog post you link was already up and I didn't feel deterred by it, despite the interesting insights. > > At the time I put online a document for discussion (https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing) and as a starting point, made a mapping to how we'd store NIFTI as HDF5 (https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing). Excellent - thank you - that is very helpful. > At the end, I had to focus on other things, including career (and country) moves, and couldn't spend time and effort on this, despite the fact that the needs from then continue very much the same as today (if not perhaps even more needed now than then...). > > I'd be happy to participate in any effort you or others may lead towards this. I've added you to the scheduling list in case you'd like a say in the timing... Cheers, Matthew From matthew.brett at gmail.com Tue Nov 9 10:27:02 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 15:27:02 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Tue, Nov 9, 2021 at 2:35 PM Satrajit Ghosh wrote: > > hi, > > i can help provide a data archiving, metadata, and cloud/hpc access perspective. we are using hdf5 quite a bit both through nwb and otherwise at present, and are also using zarr, a cloud native format. i would not write off hdf5 and instead ask which use cases in what physical environments would run you into trouble. Just to clarify - I am acutely aware of the need to think carefully about use-cases. In particular, I wanted to think about libraries for accessing the data via an API - like Zarr - and the backend storage format, such as HDF5, or a more transparent format like ASDF, Exdir or similar. What are the issues for performance now, and in the future? Can we unlock the potential of all those great Python multiprocessing tools more easily with one format rather than another? And do we need the same or a different format for sharing data between applications - such as SPM? For example, might we want to have several Zarr backends, one for transparency, one for performance and one for cross-tool compatibility (such as HDF5). I think the paper is very good at laying out those questions - that's why I thought it would be a good place to start. Cheers, Matthew From njvack at wisc.edu Tue Nov 9 09:31:11 2021 From: njvack at wisc.edu (Nate Vack) Date: Tue, 9 Nov 2021 08:31:11 -0600 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: <186685_1636465688_0R2B01HST507GB30_CALKxBNzKLJ44KY-xNdi7iu-4k2YAWccbseTtp_0iHu9gVJhMoQ@mail.gmail.com> References: <186685_1636465688_0R2B01HST507GB30_CALKxBNzKLJ44KY-xNdi7iu-4k2YAWccbseTtp_0iHu9gVJhMoQ@mail.gmail.com> Message-ID: Hey there, Thanks for putting the thought into this! Having used HDF5 myself a bit, I tend to agree that it's a bit awkward in practice. Machine-readable-only metadata is not great and the installation process is usually fine, except when it... isn't. Looking at the ASDF spec, it does seem pretty compelling. I probably don't have a deep enough background in this to add much to your discussion, but I'm super interested to see where this goes. I will be very happy to live in a world without .json sidecars :) Best regards! -Nate On Tue, Nov 9, 2021 at 7:48 AM wrote: > Hi Matthew, hi all, > > I put some thinking on this a few years ago and concluded that HDF5 would > be ideal. The blog post you link was already up and I didn't feel deterred > by it, despite the interesting insights. > > At the time I put online a document for discussion ( > https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing) > and as a starting point, made a mapping to how we'd store NIFTI as HDF5 ( > https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing > ). > > At the end, I had to focus on other things, including career (and country) > moves, and couldn't spend time and effort on this, despite the fact that > the needs from then continue very much the same as today (if not perhaps > even more needed now than then...). > > I'd be happy to participate in any effort you or others may lead towards > this. > > Cheers, > > Anderson > > > On Tue, Nov 9, 2021 at 8:24 AM KEVEEN RODRIGUEZ ZAPATA < > keveen.rodriguez at udea.edu.co> wrote: > >> Hello, thanks for the initiative Matthew, I would like to be part of the >> schedule. >> >> On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett >> wrote: >> >>> Hi, >>> >>> Summary: we are planning to do a journal club to discuss new >>> neuroimaging data formats. Please let me (Matthew) know if you want >>> to be included in the scheduling for the journal club. Of course, >>> once we've got a time, anyone can join the journal club Zoom. >>> >>> Detail: >>> >>> As part of the Chan-Zuckerberg grant work, we are thinking about new >>> formats for using and sharing neuroimaging data. >>> >>> As you can see in the grant documents: >>> >>> https://github.com/matthew-brett/czi-nibabel >>> >>> we've been thinking about HDF5, but further research (e.g. [1]) has >>> given us some pause for reflection. Meanwhile, the astronomy >>> community has done some serious thinking about this, and written this >>> up in a paper proposing a new astronomy format: >>> >>> Greenfield, M. Droettboom, E. Bray, >>> ASDF: A new data format for astronomy, >>> Astronomy and Computing, >>> Volume 12, >>> 2015, >>> Pages 240-251, >>> ISSN 2213-1337, >>> https://doi.org/10.1016/j.ascom.2015.06.004. >>> (https://www.sciencedirect.com/science/article/pii/S2213133715000645) >>> >>> Both the discussion and the format seem like a very good starting >>> point for our own thinking about data formats, so we'd like to have a >>> journal club to discuss it. >>> >>> I (Matthew) am going to organize this via Calendly, and I've made a >>> Caldendly account for Nipy-related stuff, but, if you want to have a >>> say in when we do the journal club, please let me know so I can add >>> you to that account. I'll publish the Calendly link in a day or so. >>> >>> Cheers, >>> >>> Matthew >>> >>> [1] https://cyrille.rossant.net/moving-away-hdf5/ >>> _______________________________________________ >>> Neuroimaging mailing list >>> Neuroimaging at python.org >>> https://mail.python.org/mailman/listinfo/neuroimaging >>> >> -- >> >> Keveen Rodr?guez Zapata >> B.Eng. Bioengineer >> Msc. Student in Biomedical Sciences: Neuroscience and Bsc. Computer >> Science >> GRUNECO >> Universidad de Antioquia >> >> >> >> "La informaci?n aqu? contenida es para uso exclusivo de la persona o >> entidad de destino. Est? estrictamente prohibida su utilizaci?n, copia, >> descarga, distribuci?n, modificaci?n y/o reproducci?n total o parcial, sin >> el permiso expreso de Universidad de Antioquia, pues su contenido puede ser >> de car?cter confidencial y/o contener material privilegiado. Si usted >> recibi? esta informaci?n por error, por favor contacte en forma inmediata a >> quien la envi? y borre este material de su computador. Universidad de >> Antioquia no es responsable por la informaci?n contenida en esta >> comunicaci?n, el directo responsable es quien la firma o el autor de la >> misma." >> UdeA_______________________________________________ >> Neuroimaging mailing list >> Neuroimaging at python.org >> https://mail.python.org/mailman/listinfo/neuroimaging >> > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Tue Nov 9 11:11:44 2021 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 9 Nov 2021 11:11:44 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: hi matthew, that paper is a great place to start. is there a document that you have where we can add thoughts/pointers to things that have been developed or reevaluated between 2015 and now ? quick clarification: is the scope of the discussion limited to certain types of data (e.g, MRI, transforms - i think this was in the czi proposal) or broadly speaking all things neuroimaging (e.g. MEG/EEG/Microscopy/Genetics/Surfaces) or even more general (e.g. nd-arrays, trees, graphs )? cheers, satra On Tue, Nov 9, 2021 at 10:28 AM Matthew Brett wrote: > Hi, > > On Tue, Nov 9, 2021 at 2:35 PM Satrajit Ghosh wrote: > > > > hi, > > > > i can help provide a data archiving, metadata, and cloud/hpc access > perspective. we are using hdf5 quite a bit both through nwb and otherwise > at present, and are also using zarr, a cloud native format. i would not > write off hdf5 and instead ask which use cases in what physical > environments would run you into trouble. > > Just to clarify - I am acutely aware of the need to think carefully > about use-cases. In particular, I wanted to think about libraries > for accessing the data via an API - like Zarr - and the backend > storage format, such as HDF5, or a more transparent format like ASDF, > Exdir or similar. What are the issues for performance now, and in the > future? Can we unlock the potential of all those great Python > multiprocessing tools more easily with one format rather than another? > And do we need the same or a different format for sharing data > between applications - such as SPM? For example, might we want to > have several Zarr backends, one for transparency, one for performance > and one for cross-tool compatibility (such as HDF5). > > I think the paper is very good at laying out those questions - that's > why I thought it would be a good place to start. > > Cheers, > > Matthew > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Tue Nov 9 11:20:47 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 16:20:47 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Tue, Nov 9, 2021 at 4:12 PM Satrajit Ghosh wrote: > > hi matthew, > > that paper is a great place to start. is there a document that you have where we can add thoughts/pointers to things that have been developed or reevaluated between 2015 and now ? I haven't got that far yet. Anderson - I wonder whether your doc is good place for that? Or do we need another one, less focused on HDF5? > quick clarification: is the scope of the discussion limited to certain types of data (e.g, MRI, transforms - i think this was in the czi proposal) or broadly speaking all things neuroimaging (e.g. MEG/EEG/Microscopy/Genetics/Surfaces) or even more general (e.g. nd-arrays, trees, graphs )? Certainly surfaces - these are on the CZI proposal - but I was - at the moment - thinking of MRI / CT / PET / transforms. I haven't checked out the BIDS spec in detail - but I had naively imagined something that would be compatible with that, and cover the same sort of ground - if that proves interesting and necessary. For example - I could imagine a version of the nice YaML / binary ASDF format for neuroimaging data, that would have the advantage of being easily human-readable, being a single file to make copying and sharing easier, and allowing formal validation against a JSON schema. But that's really way ahead of where I am now, personally - I've got lots of reading and listening to do. Cheers, Matthew From satra at mit.edu Tue Nov 9 11:32:28 2021 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 9 Nov 2021 11:32:28 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: thank you matthew. happy to use any doc. in the meantime a quick few pointers here: zarr, xarray, fsspec, intake (all of these on the python side). let's also keep in mind compression and chunking, two things the asdf model explicitly decided not to address. and hdf5 now has serializable remote access on s3 stores both natively in the library and through fsspec. cheers, satra On Tue, Nov 9, 2021 at 11:21 AM Matthew Brett wrote: > Hi, > > On Tue, Nov 9, 2021 at 4:12 PM Satrajit Ghosh wrote: > > > > hi matthew, > > > > that paper is a great place to start. is there a document that you have > where we can add thoughts/pointers to things that have been developed or > reevaluated between 2015 and now ? > > I haven't got that far yet. Anderson - I wonder whether your doc is > good place for that? Or do we need another one, less focused on HDF5? > > > quick clarification: is the scope of the discussion limited to certain > types of data (e.g, MRI, transforms - i think this was in the czi proposal) > or broadly speaking all things neuroimaging (e.g. > MEG/EEG/Microscopy/Genetics/Surfaces) or even more general (e.g. nd-arrays, > trees, graphs )? > > Certainly surfaces - these are on the CZI proposal - but I was - at > the moment - thinking of MRI / CT / PET / transforms. I haven't > checked out the BIDS spec in detail - but I had naively imagined > something that would be compatible with that, and cover the same sort > of ground - if that proves interesting and necessary. > > For example - I could imagine a version of the nice YaML / binary ASDF > format for neuroimaging data, that would have the advantage of being > easily human-readable, being a single file to make copying and sharing > easier, and allowing formal validation against a JSON schema. But > that's really way ahead of where I am now, personally - I've got lots > of reading and listening to do. > > Cheers, > > Matthew > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Tue Nov 9 11:36:17 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Tue, 9 Nov 2021 16:36:17 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Tue, Nov 9, 2021 at 4:33 PM Satrajit Ghosh wrote: > > thank you matthew. happy to use any doc. > > in the meantime a quick few pointers here: zarr, xarray, fsspec, intake (all of these on the python side). let's also keep in mind compression and chunking, two things the asdf model explicitly decided not to address. Yes indeed - see :https://github.com/asdf-format/asdf/issues/718 > and hdf5 now has serializable remote access on s3 stores both natively in the library and through fsspec. Thanks - I had noticed that was in the works. What version of HDF did that arrive (natively)? Cheers, Matthew From satra at mit.edu Tue Nov 9 11:46:07 2021 From: satra at mit.edu (Satrajit Ghosh) Date: Tue, 9 Nov 2021 11:46:07 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: hi matthew, > and hdf5 now has serializable remote access on s3 stores both natively in > the library and through fsspec. > > Thanks - I had noticed that was in the works. What version of HDF did > that arrive (natively)? > 1.10.6 and 1.12.0 were the first versions in those respective series that support the read only s3 virtual file driver (VFD). both available through conda with the VFD enabled. cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From andersonwinkler at gmail.com Tue Nov 9 11:46:40 2021 From: andersonwinkler at gmail.com (Anderson M. Winkler) Date: Tue, 9 Nov 2021 11:46:40 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi Matthew, Good point. It would probably be a good idea to start a fresh one, more focused on what is more current (BIDS, for example, came a long way since 2016). Maybe include a link in the new doc to that old discussion so it doesn't get lost and we don't necessarily reiterate the same arguments -- and also think about addressing concerns where they existed. I think I received further feedback at the time, I'll dig up my email this afternoon and if I find I'll add to the old doc. It's good to revisit this topic! Thanks thanks! Cheers, Anderson On Tue, Nov 9, 2021 at 11:21 AM Matthew Brett wrote: > Hi, > > On Tue, Nov 9, 2021 at 4:12 PM Satrajit Ghosh wrote: > > > > hi matthew, > > > > that paper is a great place to start. is there a document that you have > where we can add thoughts/pointers to things that have been developed or > reevaluated between 2015 and now ? > > I haven't got that far yet. Anderson - I wonder whether your doc is > good place for that? Or do we need another one, less focused on HDF5? > > > quick clarification: is the scope of the discussion limited to certain > types of data (e.g, MRI, transforms - i think this was in the czi proposal) > or broadly speaking all things neuroimaging (e.g. > MEG/EEG/Microscopy/Genetics/Surfaces) or even more general (e.g. nd-arrays, > trees, graphs )? > > Certainly surfaces - these are on the CZI proposal - but I was - at > the moment - thinking of MRI / CT / PET / transforms. I haven't > checked out the BIDS spec in detail - but I had naively imagined > something that would be compatible with that, and cover the same sort > of ground - if that proves interesting and necessary. > > For example - I could imagine a version of the nice YaML / binary ASDF > format for neuroimaging data, that would have the advantage of being > easily human-readable, being a single file to make copying and sharing > easier, and allowing formal validation against a JSON schema. But > that's really way ahead of where I am now, personally - I've got lots > of reading and listening to do. > > Cheers, > > Matthew > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dimitri.papadopoulos at cea.fr Tue Nov 9 11:51:01 2021 From: dimitri.papadopoulos at cea.fr (Dimitri Papadopoulos Orfanos) Date: Tue, 9 Nov 2021 17:51:01 +0100 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: <6c3f63f0-bc32-54b2-d621-2fbef89e71df@cea.fr> Hi, I do not know how much the community has embraced the ISMRM Raw Data Format (ISMRMRD). It is based on HDF5 so it might be useful to hear from them and have a look at their format: https://ismrmrd.github.io Best, Dimitri Papadopoulos Le 09/11/2021 ? 15:16, JB Poline a ?crit?: > Hi, > > Another thing that could be useful would be to have a look at the NWB > folks experience - they have gone through quite a few pain points - we > ask one of the developer to give a little talk / summary of their experience > > I'd be interested as well > Cheers > JB > > On Tue, Nov 9, 2021 at 8:48 AM Anderson M. Winkler > > wrote: > > Hi Matthew, hi all, > > I put some thinking on this a few years ago and concluded that HDF5 > would be ideal. The blog post you link was already up and I didn't > feel deterred by it, despite the interesting insights. > > At the time I put online a document for discussion > (https://docs.google.com/document/d/1s5DX4YPS680mc3Rb9msLetrjlPcDhaw835Um-jJb-Dw/edit?usp=sharing > ) > and as a starting point, made a mapping to how we'd store NIFTI as > HDF5 > (https://docs.google.com/document/d/1hL27J2wNqHj27aX3VrCY8ajHyIBYtgYVmNmSoZZC8aA/edit?usp=sharing > ). > > At the end, I had to focus on other things, including career (and > country) moves, and couldn't spend time and effort on this, despite > the fact that the needs from then continue very?much the same as > today (if not perhaps even more needed now than then...). > > I'd be happy to participate in any effort you or others may lead > towards this. > > Cheers, > > Anderson > > > On Tue, Nov 9, 2021 at 8:24 AM KEVEEN RODRIGUEZ ZAPATA > > > wrote: > > Hello, thanks for the initiative Matthew, I would like to be > part of the schedule. > > On Tue, 9 Nov 2021 at 6:32 AM Matthew Brett > > wrote: > > Hi, > > Summary:? we are planning to do a journal club to discuss new > neuroimaging data formats.? Please let me (Matthew) know if > you want > to be included in the scheduling for the journal club.? Of > course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking > about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > > we've been thinking about HDF5, but further research (e.g. > [1]) has > given us some pause for reflection.? Meanwhile, the astronomy > community has done some serious thinking about this, and > written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004 > . > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > ) > > Both the discussion and the format seem like a very good > starting > point for our own thinking about data formats, so we'd like > to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've > made a > Caldendly account for Nipy-related stuff, but, if you want > to have a > say in when we do the journal club, please let me know so I > can add > you to that account.? I'll publish the Calendly link in a > day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > > > -- > > Keveen Rodr?guez Zapata > B.Eng. Bioengineer > Msc. Student in Biomedical Sciences: Neuroscience and Bsc. > Computer Science > GRUNECO > Universidad de Antioquia > > > > "La informaci?n aqu? contenida es para uso exclusivo de la > persona o entidad de destino. Est? estrictamente prohibida su > utilizaci?n, copia, descarga, distribuci?n, modificaci?n y/o > reproducci?n total o parcial, sin el permiso expreso de > Universidad de Antioquia, pues su contenido puede ser de > car?cter confidencial y/o contener material privilegiado. Si > usted recibi? esta informaci?n por error, por favor contacte en > forma inmediata a quien la envi? y borre este material de su > computador. Universidad de Antioquia?no es?responsable por la > informaci?n contenida en esta comunicaci?n, el directo > responsable es quien la firma o el autor de la misma." > UdeA_______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > From lists at onerussian.com Tue Nov 9 13:08:31 2021 From: lists at onerussian.com (Yaroslav Halchenko) Date: Tue, 9 Nov 2021 13:08:31 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: I think it might be useful to separate two aspects - data and metadata schema: - what data types to be stored (volume, surface, connectomes, ...) - what metadata should accompany data - data scope for an individual "file": there is a spectrum from an individual volume (e.g. single echo) to data acquisition session (that is what NWB aims for) - data container - that is where HDF5/zarr etc would come in Indeed current (and perspective future) landscape of neuroimaging, past and currently present formats and use cases (as for which data to store, and data and metadata access and modifications) should guide the design. Another point to keep in mind (speaking with my datalad and dandi archives hats on) while deciding on a "container": with a "rudimentary" nii + .json in BIDS we kinda reached some nice trade-off for being able to adjust/fix/expand metadata without causing changes to large files. monolythic hdf5 and alike are creating a notable difficulty for such modifications. on the other end of spectrum, zarr et al - explode in number of files to store, and thus likely to hit inode limits on many systems quite quickly. On Tue, 09 Nov 2021, Satrajit Ghosh wrote: > thank you matthew. happy to use any doc. > in the meantime a quick few pointers here: zarr, xarray, fsspec, intake > (all of these on the python side). let's also keep in mind compression and > chunking, two things the asdf model explicitly decided not to address. and > hdf5 now has serializable remote access on s3 stores both natively in the > library and through fsspec.? > cheers, > satra > On Tue, Nov 9, 2021 at 11:21 AM Matthew Brett > wrote: > Hi, > On Tue, Nov 9, 2021 at 4:12 PM Satrajit Ghosh wrote: > > hi matthew, > > that paper is a great place to start. is there a document that you > have where we can add thoughts/pointers to things that have been > developed or reevaluated between 2015 and now ? > I haven't got that far yet.? Anderson - I wonder whether your doc is > good place for that?? Or do we need another one, less focused on HDF5? > > quick clarification: is the scope of the discussion limited to certain > types of data (e.g, MRI, transforms - i think this was in the czi > proposal) or broadly speaking all things neuroimaging (e.g. > MEG/EEG/Microscopy/Genetics/Surfaces) or even more general (e.g. > nd-arrays, trees, graphs )? > Certainly surfaces - these are on the CZI proposal - but I was - at > the moment - thinking of MRI / CT / PET / transforms.? I haven't > checked out the BIDS spec in detail - but I had naively imagined > something that would be compatible with that, and cover the same sort > of ground - if that proves interesting and necessary. > For example - I could imagine a version of the nice YaML / binary ASDF > format for neuroimaging data, that would have the advantage of being > easily human-readable, being a single file to make copying and sharing > easier, and allowing formal validation against a JSON schema.? But > that's really way ahead of where I am now, personally - I've got lots > of reading and listening to do. > Cheers, > Matthew > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging -- Yaroslav O. Halchenko Center for Open Neuroscience http://centerforopenneuroscience.org Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 WWW: http://www.linkedin.com/in/yarik From arokem at uw.edu Tue Nov 9 13:32:50 2021 From: arokem at uw.edu (Ariel Rokem) Date: Tue, 9 Nov 2021 10:32:50 -0800 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi Matthew and all, Sounds interesting! At some point, we might want to discuss Francois Rheault's (and others') efforts to update the formats for tractography data (https://github.com/frheault/tractography_file_format/). Thanks for organizing! Ariel On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett wrote: > Hi, > > Summary: we are planning to do a journal club to discuss new > neuroimaging data formats. Please let me (Matthew) know if you want > to be included in the scheduling for the journal club. Of course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > we've been thinking about HDF5, but further research (e.g. [1]) has > given us some pause for reflection. Meanwhile, the astronomy > community has done some serious thinking about this, and written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004. > (https://www.sciencedirect.com/science/article/pii/S2213133715000645) > > Both the discussion and the format seem like a very good starting > point for our own thinking about data formats, so we'd like to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've made a > Caldendly account for Nipy-related stuff, but, if you want to have a > say in when we do the journal club, please let me know so I can add > you to that account. I'll publish the Calendly link in a day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.flandin at ucl.ac.uk Wed Nov 10 06:41:36 2021 From: g.flandin at ucl.ac.uk (Guillaume Flandin) Date: Wed, 10 Nov 2021 11:41:36 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Hi All, Also interested in attending, time permitting. There has been many discussions on this topic over the years across several platforms - one of them is captured in Anderson's document but I could also remember this one: https://github.com/bids-standard/bids-specification/issues/197 Compression was also discussed in the early days of BIDS but it seems it was a group email exchange with no public archive. Best, Guillaume. On 09/11/2021 18:32, Ariel Rokem wrote: > ? Caution: External sender > > > Hi Matthew and all,? > > Sounds interesting! At some point, we might want to discuss Francois > Rheault's (and others') efforts to update the formats for tractography > data (https://github.com/frheault/tractography_file_format/).? > > Thanks for organizing! > Ariel > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > wrote: > > Hi, > > Summary:? we are planning to do a journal club to discuss new > neuroimaging data formats.? Please let me (Matthew) know if you want > to be included in the scheduling for the journal club.? Of course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > we've been thinking about HDF5, but further research (e.g. [1]) has > given us some pause for reflection.? Meanwhile, the astronomy > community has done some serious thinking about this, and written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004 > . > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > ) > > Both the discussion and the format seem like a very good starting > point for our own thinking about data formats, so we'd like to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've made a > Caldendly account for Nipy-related stuff, but, if you want to have a > say in when we do the journal club, please let me know so I can add > you to that account.? I'll publish the Calendly link in a day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > From pestilli at utexas.edu Wed Nov 10 07:58:37 2021 From: pestilli at utexas.edu (Franco Pestilli) Date: Wed, 10 Nov 2021 06:58:37 -0600 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi Matthew, I would love to participate, this would synergize with our efforts of defining BIDS for Connectivity Derivatives: https://osf.io/t4nxy/ And the tractography file format was one of interests we were discussing @fFran?ois Rheault Cheers Franco On Wed, Nov 10, 2021 at 5:42 AM Guillaume Flandin wrote: > Hi All, > > Also interested in attending, time permitting. There has been many > discussions on this topic over the years across several platforms - one > of them is captured in Anderson's document but I could also remember > this one: > https://github.com/bids-standard/bids-specification/issues/197 > Compression was also discussed in the early days of BIDS but it seems it > was a group email exchange with no public archive. > > Best, > Guillaume. > > On 09/11/2021 18:32, Ariel Rokem wrote: > > ? Caution: External sender > > > > > > Hi Matthew and all, > > > > Sounds interesting! At some point, we might want to discuss Francois > > Rheault's (and others') efforts to update the formats for tractography > > data (https://github.com/frheault/tractography_file_format/). > > > > Thanks for organizing! > > Ariel > > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > > wrote: > > > > Hi, > > > > Summary: we are planning to do a journal club to discuss new > > neuroimaging data formats. Please let me (Matthew) know if you want > > to be included in the scheduling for the journal club. Of course, > > once we've got a time, anyone can join the journal club Zoom. > > > > Detail: > > > > As part of the Chan-Zuckerberg grant work, we are thinking about new > > formats for using and sharing neuroimaging data. > > > > As you can see in the grant documents: > > > > https://github.com/matthew-brett/czi-nibabel > > > > we've been thinking about HDF5, but further research (e.g. [1]) has > > given us some pause for reflection. Meanwhile, the astronomy > > community has done some serious thinking about this, and written this > > up in a paper proposing a new astronomy format: > > > > Greenfield, M. Droettboom, E. Bray, > > ASDF: A new data format for astronomy, > > Astronomy and Computing, > > Volume 12, > > 2015, > > Pages 240-251, > > ISSN 2213-1337, > > https://doi.org/10.1016/j.ascom.2015.06.004 > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.ascom.2015.06.004&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862344941%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Z%2FHoOKl4tljjjzQ5IWnHjwMH65eunYlTNIdbIK6CTXI%3D&reserved=0 > >. > > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sciencedirect.com%2Fscience%2Farticle%2Fpii%2FS2213133715000645&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ialaMuABzvWaOXvlBj0KcpzRYYmXrjyLvW8Lm2CnoHc%3D&reserved=0 > >) > > > > Both the discussion and the format seem like a very good starting > > point for our own thinking about data formats, so we'd like to have a > > journal club to discuss it. > > > > I (Matthew) am going to organize this via Calendly, and I've made a > > Caldendly account for Nipy-related stuff, but, if you want to have a > > say in when we do the journal club, please let me know so I can add > > you to that account. I'll publish the Calendly link in a day or so. > > > > Cheers, > > > > Matthew > > > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcyrille.rossant.net%2Fmoving-away-hdf5%2F&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=afzzkSku7KmL1XetF6NaTsmxDDQ8LR5ebFe0q1oov5c%3D&reserved=0 > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fneuroimaging&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862364927%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=54jpAtVghfeD5eVfeGJvpcGwgn84E5l1rtvk4CXXV6U%3D&reserved=0 > > > > > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Nov 10 08:22:05 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 10 Nov 2021 13:22:05 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi, On Wed, Nov 10, 2021 at 11:42 AM Guillaume Flandin wrote: > > Hi All, > > Also interested in attending, time permitting. There has been many > discussions on this topic over the years across several platforms - one > of them is captured in Anderson's document but I could also remember > this one: > https://github.com/bids-standard/bids-specification/issues/197 > Compression was also discussed in the early days of BIDS but it seems it > was a group email exchange with no public archive. Aha - yes - that's a shame. It seems to me it would be good to keep an archive of the various problems and opinions expressed so we don't lose track. Anderson's document was very useful in that respect, but it would also be very useful to have actual emails, replies, reflections in their raw form where anyone can read them. I've made a Discourse topic for that, in the hope that it would be a good way of keeping track. Will post about it in a second. Cheers, Matthew From matthew.brett at gmail.com Wed Nov 10 08:32:09 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 10 Nov 2021 13:32:09 +0000 Subject: [Neuroimaging] Data formats Discourse category Message-ID: Hi, The data formats journal club brought up many very useful lines of discussion, and revealed that there are also discussions that have happened, that are not archived. I have taken the liberty of making a new Discourse forum for data format discussions. If you're interested, please go to: https://nipy.discourse.group/c/data-formats and sign up if you need to. As I've said in the introduction there, it's a Nipy forum, but that's only because we've got a dedicated Nipy discourse site - it doesn't imply any ownership of the discussion. I'm just hoping that the Discourse interface will prove a good way of supporting and recording the discussion for posterity - for packages and platforms that are interested. If anyone wants more admin access to the forum, just let me know. Cheers, Matthew From tpengo at umn.edu Wed Nov 10 11:34:09 2021 From: tpengo at umn.edu (Thomas Pengo) Date: Wed, 10 Nov 2021 10:34:09 -0600 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Dear all I am new to writing to this list but I simply wanted to bring a couple of pointers to related efforts in microscopy. Apologies if this is redundant. Open Microscopy Environment. They started with a data model, in XML. The data model covers data and metadata so initially you could write data and metadata in pure XML. However, a more practical solution involved TIFF with a metadata encoded jn XML embedded in the headers. Now, and this is the interesting part, a larger consortium (and here is the interesting part) is working on a Next Generation File Format, that does not use HDF5 due to parallel read/write problems, but Zarr. My point being that choosing HDF5 now might has some implications on data access down the road. The more the communities connect the more we can benefit from each other?s previous work. https://ngff.openmicroscopy.org/latest/ https://forum.image.sc/tag/ome-ngff https://quarep.org/working-groups/wg-7-metadata/ Sincerely Thomas Thomas On Wed, Nov 10, 2021 at 5:41 AM Guillaume Flandin wrote: > Hi All, > > Also interested in attending, time permitting. There has been many > discussions on this topic over the years across several platforms - one > of them is captured in Anderson's document but I could also remember > this one: > https://github.com/bids-standard/bids-specification/issues/197 > Compression was also discussed in the early days of BIDS but it seems it > was a group email exchange with no public archive. > > Best, > Guillaume. > > On 09/11/2021 18:32, Ariel Rokem wrote: > > ? Caution: External sender > > > > > > Hi Matthew and all, > > > > Sounds interesting! At some point, we might want to discuss Francois > > Rheault's (and others') efforts to update the formats for tractography > > data (https://github.com/frheault/tractography_file_format/). > > > > Thanks for organizing! > > Ariel > > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > > wrote: > > > > Hi, > > > > Summary: we are planning to do a journal club to discuss new > > neuroimaging data formats. Please let me (Matthew) know if you want > > to be included in the scheduling for the journal club. Of course, > > once we've got a time, anyone can join the journal club Zoom. > > > > Detail: > > > > As part of the Chan-Zuckerberg grant work, we are thinking about new > > formats for using and sharing neuroimaging data. > > > > As you can see in the grant documents: > > > > https://github.com/matthew-brett/czi-nibabel > > > > we've been thinking about HDF5, but further research (e.g. [1]) has > > given us some pause for reflection. Meanwhile, the astronomy > > community has done some serious thinking about this, and written this > > up in a paper proposing a new astronomy format: > > > > Greenfield, M. Droettboom, E. Bray, > > ASDF: A new data format for astronomy, > > Astronomy and Computing, > > Volume 12, > > 2015, > > Pages 240-251, > > ISSN 2213-1337, > > https://doi.org/10.1016/j.ascom.2015.06.004 > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.ascom.2015.06.004&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862344941%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Z%2FHoOKl4tljjjzQ5IWnHjwMH65eunYlTNIdbIK6CTXI%3D&reserved=0 > >. > > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.sciencedirect.com%2Fscience%2Farticle%2Fpii%2FS2213133715000645&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ialaMuABzvWaOXvlBj0KcpzRYYmXrjyLvW8Lm2CnoHc%3D&reserved=0 > >) > > > > Both the discussion and the format seem like a very good starting > > point for our own thinking about data formats, so we'd like to have a > > journal club to discuss it. > > > > I (Matthew) am going to organize this via Calendly, and I've made a > > Caldendly account for Nipy-related stuff, but, if you want to have a > > say in when we do the journal club, please let me know so I can add > > you to that account. I'll publish the Calendly link in a day or so. > > > > Cheers, > > > > Matthew > > > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcyrille.rossant.net%2Fmoving-away-hdf5%2F&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862354932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=afzzkSku7KmL1XetF6NaTsmxDDQ8LR5ebFe0q1oov5c%3D&reserved=0 > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > < > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fneuroimaging&data=04%7C01%7Cg.flandin%40ucl.ac.uk%7C5ed1afff99ca460e7dc008d9a3b04a6c%7C1faf88fea9984c5b93c9210a11d9a5c2%7C0%7C0%7C637720800862364927%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=54jpAtVghfeD5eVfeGJvpcGwgn84E5l1rtvk4CXXV6U%3D&reserved=0 > > > > > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Nov 10 12:22:02 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 10 Nov 2021 17:22:02 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi, On Wed, Nov 10, 2021 at 4:35 PM Thomas Pengo via Neuroimaging wrote: > > Dear all > > I am new to writing to this list but I simply wanted to bring a couple of pointers to related efforts in microscopy. Apologies if this is redundant. > > Open Microscopy Environment. They started with a data model, in XML. The data model covers data and metadata so initially you could write data and metadata in pure XML. However, a more practical solution involved TIFF with a metadata encoded jn XML embedded in the headers. Now, and this is the interesting part, a larger consortium (and here is the interesting part) is working on a Next Generation File Format, that does not use HDF5 due to parallel read/write problems, but Zarr. My point being that choosing HDF5 now might has some implications on data access down the road. > > The more the communities connect the more we can benefit from each other?s previous work. > > https://ngff.openmicroscopy.org/latest/ > > https://forum.image.sc/tag/ome-ngff > > https://quarep.org/working-groups/wg-7-metadata/ Thanks - these are strikingly relevant ... Cheers, Matthew From elef at indiana.edu Wed Nov 10 12:32:35 2021 From: elef at indiana.edu (Eleftherios Garyfallidis) Date: Wed, 10 Nov 2021 12:32:35 -0500 Subject: [Neuroimaging] Data formats Discourse category In-Reply-To: References: Message-ID: Matthew, thank you for doing this. A great initiative. Francois, can you please report at your convenience the plans and discussions about the new tracking format? On Wed, Nov 10, 2021 at 8:33 AM Matthew Brett wrote: > Hi, > > The data formats journal club brought up many very useful lines of > discussion, and revealed that there are also discussions that have > happened, that are not archived. > > I have taken the liberty of making a new Discourse forum for data > format discussions. If you're interested, please go to: > > https://nipy.discourse.group/c/data-formats > > and sign up if you need to. > > As I've said in the introduction there, it's a Nipy forum, but that's > only because we've got a dedicated Nipy discourse site - it doesn't > imply any ownership of the discussion. I'm just hoping that the > Discourse interface will prove a good way of supporting and recording > the discussion for posterity - for packages and platforms that are > interested. > > If anyone wants more admin access to the forum, just let me know. > > Cheers, > > Matthew > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Francois.M.Rheault at USherbrooke.ca Wed Nov 10 12:58:45 2021 From: Francois.M.Rheault at USherbrooke.ca (=?utf-8?B?RnJhbsOnb2lzIFJoZWF1bHQ=?=) Date: Wed, 10 Nov 2021 17:58:45 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi, I would be happy to discuss and collaborate on this! Because of COVID the whole project slowed down quite a bit and I needed help for an external implementation of a prototype. I will try to push everyone involve (yet again) to make sure the project is alive and well! Francois Rheault Universit? de Sherbrooke ________________________________ De : Franco Pestilli Envoy? : 10 novembre 2021 06:58 ? : Neuroimaging analysis in Python ; Fran?ois Rheault Objet : Re: [Neuroimaging] Planning for data formats - upcoming journal club Hi Matthew, I would love to participate, this would synergize with our efforts of defining BIDS for Connectivity Derivatives: https://osf.io/t4nxy/ And the tractography file format was one of interests we were discussing @fFran?ois Rheault Cheers Franco On Wed, Nov 10, 2021 at 5:42 AM Guillaume Flandin > wrote: Hi All, Also interested in attending, time permitting. There has been many discussions on this topic over the years across several platforms - one of them is captured in Anderson's document but I could also remember this one: https://github.com/bids-standard/bids-specification/issues/197 Compression was also discussed in the early days of BIDS but it seems it was a group email exchange with no public archive. Best, Guillaume. On 09/11/2021 18:32, Ariel Rokem wrote: > ? Caution: External sender > > > Hi Matthew and all, > > Sounds interesting! At some point, we might want to discuss Francois > Rheault's (and others') efforts to update the formats for tractography > data (https://github.com/frheault/tractography_file_format/). > > Thanks for organizing! > Ariel > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > >> wrote: > > Hi, > > Summary: we are planning to do a journal club to discuss new > neuroimaging data formats. Please let me (Matthew) know if you want > to be included in the scheduling for the journal club. Of course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > we've been thinking about HDF5, but further research (e.g. [1]) has > given us some pause for reflection. Meanwhile, the astronomy > community has done some serious thinking about this, and written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004 > >. > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > >) > > Both the discussion and the format seem like a very good starting > point for our own thinking about data formats, so we'd like to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've made a > Caldendly account for Nipy-related stuff, but, if you want to have a > say in when we do the journal club, please let me know so I can add > you to that account. I'll publish the Calendly link in a day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > _______________________________________________ Neuroimaging mailing list Neuroimaging at python.org https://mail.python.org/mailman/listinfo/neuroimaging -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Fri Nov 12 06:45:59 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Fri, 12 Nov 2021 11:45:59 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi, I've put up a Doodle poll for the journal club, having failed to work out how to do the same job in Calendly: https://doodle.com/poll/5vwkdt7zic2bu3wn Please do fill in your times there, Cheers, Matthew On Wed, Nov 10, 2021 at 7:22 PM Fran?ois Rheault wrote: > > Hi, > > I would be happy to discuss and collaborate on this! Because of COVID the whole project slowed down quite a bit and I needed help for an external implementation of a prototype. I will try to push everyone involve (yet again) to make sure the project is alive and well! > > Francois Rheault > > Universit? de Sherbrooke > > ________________________________ > De : Franco Pestilli > Envoy? : 10 novembre 2021 06:58 > ? : Neuroimaging analysis in Python ; Fran?ois Rheault > Objet : Re: [Neuroimaging] Planning for data formats - upcoming journal club > > Hi Matthew, > > I would love to participate, this would synergize with our efforts of defining BIDS for Connectivity Derivatives: https://osf.io/t4nxy/ > And the tractography file format was one of interests we were discussing @fFran?ois Rheault > > Cheers > Franco > > On Wed, Nov 10, 2021 at 5:42 AM Guillaume Flandin wrote: > > Hi All, > > Also interested in attending, time permitting. There has been many > discussions on this topic over the years across several platforms - one > of them is captured in Anderson's document but I could also remember > this one: > https://github.com/bids-standard/bids-specification/issues/197 > Compression was also discussed in the early days of BIDS but it seems it > was a group email exchange with no public archive. > > Best, > Guillaume. > > On 09/11/2021 18:32, Ariel Rokem wrote: > > ? Caution: External sender > > > > > > Hi Matthew and all, > > > > Sounds interesting! At some point, we might want to discuss Francois > > Rheault's (and others') efforts to update the formats for tractography > > data (https://github.com/frheault/tractography_file_format/). > > > > Thanks for organizing! > > Ariel > > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > > wrote: > > > > Hi, > > > > Summary: we are planning to do a journal club to discuss new > > neuroimaging data formats. Please let me (Matthew) know if you want > > to be included in the scheduling for the journal club. Of course, > > once we've got a time, anyone can join the journal club Zoom. > > > > Detail: > > > > As part of the Chan-Zuckerberg grant work, we are thinking about new > > formats for using and sharing neuroimaging data. > > > > As you can see in the grant documents: > > > > https://github.com/matthew-brett/czi-nibabel > > > > we've been thinking about HDF5, but further research (e.g. [1]) has > > given us some pause for reflection. Meanwhile, the astronomy > > community has done some serious thinking about this, and written this > > up in a paper proposing a new astronomy format: > > > > Greenfield, M. Droettboom, E. Bray, > > ASDF: A new data format for astronomy, > > Astronomy and Computing, > > Volume 12, > > 2015, > > Pages 240-251, > > ISSN 2213-1337, > > https://doi.org/10.1016/j.ascom.2015.06.004 > > . > > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > > ) > > > > Both the discussion and the format seem like a very good starting > > point for our own thinking about data formats, so we'd like to have a > > journal club to discuss it. > > > > I (Matthew) am going to organize this via Calendly, and I've made a > > Caldendly account for Nipy-related stuff, but, if you want to have a > > say in when we do the journal club, please let me know so I can add > > you to that account. I'll publish the Calendly link in a day or so. > > > > Cheers, > > > > Matthew > > > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging > > _______________________________________________ > Neuroimaging mailing list > Neuroimaging at python.org > https://mail.python.org/mailman/listinfo/neuroimaging From matthew.brett at gmail.com Fri Nov 12 07:34:12 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Fri, 12 Nov 2021 12:34:12 +0000 Subject: [Neuroimaging] First post on data compression over at nipy.discourse.group Message-ID: Hi, Just to say, if you're interested, I've done a first post over at the data formats Discourse category, asking about data compression: https://nipy.discourse.group/t/data-compression-metrics-for-fmri/93 Cheers, Matthew From j.a.moore at dundee.ac.uk Fri Nov 12 09:44:54 2021 From: j.a.moore at dundee.ac.uk (Joshua Moore (Staff)) Date: Fri, 12 Nov 2021 14:44:54 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club Message-ID: > Thanks - these are strikingly relevant ... I?d second that! But I (with my OME hat on) would also second Satra said that there are cases & users that will continue to prefer HDF5 and once we?ve worked through the design of NGFF, we?ll likely try to bring the same design to HDF5, keeping the two convertible to one another to support as much of the community as possible. And for what it?s worth, we also received CZI funding (with my Zarr hat on) for driving imaging forward, and the kind folks at Unidata are doing what they can to have HDF5 & Zarr supported seamlessly through NetCDF. Exciting times. Happy to discuss further or join the/a call as appropriate. ~Josh P.S. Sorry if this email breaks the mailing list thread. I?ve only just joined after being kindly pointed to it by Greg Lee. On Wed, Nov 10, 2021 at 12:22 EST Matthew Brett via Neuroimaging > On Wed, Nov 10, 2021 at 4:35 PM Thomas Pengo via Neuroimaging > wrote: > > I am new to writing to this list but I simply wanted to bring a couple of pointers to related efforts in microscopy. Apologies if this is redundant. > > > > Open Microscopy Environment. They started with a data model, in XML. The data model covers data and metadata so initially you could write data and metadata in pure XML. However, a more practical solution involved TIFF with a metadata encoded jn XML embedded in the headers. Now, and this is the interesting part, a larger consortium (and here is the interesting part) is working on a Next Generation File Format, that does not use HDF5 due to parallel read/write problems, but Zarr. My point being that choosing HDF5 now might has some implications on data access down the road. > > > > The more the communities connect the more we can benefit from each other?s previous work. > > > > https://ngff.openmicroscopy.org/latest/ > > > > https://forum.image.sc/tag/ome-ngff > > > > https://quarep.org/working-groups/wg-7-metadata/ > > Thanks - these are strikingly relevant ... > > Cheers, > > Matthew The University of Dundee is a registered Scottish Charity, No: SC015096 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Fri Nov 12 09:59:29 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Fri, 12 Nov 2021 14:59:29 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, On Fri, Nov 12, 2021 at 2:45 PM Joshua Moore (Staff) wrote: > > > Thanks - these are strikingly relevant ... > > > > I?d second that! But I (with my OME hat on) would also second Satra said that there are cases & users that will continue to prefer HDF5 and once we?ve worked through the design of NGFF, we?ll likely try to bring the same design to HDF5, keeping the two convertible to one another to support as much of the community as possible. > > > > And for what it?s worth, we also received CZI funding (with my Zarr hat on) for driving imaging forward, and the kind folks at Unidata are doing what they can to have HDF5 & Zarr supported seamlessly through NetCDF. Exciting times. Thanks - yes - it would be very good if you had time to come to the journal club - Doodle: https://doodle.com/poll/5vwkdt7zic2bu3wn I think we're going to be reflecting there on: * Whether we should think in terms of API (e.g. Zarr) rather than data format * Whether we should distinguish between a sharing format and a for-processing format. * What criteria we should be applying to each of these formats. I'm sure you've thought hard about these, and it would be very helpful to have your thoughts. I would also, personally, love to see a show-and-tell for the various parts we are thinking about, such as Zarr, HDF5 / Minc, NeuroJSON (http://neurojson.org), and so on. And maybe "Why I will never use an HDF5 / NeuroJSON / format. Perhaps we could plan that after the journal club ... Cheers, Matthew From matthew.brett at gmail.com Wed Nov 17 11:57:54 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 17 Nov 2021 16:57:54 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Dear all, There haven't been any responses on the Doodle poll in the last day or so, so time to call it for the meeting time. It looks like November 24th at 19:00 - 20:30 UTC is the best date overall, with 12 green ticks and one orange, so let's go with that. I'll send out a Zoom invite soon, Cheers, Matthew On Fri, Nov 12, 2021 at 11:45 AM Matthew Brett wrote: > > Hi, > > I've put up a Doodle poll for the journal club, having failed to work > out how to do the same job in Calendly: > > https://doodle.com/poll/5vwkdt7zic2bu3wn > > Please do fill in your times there, > > Cheers, > > Matthew > > On Wed, Nov 10, 2021 at 7:22 PM Fran?ois Rheault > wrote: > > > > Hi, > > > > I would be happy to discuss and collaborate on this! Because of COVID the whole project slowed down quite a bit and I needed help for an external implementation of a prototype. I will try to push everyone involve (yet again) to make sure the project is alive and well! > > > > Francois Rheault > > > > Universit? de Sherbrooke > > > > ________________________________ > > De : Franco Pestilli > > Envoy? : 10 novembre 2021 06:58 > > ? : Neuroimaging analysis in Python ; Fran?ois Rheault > > Objet : Re: [Neuroimaging] Planning for data formats - upcoming journal club > > > > Hi Matthew, > > > > I would love to participate, this would synergize with our efforts of defining BIDS for Connectivity Derivatives: https://osf.io/t4nxy/ > > And the tractography file format was one of interests we were discussing @fFran?ois Rheault > > > > Cheers > > Franco > > > > On Wed, Nov 10, 2021 at 5:42 AM Guillaume Flandin wrote: > > > > Hi All, > > > > Also interested in attending, time permitting. There has been many > > discussions on this topic over the years across several platforms - one > > of them is captured in Anderson's document but I could also remember > > this one: > > https://github.com/bids-standard/bids-specification/issues/197 > > Compression was also discussed in the early days of BIDS but it seems it > > was a group email exchange with no public archive. > > > > Best, > > Guillaume. > > > > On 09/11/2021 18:32, Ariel Rokem wrote: > > > ? Caution: External sender > > > > > > > > > Hi Matthew and all, > > > > > > Sounds interesting! At some point, we might want to discuss Francois > > > Rheault's (and others') efforts to update the formats for tractography > > > data (https://github.com/frheault/tractography_file_format/). > > > > > > Thanks for organizing! > > > Ariel > > > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > > > wrote: > > > > > > Hi, > > > > > > Summary: we are planning to do a journal club to discuss new > > > neuroimaging data formats. Please let me (Matthew) know if you want > > > to be included in the scheduling for the journal club. Of course, > > > once we've got a time, anyone can join the journal club Zoom. > > > > > > Detail: > > > > > > As part of the Chan-Zuckerberg grant work, we are thinking about new > > > formats for using and sharing neuroimaging data. > > > > > > As you can see in the grant documents: > > > > > > https://github.com/matthew-brett/czi-nibabel > > > > > > we've been thinking about HDF5, but further research (e.g. [1]) has > > > given us some pause for reflection. Meanwhile, the astronomy > > > community has done some serious thinking about this, and written this > > > up in a paper proposing a new astronomy format: > > > > > > Greenfield, M. Droettboom, E. Bray, > > > ASDF: A new data format for astronomy, > > > Astronomy and Computing, > > > Volume 12, > > > 2015, > > > Pages 240-251, > > > ISSN 2213-1337, > > > https://doi.org/10.1016/j.ascom.2015.06.004 > > > . > > > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > > > ) > > > > > > Both the discussion and the format seem like a very good starting > > > point for our own thinking about data formats, so we'd like to have a > > > journal club to discuss it. > > > > > > I (Matthew) am going to organize this via Calendly, and I've made a > > > Caldendly account for Nipy-related stuff, but, if you want to have a > > > say in when we do the journal club, please let me know so I can add > > > you to that account. I'll publish the Calendly link in a day or so. > > > > > > Cheers, > > > > > > Matthew > > > > > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > > > > > _______________________________________________ > > > Neuroimaging mailing list > > > Neuroimaging at python.org > > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > > > > > > > > _______________________________________________ > > > Neuroimaging mailing list > > > Neuroimaging at python.org > > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > _______________________________________________ > > Neuroimaging mailing list > > Neuroimaging at python.org > > https://mail.python.org/mailman/listinfo/neuroimaging From matthew.brett at gmail.com Wed Nov 17 12:30:55 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 17 Nov 2021 17:30:55 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: <3197fe80-1c4c-104c-a695-175b119ba92e@ucl.ac.uk> Message-ID: Hi, Zoom link is https://bham-ac-uk.zoom.us/j/85378614517?pwd=Y09EU2hxNWdIOWNwWmwxOVpBVStMZz09 Anyone welcome. I believe I've sent email invites to everyone on the Doodle poll that I knew email addresses for. Cheers, Matthew On Wed, Nov 17, 2021 at 4:57 PM Matthew Brett wrote: > > Dear all, > > There haven't been any responses on the Doodle poll in the last day or > so, so time to call it for the meeting time. > > It looks like November 24th at 19:00 - 20:30 UTC is the best date > overall, with 12 green ticks and one orange, so let's go with that. > > I'll send out a Zoom invite soon, > > Cheers, > > Matthew > > On Fri, Nov 12, 2021 at 11:45 AM Matthew Brett wrote: > > > > Hi, > > > > I've put up a Doodle poll for the journal club, having failed to work > > out how to do the same job in Calendly: > > > > https://doodle.com/poll/5vwkdt7zic2bu3wn > > > > Please do fill in your times there, > > > > Cheers, > > > > Matthew > > > > On Wed, Nov 10, 2021 at 7:22 PM Fran?ois Rheault > > wrote: > > > > > > Hi, > > > > > > I would be happy to discuss and collaborate on this! Because of COVID the whole project slowed down quite a bit and I needed help for an external implementation of a prototype. I will try to push everyone involve (yet again) to make sure the project is alive and well! > > > > > > Francois Rheault > > > > > > Universit? de Sherbrooke > > > > > > ________________________________ > > > De : Franco Pestilli > > > Envoy? : 10 novembre 2021 06:58 > > > ? : Neuroimaging analysis in Python ; Fran?ois Rheault > > > Objet : Re: [Neuroimaging] Planning for data formats - upcoming journal club > > > > > > Hi Matthew, > > > > > > I would love to participate, this would synergize with our efforts of defining BIDS for Connectivity Derivatives: https://osf.io/t4nxy/ > > > And the tractography file format was one of interests we were discussing @fFran?ois Rheault > > > > > > Cheers > > > Franco > > > > > > On Wed, Nov 10, 2021 at 5:42 AM Guillaume Flandin wrote: > > > > > > Hi All, > > > > > > Also interested in attending, time permitting. There has been many > > > discussions on this topic over the years across several platforms - one > > > of them is captured in Anderson's document but I could also remember > > > this one: > > > https://github.com/bids-standard/bids-specification/issues/197 > > > Compression was also discussed in the early days of BIDS but it seems it > > > was a group email exchange with no public archive. > > > > > > Best, > > > Guillaume. > > > > > > On 09/11/2021 18:32, Ariel Rokem wrote: > > > > ? Caution: External sender > > > > > > > > > > > > Hi Matthew and all, > > > > > > > > Sounds interesting! At some point, we might want to discuss Francois > > > > Rheault's (and others') efforts to update the formats for tractography > > > > data (https://github.com/frheault/tractography_file_format/). > > > > > > > > Thanks for organizing! > > > > Ariel > > > > On Tue, Nov 9, 2021 at 3:32 AM Matthew Brett > > > > wrote: > > > > > > > > Hi, > > > > > > > > Summary: we are planning to do a journal club to discuss new > > > > neuroimaging data formats. Please let me (Matthew) know if you want > > > > to be included in the scheduling for the journal club. Of course, > > > > once we've got a time, anyone can join the journal club Zoom. > > > > > > > > Detail: > > > > > > > > As part of the Chan-Zuckerberg grant work, we are thinking about new > > > > formats for using and sharing neuroimaging data. > > > > > > > > As you can see in the grant documents: > > > > > > > > https://github.com/matthew-brett/czi-nibabel > > > > > > > > we've been thinking about HDF5, but further research (e.g. [1]) has > > > > given us some pause for reflection. Meanwhile, the astronomy > > > > community has done some serious thinking about this, and written this > > > > up in a paper proposing a new astronomy format: > > > > > > > > Greenfield, M. Droettboom, E. Bray, > > > > ASDF: A new data format for astronomy, > > > > Astronomy and Computing, > > > > Volume 12, > > > > 2015, > > > > Pages 240-251, > > > > ISSN 2213-1337, > > > > https://doi.org/10.1016/j.ascom.2015.06.004 > > > > . > > > > (https://www.sciencedirect.com/science/article/pii/S2213133715000645 > > > > ) > > > > > > > > Both the discussion and the format seem like a very good starting > > > > point for our own thinking about data formats, so we'd like to have a > > > > journal club to discuss it. > > > > > > > > I (Matthew) am going to organize this via Calendly, and I've made a > > > > Caldendly account for Nipy-related stuff, but, if you want to have a > > > > say in when we do the journal club, please let me know so I can add > > > > you to that account. I'll publish the Calendly link in a day or so. > > > > > > > > Cheers, > > > > > > > > Matthew > > > > > > > > [1] https://cyrille.rossant.net/moving-away-hdf5/ > > > > > > > > _______________________________________________ > > > > Neuroimaging mailing list > > > > Neuroimaging at python.org > > > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > > > > > > > > > > > > _______________________________________________ > > > > Neuroimaging mailing list > > > > Neuroimaging at python.org > > > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > > > _______________________________________________ > > > Neuroimaging mailing list > > > Neuroimaging at python.org > > > https://mail.python.org/mailman/listinfo/neuroimaging > > > > > > _______________________________________________ > > > Neuroimaging mailing list > > > Neuroimaging at python.org > > > https://mail.python.org/mailman/listinfo/neuroimaging From mokhtarabc495 at gmail.com Tue Nov 16 10:04:51 2021 From: mokhtarabc495 at gmail.com (Mokhtar Mokhtar) Date: Tue, 16 Nov 2021 16:04:51 +0100 Subject: [Neuroimaging] IRM de diffusion Message-ID: Good afternoon , sir how to make a graphical interface for the visualization of dicom images (diffusion MRI) in 3 dimensions after transforming them into nifti images and load all the codes you made in python please Thank you Best regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From fangqq at gmail.com Wed Nov 24 01:04:31 2021 From: fangqq at gmail.com (Qianqian Fang) Date: Wed, 24 Nov 2021 01:04:31 -0500 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: <4728dc7d-f62e-85a1-4063-721ad3b7ab2e@gmail.com> Dear list, just want to share this offline self-intro to Matt, following the thread of data format discussions - I look forward to the journal club discussions tomorrow. -------- Forwarded Message -------- Subject: Fwd: data formats Date: Thu, 11 Nov 2021 15:03:17 -0500 From: Qianqian Fang To: matthew.brett at gmail.com CC: Guillaume Flandin hi Matt, Guillaume forwarded me a thread of discussions regarding an upcoming journal club on data formats. I tried to register to the mailing list but it has not been approved (the list is moderated). just a quick self-introduction - I recently received a U24 award (NeuroJSON, http://neurojson.org) from the NIH on developing JSON-based universal data exchange formats (with NoSQL db support) for the broad neuroimaging community. Robert Oostenveld (BIDS), Guilaume Flandin (SPM) and a few other project leads (freesurfer, brainstorm, homer ...) are in our workgroup. You can find more info about the NeuroJSON project on our websites and a recent poster https://pbs.twimg.com/media/FCD_JNtWQAgLq6N?format=png&name=4096x4096 (a poster at fNIRS2021) http://openjdata.org http://neurojson.org? (partially completed) the idea behind is very simple - we want to make data human-readable, universally parsable with minimal overhead, scalable, and flexible for future extensions. We have been developing 1-to-1 mapping specifications to mirror neuroimaging data files to JSON (JNIfTI : nifti->json, JSNIRF : SNIRF ->json etc); the JSON serialization of complex data is based on a lightweight annotation specification called JData - supporting N-D strongly-typed binary arrays , complex /sparse arrays , trees, graphs, etc. Ultimately, we will map all BIDS folder structure to a JSON tree and optionally push to a NoSQL db (mongodb, couchdb ...) for scalability and dissemination. We currently have lightweight parsers/writers to process JData-styled annotations for Python (pyjdata and pybj ), MATLAB/Octave (jsonlab ), javascript (jsdata ) and C/C++. I'd love to participate this discussion, either as part of the journal club, or schedule a separate meeting, and present our project roadmaps/resources available. let me know if you are interested. Qianqian -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Nov 24 13:58:56 2021 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 24 Nov 2021 18:58:56 +0000 Subject: [Neuroimaging] Planning for data formats - upcoming journal club In-Reply-To: References: Message-ID: Hi, Just a reminder - meeting starting now, anyone welcome. Zoom link is https://bham-ac-uk.zoom.us/j/85378614517?pwd=Y09EU2hxNWdIOWNwWmwxOVpBVStMZz09 I'll record the meeting if y'all give me permission. Cheers, Matthew On Tue, Nov 9, 2021 at 11:32 AM Matthew Brett wrote: > > Hi, > > Summary: we are planning to do a journal club to discuss new > neuroimaging data formats. Please let me (Matthew) know if you want > to be included in the scheduling for the journal club. Of course, > once we've got a time, anyone can join the journal club Zoom. > > Detail: > > As part of the Chan-Zuckerberg grant work, we are thinking about new > formats for using and sharing neuroimaging data. > > As you can see in the grant documents: > > https://github.com/matthew-brett/czi-nibabel > > we've been thinking about HDF5, but further research (e.g. [1]) has > given us some pause for reflection. Meanwhile, the astronomy > community has done some serious thinking about this, and written this > up in a paper proposing a new astronomy format: > > Greenfield, M. Droettboom, E. Bray, > ASDF: A new data format for astronomy, > Astronomy and Computing, > Volume 12, > 2015, > Pages 240-251, > ISSN 2213-1337, > https://doi.org/10.1016/j.ascom.2015.06.004. > (https://www.sciencedirect.com/science/article/pii/S2213133715000645) > > Both the discussion and the format seem like a very good starting > point for our own thinking about data formats, so we'd like to have a > journal club to discuss it. > > I (Matthew) am going to organize this via Calendly, and I've made a > Caldendly account for Nipy-related stuff, but, if you want to have a > say in when we do the journal club, please let me know so I can add > you to that account. I'll publish the Calendly link in a day or so. > > Cheers, > > Matthew > > [1] https://cyrille.rossant.net/moving-away-hdf5/