From jni.soma at gmail.com  Sat Jul  1 03:07:42 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Sat, 1 Jul 2017 17:07:42 +1000
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
Message-ID: <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>

Hi Vighnesh,

There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and comes with a bunch of nice notebooks. I suspect it will be useful.

Juan.

On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>, wrote:
> I don?t know your status of knowledge, so I apologize if I state the obvious.
>
> Your issue is that of the co-registration of images.
>
> You have to always decide what your ?prime? or ?truth? is. How was your ?GIS? data precisely located? How do you know it?s not that which is off? (Even so, if you have ?ground truth? measurements, they are usually more precisely located than remote sensing data.
>
> Are the satellite images ?map-projected? ? If so, against what ground truth was that projection performed? What geographical coordinate system of Earth was used for the map projection?
>
> This stackexchange might help.
>
> There are a bazillion books out there for that, don?t really have a recommendation, mostly learned my stuff by internet searches. ;)
>
> HTH,
> Michael
>
>
> > On Jun 30, 2017, at 09:49, Vighnesh Birodkar <vighneshbirodkar at gmail.com> wrote:
> >
> > Hello
> >
> > Has anyone here worked with satellite images before ? I am dealing with the problem of aligning satellite images to GIS data which can be off by upto 15pixels. Could someone point me to good literature to read on the topic ?
> >
> > Thanks
> > Vighnesh
> > _______________________________________________
> > scikit-image mailing list
> > scikit-image at python.org
> > https://mail.python.org/mailman/listinfo/scikit-image
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/7fd72eed/attachment.html>

From vighneshbirodkar at gmail.com  Sat Jul  1 15:27:19 2017
From: vighneshbirodkar at gmail.com (Vighnesh Birodkar)
Date: Sat, 1 Jul 2017 15:27:19 -0400
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
 <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
Message-ID: <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>

Hello

Answering Michael's questions

The images I have are already ortho rectified. I don't want to co register
2 images. I want to register an image with some GIS data (like Open Street
Maps). So in that sense it is the problem of registering an image with a
vector of features.

Thanks for the pointer to the talk Juan.

Thanks
Vighnesh

On Sat, Jul 1, 2017 at 3:07 AM, Juan Nunez-Iglesias <jni.soma at gmail.com>
wrote:

> Hi Vighnesh,
>
> There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and
> comes with a bunch of nice notebooks. I suspect it will be useful.
>
> Juan.
>
> On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>,
> wrote:
>
> I don?t know your status of knowledge, so I apologize if I state the
> obvious.
>
> Your issue is that of the co-registration of images.
>
> You have to always decide what your ?prime? or ?truth? is. How was your
> ?GIS? data precisely located? How do you know it?s not that which is off?
> (Even so, if you have ?ground truth? measurements, they are usually more
> precisely located than remote sensing data.
>
> Are the satellite images ?map-projected? ? If so, against what ground
> truth was that projection performed? What geographical coordinate system of
> Earth was used for the map projection?
>
> This stackexchange might help.
>
> There are a bazillion books out there for that, don?t really have a
> recommendation, mostly learned my stuff by internet searches. ;)
>
> HTH,
> Michael
>
>
> On Jun 30, 2017, at 09:49, Vighnesh Birodkar <vighneshbirodkar at gmail.com>
> wrote:
>
> Hello
>
> Has anyone here worked with satellite images before ? I am dealing with
> the problem of aligning satellite images to GIS data which can be off by
> upto 15pixels. Could someone point me to good literature to read on the
> topic ?
>
> Thanks
> Vighnesh
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/44a7da03/attachment.html>

From robbmcleod at gmail.com  Sat Jul  1 16:06:42 2017
From: robbmcleod at gmail.com (Robert McLeod)
Date: Sat, 1 Jul 2017 13:06:42 -0700
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaUmU6IGJ1aWxkaW5nIHNjaWtp?=
	=?utf-8?q?t-image_plugin_and_tool_set=2E?=
In-Reply-To: <20170701011328.7CD0EC800B0@webmail.sinamail.sina.com.cn>
References: <20170701011328.7CD0EC800B0@webmail.sinamail.sina.com.cn>
Message-ID: <CAEFUWWWECDZ+wYNETLcxR9ftRiAVf1Li6drLN3EBH5NNaT3yWA@mail.gmail.com>

Just my two thoughts, I suspect this project would have more success if you
used OpenCV as the basis.  OpenCV already has many of the features you will
need to implement, such as Regions of Interest, in the core engine.  It
also already has Qt bindings.



On Fri, Jun 30, 2017 at 6:13 PM, <imagepy at sina.com> wrote:

> Hi Stefan and Dongyao:
>     I think the most difference between camera image, semiconductor
> industry and biology is the image format, and some specific algrism, As a
> plugin system, we just need to build general environment, supporting
> Numpy(uint8, int16, float32, float64), and some common mark(lookup table,
> scale ruler). about the difference, Developer in different fields can wrote
> their own reader plugins(such as pydicom).
>     bwt:I do not appreciate integrate ui and plugins in scikit-image,
> scikit-image should be pure a processing library.
>
>     About the ui, wxPhoenix has support Py2, Py3, on Win, Mac ,Linux. You
> can find Linux whl here
> <https://wxpython.org/Phoenix/snapshot-builds/linux/> and will be
> avalible in pypi soon. (In fact I had tested ImagePy on win, mac, linux
> with py2/py3) I think QT and WX are both OK, but wx is more light weighted
> (just 10M on win), It is friendly for user to download.
>
>     I also agree with That web is the best choice. But I have no energy to
> do it myself, I am not good at web, js, I just see the image show and
> parameter adjust in Jupyter's demo (but the most difficult is to edit the
> roi line, curve, circle, and polygon, edit the node, sometimes with holes).
> I would be very happy If I can get contact with their developer!
>
> Best
> YXDragon
>
> ----- ???? -----
> ????Dongyao Li <lidongyao.br at gmail.com>
> ????"Mailing list for scikit-image (http://scikit-image.org)" <
> scikit-image at python.org>
> ???Re: [scikit-image] building scikit-image plugin and tool set.
> ???2017?07?01? 03?40?
>
> Hi YXDragon and Stefan:
>
> I am a process engineer in semiconductor industry. I wrote a SEM/STEM/TEM
> analysis software with UI based on scikit-image, pyqt, and others, using
> the concept of plugins. I agree with Stefan's point that there is a market
> for this. However my experience is that the images from scientific
> instruments like SEM/STEM/TEM/MRI are quite different with images from
> cameras. And people in semiconductor industry and people working on biology
> have very different needs. I feel like the focus right now is mostly on the
> biology community.
>
> I am very happy to hear that we can possibly build a web-based UI for
> image analysis now. It will integrate the image analysis and further data
> analysis greatly, which is the our future direction. It would be great if
> scikit-image could include some demo of it in the future, just like the
> "viewer" package skimage has right now.
>
> Best,
> Dongyao
>
>
> On Fri, Jun 30, 2017 at 10:48 AM, Stefan van der Walt <
> stefanv at berkeley.edu> wrote:
>
> Hi, YXDragon
>
> On Fri, Jun 30, 2017, at 01:46, imagepy at sina.com wrote:
>
>      Now I think It is time to build a project named skimg-plgs
> <https://github.com/Image-Py/skimg-plgs>. And I had wrote some
> presentative demo <https://github.com/Image-Py/skimg-plgs> with friendly
> interact. these demo are all from the gallery, (interactive snake contours,
> interactive watershed...) I want to build a full plugin system and tool set
> covering scikit-image's every module. Then we can build a large user
> community like ImageJ, not only programers but also the doctors and
> biologists.
>
>
> Given the immense success of projects such as ImageJ, and based on
> feedback I've received from practitioners in microscopy, there is certainly
> a market for a GUI built on top of scikit-image.
>
> I don't personally have the time to contribute to such an effort at the
> moment (my focus is probably best spent keeping skimage on track!), but I
> am appreciative of the work you are doing, and encourage you to continue.
>
> I may have mentioned this to you before, but I am not sure about the
> long-term viability of WXPython.  I heard that they have a Phoenix rewrite
> underway, but looking at the latest PyPI release (
> https://pypi.python.org/pypi/wxPython/4.0.0a1) they also only support
> Windows and MacOS.
>
> With a Jupyter Lab release imminent, I would strongly consider using the
> web as UI, and building components that integrate with that project.  I can
> put you in contact with their developers, if you like.
>
> Best regards
> St?fan
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>


-- 
Robert McLeod, Ph.D.
robert.mcleod at unibas.ch
robert.mcleod at bsse.ethz.ch <robert.mcleod at ethz.ch>
robbmcleod at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/da98f1a0/attachment-0001.html>

From joferkington at gmail.com  Sat Jul  1 20:56:03 2017
From: joferkington at gmail.com (Joe Kington)
Date: Sat, 1 Jul 2017 19:56:03 -0500
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
 <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
 <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>
Message-ID: <CACe1pJd8ZC4+B14zkGh=9dWgKnehQr4O=+3Ddz5WyhM0XfWSiQ@mail.gmail.com>

You'll need to use geospatial software or libraries for this.  GDAL is your
best choice. gdal_translate with the -gcp option is what you want in this
case. You can do the equivalent from gdal's Python interface, but it's a
bit more involved. There's also a QGIS plugin to interactively build the
GDAL command, if you'd like to interactively select the coregistration
points.

On Jul 1, 2017 2:27 PM, "Vighnesh Birodkar" <vighneshbirodkar at gmail.com>
wrote:

> Hello
>
> Answering Michael's questions
>
> The images I have are already ortho rectified. I don't want to co register
> 2 images. I want to register an image with some GIS data (like Open Street
> Maps). So in that sense it is the problem of registering an image with a
> vector of features.
>
> Thanks for the pointer to the talk Juan.
>
> Thanks
> Vighnesh
>
> On Sat, Jul 1, 2017 at 3:07 AM, Juan Nunez-Iglesias <jni.soma at gmail.com>
> wrote:
>
>> Hi Vighnesh,
>>
>> There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and
>> comes with a bunch of nice notebooks. I suspect it will be useful.
>>
>> Juan.
>>
>> On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>,
>> wrote:
>>
>> I don?t know your status of knowledge, so I apologize if I state the
>> obvious.
>>
>> Your issue is that of the co-registration of images.
>>
>> You have to always decide what your ?prime? or ?truth? is. How was your
>> ?GIS? data precisely located? How do you know it?s not that which is off?
>> (Even so, if you have ?ground truth? measurements, they are usually more
>> precisely located than remote sensing data.
>>
>> Are the satellite images ?map-projected? ? If so, against what ground
>> truth was that projection performed? What geographical coordinate system of
>> Earth was used for the map projection?
>>
>> This stackexchange might help.
>>
>> There are a bazillion books out there for that, don?t really have a
>> recommendation, mostly learned my stuff by internet searches. ;)
>>
>> HTH,
>> Michael
>>
>>
>> On Jun 30, 2017, at 09:49, Vighnesh Birodkar <vighneshbirodkar at gmail.com>
>> wrote:
>>
>> Hello
>>
>> Has anyone here worked with satellite images before ? I am dealing with
>> the problem of aligning satellite images to GIS data which can be off by
>> upto 15pixels. Could someone point me to good literature to read on the
>> topic ?
>>
>> Thanks
>> Vighnesh
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/8014e2ef/attachment.html>

From omer.ozak at gmail.com  Sat Jul  1 21:35:21 2017
From: omer.ozak at gmail.com (=?utf-8?Q?=C3=96mer_=C3=96zak?=)
Date: Sat, 1 Jul 2017 20:35:21 -0500
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <CACe1pJd8ZC4+B14zkGh=9dWgKnehQr4O=+3Ddz5WyhM0XfWSiQ@mail.gmail.com>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
 <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
 <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>
 <CACe1pJd8ZC4+B14zkGh=9dWgKnehQr4O=+3Ddz5WyhM0XfWSiQ@mail.gmail.com>
Message-ID: <18D458FF-953F-4BE3-828D-0B80359CC698@gmail.com>

Hi Vigresh,

You can try Geopandas and Georasters which are specialized packages for GIS.

Best,

?mer

Sent from my mobile
Please excuse any typos

> On Jul 1, 2017, at 7:56 PM, Joe Kington <joferkington at gmail.com> wrote:
> 
> You'll need to use geospatial software or libraries for this.  GDAL is your best choice. gdal_translate with the -gcp option is what you want in this case. You can do the equivalent from gdal's Python interface, but it's a bit more involved. There's also a QGIS plugin to interactively build the GDAL command, if you'd like to interactively select the coregistration points.
> 
>> On Jul 1, 2017 2:27 PM, "Vighnesh Birodkar" <vighneshbirodkar at gmail.com> wrote:
>> Hello
>> 
>> Answering Michael's questions
>> 
>> The images I have are already ortho rectified. I don't want to co register 2 images. I want to register an image with some GIS data (like Open Street Maps). So in that sense it is the problem of registering an image with a vector of features. 
>> 
>> Thanks for the pointer to the talk Juan.
>> 
>> Thanks
>> Vighnesh
>> 
>>> On Sat, Jul 1, 2017 at 3:07 AM, Juan Nunez-Iglesias <jni.soma at gmail.com> wrote:
>>> Hi Vighnesh,
>>> 
>>> There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and comes with a bunch of nice notebooks. I suspect it will be useful.
>>> 
>>> Juan.
>>> 
>>>> On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>, wrote:
>>>> I don?t know your status of knowledge, so I apologize if I state the obvious.
>>>> 
>>>> Your issue is that of the co-registration of images.
>>>> 
>>>> You have to always decide what your ?prime? or ?truth? is. How was your ?GIS? data precisely located? How do you know it?s not that which is off? (Even so, if you have ?ground truth? measurements, they are usually more precisely located than remote sensing data.
>>>> 
>>>> Are the satellite images ?map-projected? ? If so, against what ground truth was that projection performed? What geographical coordinate system of Earth was used for the map projection?
>>>> 
>>>> This stackexchange might help.
>>>> 
>>>> There are a bazillion books out there for that, don?t really have a recommendation, mostly learned my stuff by internet searches. ;)
>>>> 
>>>> HTH,
>>>> Michael
>>>> 
>>>> 
>>>>> On Jun 30, 2017, at 09:49, Vighnesh Birodkar <vighneshbirodkar at gmail.com> wrote:
>>>>> 
>>>>> Hello
>>>>> 
>>>>> Has anyone here worked with satellite images before ? I am dealing with the problem of aligning satellite images to GIS data which can be off by upto 15pixels. Could someone point me to good literature to read on the topic ?
>>>>> 
>>>>> Thanks
>>>>> Vighnesh
>>>>> _______________________________________________
>>>>> scikit-image mailing list
>>>>> scikit-image at python.org
>>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>> 
>>>> _______________________________________________
>>>> scikit-image mailing list
>>>> scikit-image at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>> 
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>> 
>> 
>> 
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>> 
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/88e7c382/attachment-0001.html>

From vighneshbirodkar at gmail.com  Sat Jul  1 23:07:13 2017
From: vighneshbirodkar at gmail.com (Vighnesh Birodkar)
Date: Sat, 1 Jul 2017 23:07:13 -0400
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <18D458FF-953F-4BE3-828D-0B80359CC698@gmail.com>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
 <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
 <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>
 <CACe1pJd8ZC4+B14zkGh=9dWgKnehQr4O=+3Ddz5WyhM0XfWSiQ@mail.gmail.com>
 <18D458FF-953F-4BE3-828D-0B80359CC698@gmail.com>
Message-ID: <CAD82iYr17AWG799+Lts+dnRGM9zzVGFHywKqGkh3=jk=JP95=A@mail.gmail.com>

Thanks for the input Joe.

What you are suggesting seems to be manual annotation, are you aware of any
literature which does this automatically ?

Thanks
Vighnesh

On Sat, Jul 1, 2017 at 9:35 PM, ?mer ?zak <omer.ozak at gmail.com> wrote:

> Hi Vigresh,
>
> You can try Geopandas and Georasters which are specialized packages for
> GIS.
>
> Best,
>
> ?mer
>
> Sent from my mobile
> Please excuse any typos
>
> On Jul 1, 2017, at 7:56 PM, Joe Kington <joferkington at gmail.com> wrote:
>
> You'll need to use geospatial software or libraries for this.  GDAL is
> your best choice. gdal_translate with the -gcp option is what you want in
> this case. You can do the equivalent from gdal's Python interface, but it's
> a bit more involved. There's also a QGIS plugin to interactively build the
> GDAL command, if you'd like to interactively select the coregistration
> points.
>
> On Jul 1, 2017 2:27 PM, "Vighnesh Birodkar" <vighneshbirodkar at gmail.com>
> wrote:
>
>> Hello
>>
>> Answering Michael's questions
>>
>> The images I have are already ortho rectified. I don't want to co
>> register 2 images. I want to register an image with some GIS data (like
>> Open Street Maps). So in that sense it is the problem of registering an
>> image with a vector of features.
>>
>> Thanks for the pointer to the talk Juan.
>>
>> Thanks
>> Vighnesh
>>
>> On Sat, Jul 1, 2017 at 3:07 AM, Juan Nunez-Iglesias <jni.soma at gmail.com>
>> wrote:
>>
>>> Hi Vighnesh,
>>>
>>> There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and
>>> comes with a bunch of nice notebooks. I suspect it will be useful.
>>>
>>> Juan.
>>>
>>> On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>,
>>> wrote:
>>>
>>> I don?t know your status of knowledge, so I apologize if I state the
>>> obvious.
>>>
>>> Your issue is that of the co-registration of images.
>>>
>>> You have to always decide what your ?prime? or ?truth? is. How was your
>>> ?GIS? data precisely located? How do you know it?s not that which is off?
>>> (Even so, if you have ?ground truth? measurements, they are usually more
>>> precisely located than remote sensing data.
>>>
>>> Are the satellite images ?map-projected? ? If so, against what ground
>>> truth was that projection performed? What geographical coordinate system of
>>> Earth was used for the map projection?
>>>
>>> This stackexchange might help.
>>>
>>> There are a bazillion books out there for that, don?t really have a
>>> recommendation, mostly learned my stuff by internet searches. ;)
>>>
>>> HTH,
>>> Michael
>>>
>>>
>>> On Jun 30, 2017, at 09:49, Vighnesh Birodkar <vighneshbirodkar at gmail.com>
>>> wrote:
>>>
>>> Hello
>>>
>>> Has anyone here worked with satellite images before ? I am dealing with
>>> the problem of aligning satellite images to GIS data which can be off by
>>> upto 15pixels. Could someone point me to good literature to read on the
>>> topic ?
>>>
>>> Thanks
>>> Vighnesh
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>
>>>
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>
>>>
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>
>>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170701/73ca8712/attachment.html>

From joferkington at gmail.com  Sun Jul  2 14:05:58 2017
From: joferkington at gmail.com (Joe Kington)
Date: Sun, 2 Jul 2017 13:05:58 -0500
Subject: [scikit-image] Techniques for Aligning images to GIS data
In-Reply-To: <CAD82iYr17AWG799+Lts+dnRGM9zzVGFHywKqGkh3=jk=JP95=A@mail.gmail.com>
References: <CAD82iYo_Oa-OawmvzmeBd_mcNb1Xrf+W182U0Lti_6PO1Ok3fQ@mail.gmail.com>
 <007A1247-233E-42B9-AC82-CE93AE9311C5@gmail.com>
 <e8241eab-04d3-40b0-a4a1-09d713ad2c9a@Spark>
 <CAD82iYr6qdGpyj0LYcrm2yhZ4YdX4uotqa=AWOX+mWsFj-4ZmA@mail.gmail.com>
 <CACe1pJd8ZC4+B14zkGh=9dWgKnehQr4O=+3Ddz5WyhM0XfWSiQ@mail.gmail.com>
 <18D458FF-953F-4BE3-828D-0B80359CC698@gmail.com>
 <CAD82iYr17AWG799+Lts+dnRGM9zzVGFHywKqGkh3=jk=JP95=A@mail.gmail.com>
Message-ID: <CACe1pJdgvEUksyZNhybXtu90xz=4iC5mLkkLgn_56PvoJDogJQ@mail.gmail.com>

The automated approaches use other imagery as a reference. They rely on
matching various scale invariant features. I'm not aware of any open
implementations.

It's not impossible to roll your own, but it will require significant
effort and significant computing resources. Also, I'm guessing things are
already orthorectified, otherwise you'll need to take that into account.

Using vector datasets as a reference instead of other imagery will more or
less require you to do it manually, if accuracy is a concern at all.
Accurately segmenting roads on general imagery without manual intervention
is a very tough problem. It's easy to do for a few specific images and hard
to do in general. You'd need to be able to do that to use a vector road
dataset as a reference. Also keep in mind that vector features from most
sources are unlikely to be accurate to within more than 10m or so. (i.e.
about the width of the road) National level datasets will be even less
precise.

At any rate, aligning images to each other is an easier task than aligning
them to something like a road dataset. What are you trying to do?

On Jul 1, 2017 11:07 PM, "Vighnesh Birodkar" <vighneshbirodkar at gmail.com>
wrote:

> Thanks for the input Joe.
>
> What you are suggesting seems to be manual annotation, are you aware of
> any literature which does this automatically ?
>
> Thanks
> Vighnesh
>
> On Sat, Jul 1, 2017 at 9:35 PM, ?mer ?zak <omer.ozak at gmail.com> wrote:
>
>> Hi Vigresh,
>>
>> You can try Geopandas and Georasters which are specialized packages for
>> GIS.
>>
>> Best,
>>
>> ?mer
>>
>> Sent from my mobile
>> Please excuse any typos
>>
>> On Jul 1, 2017, at 7:56 PM, Joe Kington <joferkington at gmail.com> wrote:
>>
>> You'll need to use geospatial software or libraries for this.  GDAL is
>> your best choice. gdal_translate with the -gcp option is what you want in
>> this case. You can do the equivalent from gdal's Python interface, but it's
>> a bit more involved. There's also a QGIS plugin to interactively build the
>> GDAL command, if you'd like to interactively select the coregistration
>> points.
>>
>> On Jul 1, 2017 2:27 PM, "Vighnesh Birodkar" <vighneshbirodkar at gmail.com>
>> wrote:
>>
>>> Hello
>>>
>>> Answering Michael's questions
>>>
>>> The images I have are already ortho rectified. I don't want to co
>>> register 2 images. I want to register an image with some GIS data (like
>>> Open Street Maps). So in that sense it is the problem of registering an
>>> image with a vector of features.
>>>
>>> Thanks for the pointer to the talk Juan.
>>>
>>> Thanks
>>> Vighnesh
>>>
>>> On Sat, Jul 1, 2017 at 3:07 AM, Juan Nunez-Iglesias <jni.soma at gmail.com>
>>> wrote:
>>>
>>>> Hi Vighnesh,
>>>>
>>>> There was a talk by Kat Scott from Planet at PyCon 2017. It?s great and
>>>> comes with a bunch of nice notebooks. I suspect it will be useful.
>>>>
>>>> Juan.
>>>>
>>>> On 1 Jul 2017, 2:16 AM +1000, K.-Michael Aye <kmichael.aye at gmail.com>,
>>>> wrote:
>>>>
>>>> I don?t know your status of knowledge, so I apologize if I state the
>>>> obvious.
>>>>
>>>> Your issue is that of the co-registration of images.
>>>>
>>>> You have to always decide what your ?prime? or ?truth? is. How was your
>>>> ?GIS? data precisely located? How do you know it?s not that which is off?
>>>> (Even so, if you have ?ground truth? measurements, they are usually more
>>>> precisely located than remote sensing data.
>>>>
>>>> Are the satellite images ?map-projected? ? If so, against what ground
>>>> truth was that projection performed? What geographical coordinate system of
>>>> Earth was used for the map projection?
>>>>
>>>> This stackexchange might help.
>>>>
>>>> There are a bazillion books out there for that, don?t really have a
>>>> recommendation, mostly learned my stuff by internet searches. ;)
>>>>
>>>> HTH,
>>>> Michael
>>>>
>>>>
>>>> On Jun 30, 2017, at 09:49, Vighnesh Birodkar <
>>>> vighneshbirodkar at gmail.com> wrote:
>>>>
>>>> Hello
>>>>
>>>> Has anyone here worked with satellite images before ? I am dealing with
>>>> the problem of aligning satellite images to GIS data which can be off by
>>>> upto 15pixels. Could someone point me to good literature to read on the
>>>> topic ?
>>>>
>>>> Thanks
>>>> Vighnesh
>>>> _______________________________________________
>>>> scikit-image mailing list
>>>> scikit-image at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>>
>>>>
>>>> _______________________________________________
>>>> scikit-image mailing list
>>>> scikit-image at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>>
>>>>
>>>> _______________________________________________
>>>> scikit-image mailing list
>>>> scikit-image at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>>
>>>>
>>>
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>
>>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170702/6a75ccb6/attachment.html>

From stefanv at berkeley.edu  Fri Jul  7 18:59:46 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Fri, 07 Jul 2017 15:59:46 -0700
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaUmU6IGJ1aWxkaW5nIHNjaWtp?=
 =?utf-8?q?t-image_plugin_and_tool_set=2E?=
In-Reply-To: <CAEFUWWWECDZ+wYNETLcxR9ftRiAVf1Li6drLN3EBH5NNaT3yWA@mail.gmail.com>
References: <20170701011328.7CD0EC800B0@webmail.sinamail.sina.com.cn>
 <CAEFUWWWECDZ+wYNETLcxR9ftRiAVf1Li6drLN3EBH5NNaT3yWA@mail.gmail.com>
Message-ID: <1499468386.3040303.1034031752.32C6D796@webmail.messagingengine.com>

On Sat, Jul 1, 2017, at 13:06, Robert McLeod wrote:
> Just my two thoughts, I suspect this project would have more success if you used OpenCV as the basis.  OpenCV already has many of the features you will need to implement, such as Regions of Interest, in the core engine.  It also already has Qt bindings.  
I think the idea is to wrap many different tools this way.  E.g., on the project README I saw SIFT features computed through OpenCV:
https://github.com/Image-Py/imagepy

St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170707/80be6f7f/attachment.html>

From stefanv at berkeley.edu  Fri Jul  7 19:17:10 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Fri, 07 Jul 2017 16:17:10 -0700
Subject: [scikit-image] Parallel processing segmentation while keeping
 labels
In-Reply-To: <CAJE86+FoPXXMemurLxg+ocvVz1R+-=z7xM2-cH=a=mC7QurOYg@mail.gmail.com>
References: <CAJE86+G-ZxNtE7ai41k9XLW35NL5CUBmr4L=jDj2Ekf2YA=vfw@mail.gmail.com>
 <99e57691-6899-4273-8fa3-a831247bfcbc@Spark>
 <CAJE86+FoPXXMemurLxg+ocvVz1R+-=z7xM2-cH=a=mC7QurOYg@mail.gmail.com>
Message-ID: <1499469430.3043075.1034042568.056E5A94@webmail.messagingengine.com>

On Wed, Jun 28, 2017, at 15:37, Cedric Espenel wrote:
> 2) jolib for parallel processing (I'd like to use skimage.util.apply_parallel but I'm not sure how I can use a fonction with more than 1 arg?) 
extra_arguments and extra_keywords may do the trick.

http://scikit-image.org/docs/dev/api/skimage.util.html#skimage.util.apply_parallel
St?fan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170707/2230acd4/attachment.html>

From jni.soma at gmail.com  Mon Jul 10 22:07:12 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Tue, 11 Jul 2017 12:07:12 +1000
Subject: [scikit-image] Python 3, revisited
Message-ID: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>

Hi everyone,

I?d like to revisit the Python 2 deprecation issue. Since the last discussion, IPython has gone Python 3-only, and Astropy just announced their 2.0 release as being the last one to support Python 2, with 3.0, scheduled in six months, out in 6 months.

I?ve been obsessively cataloguing all Python 2-related annoyances that have cropped up since our last discussion. (See below for the full list.) As expected, nothing was a show-stopper, but they do add up to something non-trivial.

I?d very much like a roadmap along these lines:

* 0.14, due before end of the year: last version to support Python 2.
* 1.0, due by SciPy next year: Python 3.5+ only (or even 3.6, though I definitely won?t push on this ? but f-strings are so nice! =)

Thoughts? [*ducks*] =P

Juan.

??????????

Several rounds of reviewing this because of a difference in how Python 2.7 and 3.x handle warnings:
https://github.com/scikit-image/scikit-image/pull/2327

Beautiful linalg syntax ruined by using Python 2.7
https://github.com/scikit-image/scikit-image/pull/2394#discussion_r94191032

Unicode character in __init__.py caused a delay in PR 1357.
https://github.com/scikit-image/scikit-image/pull/1357#issuecomment-249601485

Use of "range" causes MemoryError in Python 2:
https://travis-ci.org/scikit-image/scikit-image/jobs/188150849#L3345

Merging dictionaries is unnecessarily verbose and ugly in this PR
https://github.com/scikit-image/scikit-image/pull/2455/files
http://treyhunner.com/2016/02/how-to-merge-dictionaries-in-python/

Pretty terrible linalg in:
https://github.com/scikit-image/scikit-image/pull/2482/files#diff-e2663b696b69140e0947e91cb6c61137R533

Bad API choices because we have to either change argument order relative to ndi or run a 2-release deprecation cycle; issue would be avoided if we used Python 3.5 only.
https://github.com/scikit-image/scikit-image/pull/2508#discussion_r101049062

conda-forge release 0.13 delayed by 2.7 build error on Windows.
https://github.com/conda-forge/scikit-image-feedstock/pull/8#issuecomment-290413922

Python 2.7 lists don?t have a `.copy()` method
https://github.com/scikit-image/scikit-image/pull/2584#discussion_r111285939

Whatever this is (six.get_self_method):
https://github.com/scikit-image/scikit-image/pull/1522/files#diff-c0f33f5600401be736e79b65164b4c81R775

Concurrent futures not available:
https://github.com/scikit-image/scikit-image/pull/2647#discussion_r114763804

ASCII codec can?t encode em-dash
'ascii' codec can't encode character u'\u2014' in position 41: ordinal not in range(128)
https://travis-ci.org/jni/scikit-image/jobs/234223563#L3529




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170711/9b0e4e3b/attachment.html>

From mattwcraig at gmail.com  Mon Jul 10 23:58:53 2017
From: mattwcraig at gmail.com (Matt Craig)
Date: Mon, 10 Jul 2017 22:58:53 -0500
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
Message-ID: <CAKxiDYm6U8xatQNSH4EWDPpOJSdhvdORebnpjf411jJi8BxjgA@mail.gmail.com>

Hi,

Astropy dev here -- just to clarify a little bit, bug fixes will be
backported to the 2.0 release for two years, so until June 2019.

Very much looking forward to 3.0 in 6 months though!

Matt


On Mon, Jul 10, 2017 at 9:07 PM, Juan Nunez-Iglesias <jni.soma at gmail.com>
wrote:

> Hi everyone,
>
> I?d like to revisit the Python 2 deprecation issue. Since the last
> discussion, IPython has gone Python 3-only, and Astropy just announced
> <https://twitter.com/astropy/status/884402856945733634> their 2.0 release
> as being the last one to support Python 2, with 3.0, scheduled in six
> months, out in 6 months.
>
> I?ve been obsessively cataloguing all Python 2-related annoyances that
> have cropped up since our last discussion. (See below for the full list.)
> As expected, nothing was a show-stopper, but they do add up to something
> non-trivial.
>
> I?d very much like a roadmap along these lines:
>
> * 0.14, due before end of the year: last version to support Python 2.
> * 1.0, due by SciPy next year: Python 3.5+ only (or even 3.6, though I
> definitely won?t push on this ? but f-strings are so nice! =)
>
> Thoughts? [*ducks*] =P
>
> Juan.
>
> ??????????
>
> Several rounds of reviewing this because of a difference in how Python 2.7
> and 3.x handle warnings:
> https://github.com/scikit-image/scikit-image/pull/2327
>
> Beautiful linalg syntax ruined by using Python 2.7
> https://github.com/scikit-image/scikit-image/pull/2394#
> discussion_r94191032
>
> Unicode character in __init__.py caused a delay in PR 1357.
> https://github.com/scikit-image/scikit-image/pull/1357#
> issuecomment-249601485
>
> Use of "range" causes MemoryError in Python 2:
> https://travis-ci.org/scikit-image/scikit-image/jobs/188150849#L3345
>
> Merging dictionaries is unnecessarily verbose and ugly in this PR
> https://github.com/scikit-image/scikit-image/pull/2455/files
> http://treyhunner.com/2016/02/how-to-merge-dictionaries-in-python/
>
> Pretty terrible linalg in:
> https://github.com/scikit-image/scikit-image/pull/2482/files#diff-
> e2663b696b69140e0947e91cb6c61137R533
>
> Bad API choices because we have to either change argument order relative
> to ndi or run a 2-release deprecation cycle; issue would be avoided if we
> used Python 3.5 only.
> https://github.com/scikit-image/scikit-image/pull/2508#
> discussion_r101049062
>
> conda-forge release 0.13 delayed by 2.7 build error on Windows.
> https://github.com/conda-forge/scikit-image-feedstock/
> pull/8#issuecomment-290413922
>
> Python 2.7 lists don?t have a `.copy()` method
> https://github.com/scikit-image/scikit-image/pull/2584#
> discussion_r111285939
>
> Whatever this is (six.get_self_method):
> https://github.com/scikit-image/scikit-image/pull/1522/files#diff-
> c0f33f5600401be736e79b65164b4c81R775
>
> Concurrent futures not available:
> https://github.com/scikit-image/scikit-image/pull/2647#
> discussion_r114763804
>
> ASCII codec can?t encode em-dash
> 'ascii' codec can't encode character u'\u2014' in position 41: ordinal not
> in range(128)
> https://travis-ci.org/jni/scikit-image/jobs/234223563#L3529
>
>
>
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170710/8d6c50b3/attachment.html>

From tcaswell at gmail.com  Tue Jul 11 00:34:06 2017
From: tcaswell at gmail.com (Thomas Caswell)
Date: Tue, 11 Jul 2017 04:34:06 +0000
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
Message-ID: <CAA48SF9s4ikQ4dpvQFTXkHg8+G_qn3=qnwrsiU=vuJKDtKNvrQ@mail.gmail.com>

It is on the Matplotlib roadmap to have a python3 only release targeted for
SciPy 2018 as well.

Python 3.7 is scheduled for June 2018 (
https://www.python.org/dev/peps/pep-0537/) so targeting {3.6, 3.7} as the
supported versions of python (which at that point should be the two most
recent) is not a bad plan *ducks*.

Tom

On Mon, Jul 10, 2017 at 9:22 PM Juan Nunez-Iglesias <jni.soma at gmail.com>
wrote:

> Hi everyone,
>
> I?d like to revisit the Python 2 deprecation issue. Since the last
> discussion, IPython has gone Python 3-only, and Astropy just announced
> <https://twitter.com/astropy/status/884402856945733634> their 2.0 release
> as being the last one to support Python 2, with 3.0, scheduled in six
> months, out in 6 months.
>
> I?ve been obsessively cataloguing all Python 2-related annoyances that
> have cropped up since our last discussion. (See below for the full list.)
> As expected, nothing was a show-stopper, but they do add up to something
> non-trivial.
>
> I?d very much like a roadmap along these lines:
>
> * 0.14, due before end of the year: last version to support Python 2.
> * 1.0, due by SciPy next year: Python 3.5+ only (or even 3.6, though I
> definitely won?t push on this ? but f-strings are so nice! =)
>
> Thoughts? [*ducks*] =P
>
> Juan.
>
> ??????????
>
> Several rounds of reviewing this because of a difference in how Python 2.7
> and 3.x handle warnings:
> https://github.com/scikit-image/scikit-image/pull/2327
>
> Beautiful linalg syntax ruined by using Python 2.7
> https://github.com/scikit-image/scikit-image/pull/2394#discussion_r94191032
>
> Unicode character in __init__.py caused a delay in PR 1357.
>
> https://github.com/scikit-image/scikit-image/pull/1357#issuecomment-249601485
>
> Use of "range" causes MemoryError in Python 2:
> https://travis-ci.org/scikit-image/scikit-image/jobs/188150849#L3345
>
> Merging dictionaries is unnecessarily verbose and ugly in this PR
> https://github.com/scikit-image/scikit-image/pull/2455/files
> http://treyhunner.com/2016/02/how-to-merge-dictionaries-in-python/
>
> Pretty terrible linalg in:
>
> https://github.com/scikit-image/scikit-image/pull/2482/files#diff-e2663b696b69140e0947e91cb6c61137R533
>
> Bad API choices because we have to either change argument order relative
> to ndi or run a 2-release deprecation cycle; issue would be avoided if we
> used Python 3.5 only.
>
> https://github.com/scikit-image/scikit-image/pull/2508#discussion_r101049062
>
> conda-forge release 0.13 delayed by 2.7 build error on Windows.
>
> https://github.com/conda-forge/scikit-image-feedstock/pull/8#issuecomment-290413922
>
> Python 2.7 lists don?t have a `.copy()` method
>
> https://github.com/scikit-image/scikit-image/pull/2584#discussion_r111285939
>
> Whatever this is (six.get_self_method):
>
> https://github.com/scikit-image/scikit-image/pull/1522/files#diff-c0f33f5600401be736e79b65164b4c81R775
>
> Concurrent futures not available:
>
> https://github.com/scikit-image/scikit-image/pull/2647#discussion_r114763804
>
> ASCII codec can?t encode em-dash
> 'ascii' codec can't encode character u'\u2014' in position 41: ordinal not
> in range(128)
> https://travis-ci.org/jni/scikit-image/jobs/234223563#L3529
>
>
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170711/9e23e792/attachment-0001.html>

From stefanv at berkeley.edu  Tue Jul 11 00:47:46 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Mon, 10 Jul 2017 21:47:46 -0700
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
Message-ID: <1499748466.3042078.1036846400.65CF27AD@webmail.messagingengine.com>

Hi Juan

On Mon, Jul 10, 2017, at 19:07, Juan Nunez-Iglesias wrote:
> I?d like to revisit the Python 2 deprecation issue. Since the last discussion, IPython has gone Python 3-only, and Astropy just announced[1] their 2.0 release as being the last one to support Python 2, with 3.0, scheduled in six months, out in 6 months.
As you know, I am excited to see the transition to 3.5.  Our last conversation on the matter is archived here:
https://mail.python.org/pipermail//scikit-image/2016-August/004807.html
It would be interesting to see how many of those arguments still hold.

St?fan


Links:

  1. https://twitter.com/astropy/status/884402856945733634
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170710/b411016e/attachment.html>

From jni.soma at gmail.com  Tue Jul 11 03:43:10 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Tue, 11 Jul 2017 17:43:10 +1000
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <1499748466.3042078.1036846400.65CF27AD@webmail.messagingengine.com>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
 <1499748466.3042078.1036846400.65CF27AD@webmail.messagingengine.com>
Message-ID: <59fa3947-a6c8-4e57-99a2-883f2a85214b@Spark>

Thanks for the input, Matt! Yes, if you look at our original discussion, my idea would be to continue to do bug-fix releases on the 0.14 branch for some time, but only provide new features on the Python 3 master.

On 11 Jul 2017, 2:48 PM +1000, Stefan van der Walt <stefanv at berkeley.edu>, wrote:
> Hi Juan
>
> On Mon, Jul 10, 2017, at 19:07, Juan Nunez-Iglesias wrote:
> > I?d like to revisit the Python 2 deprecation issue. Since the last discussion, IPython has gone Python 3-only, and Astropy just announced their 2.0 release as being the last one to support Python 2, with 3.0, scheduled in six months, out in 6 months.
>
> As you know, I am excited to see the transition to 3.5.? Our last conversation on the matter is archived here:
>
> https://mail.python.org/pipermail//scikit-image/2016-August/004807.html
>
> It would be interesting to see how many of those arguments still hold.
>
> St?fan
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170711/61994095/attachment.html>

From jni.soma at gmail.com  Tue Jul 11 03:44:46 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Tue, 11 Jul 2017 17:44:46 +1000
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <1499748466.3042078.1036846400.65CF27AD@webmail.messagingengine.com>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
 <1499748466.3042078.1036846400.65CF27AD@webmail.messagingengine.com>
Message-ID: <0d00c15b-5bea-4536-a651-4645e5bb2777@Spark>

Oh and thanks Tom, adding Matplotlib to the list of Python 3-only libs will be a massive boost!

On 11 Jul 2017, 2:48 PM +1000, Stefan van der Walt <stefanv at berkeley.edu>, wrote:
> Hi Juan
>
> On Mon, Jul 10, 2017, at 19:07, Juan Nunez-Iglesias wrote:
> > I?d like to revisit the Python 2 deprecation issue. Since the last discussion, IPython has gone Python 3-only, and Astropy just announced their 2.0 release as being the last one to support Python 2, with 3.0, scheduled in six months, out in 6 months.
>
> As you know, I am excited to see the transition to 3.5.? Our last conversation on the matter is archived here:
>
> https://mail.python.org/pipermail//scikit-image/2016-August/004807.html
>
> It would be interesting to see how many of those arguments still hold.
>
> St?fan
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170711/29b7418c/attachment.html>

From matthew.brett at gmail.com  Tue Jul 11 04:42:32 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Tue, 11 Jul 2017 09:42:32 +0100
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
Message-ID: <CAH6Pt5pC-B+MkPUozpxKheYG3GM=+CX0iLpuw6SRWcQ3hykw2g@mail.gmail.com>

Hi,

Certainly no need to duck, from my point view - but for this:

On Tue, Jul 11, 2017 at 3:07 AM, Juan Nunez-Iglesias
> Beautiful linalg syntax ruined by using Python 2.7
> https://github.com/scikit-image/scikit-image/pull/2394#discussion_r94191032

- do you mean differences like:

D1 = np.vstack([x.dot(x), x.dot(y), y.dot(y)]).T

compared to:

D1 = np.vstack([x @ x, x @ y, y @ y]).T

?

Cheers,

Matthew

From nelle.varoquaux at gmail.com  Tue Jul 11 13:13:49 2017
From: nelle.varoquaux at gmail.com (Nelle Varoquaux)
Date: Tue, 11 Jul 2017 10:13:49 -0700
Subject: [scikit-image] Sprint @ scipy: ticket triage and gathering ideas
Message-ID: <CAE-UAvTQ=0AyAi48ZPxuckXCPPcK34Kw9wf_cDNzir9sEikpwQ@mail.gmail.com>

Hi folks,

Scikit-image will be hosting a sprint this week-end at scipy. In order
to welcome new contributors, it'd be nice if we could gather ideas and
start triaging tickets. Contact me or St?fan if you have any
questions.

Also, expect an increase in pull request activity! This might be a
good opportunity to make progress and close some old pull requests.

Cheers,
N

From jni.soma at gmail.com  Tue Jul 11 22:45:06 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Wed, 12 Jul 2017 12:45:06 +1000
Subject: [scikit-image] Python 3, revisited
In-Reply-To: <CAH6Pt5pC-B+MkPUozpxKheYG3GM=+CX0iLpuw6SRWcQ3hykw2g@mail.gmail.com>
References: <606d2dcb-cf88-44ea-a4fd-3f51a4bd057c@Spark>
 <CAH6Pt5pC-B+MkPUozpxKheYG3GM=+CX0iLpuw6SRWcQ3hykw2g@mail.gmail.com>
Message-ID: <5a93e2d1-168a-4657-acff-18e708092e53@Spark>

Yes Matthew, that is exactly my issue with that line (and many other linear algebra-heavy parts of skimage).

On 11 Jul 2017, 6:43 PM +1000, Matthew Brett <matthew.brett at gmail.com>, wrote:
> Hi,
>
> Certainly no need to duck, from my point view - but for this:
>
> On Tue, Jul 11, 2017 at 3:07 AM, Juan Nunez-Iglesias
> > Beautiful linalg syntax ruined by using Python 2.7
> > https://github.com/scikit-image/scikit-image/pull/2394#discussion_r94191032
>
> - do you mean differences like:
>
> D1 = np.vstack([x.dot(x), x.dot(y), y.dot(y)]).T
>
> compared to:
>
> D1 = np.vstack([x @ x, x @ y, y @ y]).T
>
> ?
>
> Cheers,
>
> Matthew
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170712/e6dd1f09/attachment.html>

From martin.fleck at uni-konstanz.de  Wed Jul 12 13:29:31 2017
From: martin.fleck at uni-konstanz.de (Martin Fleck)
Date: Wed, 12 Jul 2017 19:29:31 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
Message-ID: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>

Hello skimage users and devs,

in an image analysis of mine, I use skimage.measure.label and
skimage.morphology of  skimage 0.13.0 .

My problem my analysis uses much more memory than I expect.
I attached output from the memory_profiler package, with which I tried
to keep track of the memory consumption of my analysis.
You can see that for an ~8MiB file that I used for testing,
skimage.measure.label needs to use 56MiB of memory, which surprised me.
My first question is: Am I doing something wrong here?
The memory consumption is skimage.morphology.remove_small_objects is
surprisingly high, too.

I want to be able to process images of several GiB on my machine.
I hope I can excuse the following comparison with matlab by pointing to
the similarity between skimage.measure.regionprops and matlabs regionprops:
With Matlab I don't run into any memory problems on the very same machine.

I would be very glad if I could continue using python for this analysis
and not have to rely on matlab.
Is there a way for me to reduce the memory consumption if I want to use
skimage.morphology.remove_small_objects() and
skimage.measure.regionprops() for which I need the label()?

Best Regards,
Martin
-------------- next part --------------
Line #    Mem usage    Increment   Line Contents
================================================
   154  102.523 MiB    0.000 MiB   @profile
   155                             def run():
   156  110.500 MiB    7.977 MiB       image = data.imread(image_filename)
   157  110.500 MiB    0.000 MiB       image = color.rgb2gray(image)
   158                                 
   159                                 # make binary image with threshold using Li's method
   160  117.328 MiB    6.828 MiB       bwSource = image<filters.threshold_minimum(image)
   161                                 
   162                                 # Filter bwSource image -> remove small specks
   163  131.715 MiB   14.387 MiB       bwFiltered = morphology.remove_small_objects(bwSource, min_size=minSingleDislocationArea, connectivity=2)
   164                                 
   165                                 # analyze regions:
   166  187.551 MiB   55.836 MiB       label_img = label(bwFiltered)
   167  187.809 MiB    0.258 MiB       regions = regionprops(label_img)


From jaime.frio at gmail.com  Wed Jul 12 15:49:49 2017
From: jaime.frio at gmail.com (=?UTF-8?Q?Jaime_Fern=C3=A1ndez_del_R=C3=ADo?=)
Date: Wed, 12 Jul 2017 21:49:49 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
Message-ID: <CAPOWHWmeaMeK1u69qYjqasys+5Pv-cNbvFjxy2bNqWONsyWZ-A@mail.gmail.com>

On Wed, Jul 12, 2017 at 7:29 PM, Martin Fleck <martin.fleck at uni-konstanz.de>
wrote:

> Hello skimage users and devs,
>
> in an image analysis of mine, I use skimage.measure.label and
> skimage.morphology of  skimage 0.13.0 .
>

I have no answer to your question, but if anyone more knowledgeable on
skimage internals could confirm if any of these operations rely on
scipy.ndimage functionality, I could try to look at potential improvements
there.


>
> My problem my analysis uses much more memory than I expect.
> I attached output from the memory_profiler package, with which I tried
> to keep track of the memory consumption of my analysis.
> You can see that for an ~8MiB file that I used for testing,
> skimage.measure.label needs to use 56MiB of memory, which surprised me.
> My first question is: Am I doing something wrong here?
> The memory consumption is skimage.morphology.remove_small_objects is
> surprisingly high, too.
>
> I want to be able to process images of several GiB on my machine.
> I hope I can excuse the following comparison with matlab by pointing to
> the similarity between skimage.measure.regionprops and matlabs regionprops:
> With Matlab I don't run into any memory problems on the very same machine.
>
> I would be very glad if I could continue using python for this analysis
> and not have to rely on matlab.
> Is there a way for me to reduce the memory consumption if I want to use
> skimage.morphology.remove_small_objects() and
> skimage.measure.regionprops() for which I need the label()?
>
> Best Regards,
> Martin
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>


-- 
(\__/)
( O.o)
( > <) Este es Conejo. Copia a Conejo en tu firma y ay?dale en sus planes
de dominaci?n mundial.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170712/cc24d782/attachment.html>

From grlee77 at gmail.com  Wed Jul 12 19:58:43 2017
From: grlee77 at gmail.com (Gregory Lee)
Date: Wed, 12 Jul 2017 19:58:43 -0400
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
Message-ID: <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>

Hi Martin,

My problem my analysis uses much more memory than I expect.
> I attached output from the memory_profiler package, with which I tried
> to keep track of the memory consumption of my analysis.
> You can see that for an ~8MiB file that I used for testing,
> skimage.measure.label needs to use 56MiB of memory, which surprised me.
>

I haven't looked at it in much detail, but I did find what appear to be
some unnecessary copies in the top-level Cython routine called by
skimage.morphology.label.  I opened a PR to try and avoid this here:
https://github.com/scikit-image/scikit-image/pull/2701

However, I think that PR is going to give a minor performance improvement,
but not help with memory use much if at all.  I think the main reason for
the increased memory usage is that the output type of the label function is
int64 while your input is most likely uint8.  This means that the labels
array requires 8 times the memory usage of the uint8 input.  I don't think
there is much way around that without making a version of the routines that
allows specifying a smaller integer dtype.

- Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170712/32f9fd02/attachment.html>

From jni.soma at gmail.com  Wed Jul 12 21:05:10 2017
From: jni.soma at gmail.com (Juan Nunez-Iglesias)
Date: Thu, 13 Jul 2017 11:05:10 +1000
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
Message-ID: <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>

Hi Martin,

No one on this list wants to push you to more Matlab usage, believe me. ;)

Do you think you could provide a script and sample data that we can use for troubleshooting? As Greg pointed out, the optimization approach *might* have to be data-type dependent. We could, for example, provide a dtype= keyword argument that would force the output to be of a particular, more memory-efficient type, if you know in advance how many objects you expect.

If you can provide something similar to a memory profile, and diagnostic information, for your equivalent Matlab script, that would be really useful, so we know what we are aiming for. For example, what are the data types of the outputs in Matlab?

Juan.

On 13 Jul 2017, 9:59 AM +1000, Gregory Lee <grlee77 at gmail.com>, wrote:
> Hi Martin,
>
> > > My problem my analysis uses much more memory than I expect.
> > > I attached output from the memory_profiler package, with which I tried
> > > to keep track of the memory consumption of my analysis.
> > > You can see that for an ~8MiB file that I used for testing,
> > > skimage.measure.label needs to use 56MiB of memory, which surprised me.
> >
> > I haven't looked at it in much detail, but I did find what appear to be some unnecessary copies in the top-level Cython routine called by skimage.morphology.label.? I opened a PR to try and avoid this here:
> > https://github.com/scikit-image/scikit-image/pull/2701
> >
> > However, I think that PR is going to give a minor performance improvement, but not help with memory use much if at all.? I think the main reason for the increased memory usage is that the output type of the label function is int64 while your input is most likely uint8.? This means that the labels array requires 8 times the memory usage of the uint8 input.? I don't think there is much way around that without making a version of the routines that allows specifying a smaller integer dtype.
> >
> - Greg
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/7f9ed6d6/attachment-0001.html>

From stefanv at berkeley.edu  Thu Jul 13 02:16:37 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Wed, 12 Jul 2017 23:16:37 -0700
Subject: [scikit-image] Numba on pypi
Message-ID: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>

Hi everyone,

As many of you know, speed has been a point of contention in
scikit-image for a long time.  We've made a very deliberate decision to
focus on writing high-level, understandable code (via Python and
Cython): both to lower the barrier to entry for newcomers, and to lessen
the burden on maintainers.  But execution time comparisons, vs OpenCV
e.g., left much to be desired.

I think we have hit a turning point in the road.  Binary wheels for
Numba (actually, llvmlite) were recently uploaded to PyPi, making this
technology available to users on both pip and conda installations.  The
importance of this release on pypi should not be dismissed, and I am
grateful to the numba team and Continuum for making that decision.

So, how does that impact scikit-image?  Well, imagine we choose to
optimize various procedures via numba (see Juan's blog post for exactly
how impactful this can be:
https://ilovesymposia.com/2017/03/15/prettier-lowlevelcallables-with-numba-jit-and-decorators/).
The only question we have to answer (from a survival point of view)
needs to be: if, somehow, something happens to numba, will an
alternative will be available at that time?  Looking at the Python JIT
landscape (which is very active), and the current state of numba
development, I think this is likely.  And, if we choose to use numba, of
course we'll help to keep it healthy, as far as we can.

I'd love to hear your thoughts.  I, for one, am excited about the
prospect of writing kernels as simply as:

>>> @jit_filter_function
... def fmin(values):
...     result = np.inf
...     for v in values:
...         if v < result:
...             result = v
...     return result

>>> ndi.generic_filter(image, fmin, footprint=fp)

Best regards
St?fan

From imagepy at sina.com  Thu Jul 13 02:49:12 2017
From: imagepy at sina.com (imagepy at sina.com)
Date: Thu, 13 Jul 2017 14:49:12 +0800
Subject: [scikit-image] =?gbk?b?u9i4tKO6IE51bWJhIG9uIHB5cGk=?=
Message-ID: <20170713064912.63C55B000D7@webmail.sinamail.sina.com.cn>

Hi St?fan:
  I appreciate Numba. for sometimes, we must do a 'for' in our python code, but just a 'for' with a 'if', It is fussy to compile a so/dll or write cython. Numba is very portable, and can run anywhere, just need to install numba and llvmlite. (That means our package could be a light-weight library, undepended any native so/dll)
  As I metioned befor, many scikit-image's algrisms are not exquisite enough(just my own opinion),      the mid_axi function results too many branch and sometimes with hols,        the local_max has not a tolerance, so the result is too massy, then do a watershed end with too many fragments...,      and how to build a graph from the skeleton, then do a network analysis.
I want to do a contribute to scikit-image, But after some effort, I give up, I prefor to write a dynamic lib rather then Cython. In the end, I wrote them in Numba. So I appreciate to use Numba.
BestYXDragon
----- ???? -----
????Stefan van der Walt <stefanv at berkeley.edu>
????scikit-image at python.org
???[scikit-image] Numba on pypi
???2017?07?13? 14?17?

Hi everyone,
As many of you know, speed has been a point of contention in
scikit-image for a long time.  We've made a very deliberate decision to
focus on writing high-level, understandable code (via Python and
Cython): both to lower the barrier to entry for newcomers, and to lessen
the burden on maintainers.  But execution time comparisons, vs OpenCV
e.g., left much to be desired.
I think we have hit a turning point in the road.  Binary wheels for
Numba (actually, llvmlite) were recently uploaded to PyPi, making this
technology available to users on both pip and conda installations.  The
importance of this release on pypi should not be dismissed, and I am
grateful to the numba team and Continuum for making that decision.
So, how does that impact scikit-image?  Well, imagine we choose to
optimize various procedures via numba (see Juan's blog post for exactly
how impactful this can be:
https://ilovesymposia.com/2017/03/15/prettier-lowlevelcallables-with-numba-jit-and-decorators/).
The only question we have to answer (from a survival point of view)
needs to be: if, somehow, something happens to numba, will an
alternative will be available at that time?  Looking at the Python JIT
landscape (which is very active), and the current state of numba
development, I think this is likely.  And, if we choose to use numba, of
course we'll help to keep it healthy, as far as we can.
I'd love to hear your thoughts.  I, for one, am excited about the
prospect of writing kernels as simply as:
>>> @jit_filter_function
... def fmin(values):
...     result = np.inf
...     for v in values:
...         if v < result:
...             result = v
...     return result
>>> ndi.generic_filter(image, fmin, footprint=fp)
Best regards
St?fan
_______________________________________________
scikit-image mailing list
scikit-image at python.org
https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/ac2d8577/attachment.html>

From ralf.gommers at gmail.com  Thu Jul 13 03:50:39 2017
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Thu, 13 Jul 2017 19:50:39 +1200
Subject: [scikit-image] Numba on pypi
In-Reply-To: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
Message-ID: <CABL7CQign+++EtqZqyJG88FEnG2w_dRps5WD3M_E5r=0YQMgGg@mail.gmail.com>

On Thu, Jul 13, 2017 at 6:16 PM, Stefan van der Walt <stefanv at berkeley.edu>
wrote:

> Hi everyone,
>
> As many of you know, speed has been a point of contention in
> scikit-image for a long time.  We've made a very deliberate decision to
> focus on writing high-level, understandable code (via Python and
> Cython): both to lower the barrier to entry for newcomers, and to lessen
> the burden on maintainers.  But execution time comparisons, vs OpenCV
> e.g., left much to be desired.
>
> I think we have hit a turning point in the road.  Binary wheels for
> Numba (actually, llvmlite) were recently uploaded to PyPi, making this
> technology available to users on both pip and conda installations.  The
> importance of this release on pypi should not be dismissed, and I am
> grateful to the numba team and Continuum for making that decision.
>

Agreed. Note that there are no Windows wheels up on PyPI (yet, or not
coming?). Given that there are no SciPy wheels for Windows either I don't
think that that changes your argument much - people should just use a
binary distribution on Windows - but I thought I'd point it out anway.

>
> So, how does that impact scikit-image?  Well, imagine we choose to
> optimize various procedures via numba (see Juan's blog post for exactly
> how impactful this can be:
> https://ilovesymposia.com/2017/03/15/prettier-
> lowlevelcallables-with-numba-jit-and-decorators/).
>

That's a great post. @Juan: get yourself on Planet Python!


> The only question we have to answer (from a survival point of view)
> needs to be: if, somehow, something happens to numba, will an
> alternative will be available at that time?  Looking at the Python JIT
> landscape (which is very active), and the current state of numba
> development, I think this is likely.  And, if we choose to use numba, of
> course we'll help to keep it healthy, as far as we can.
>
> I'd love to hear your thoughts.


I'm only an occasional user at the moment, so won't express an opinion
either way. But will be following this thread with interest.

Cheers,
Ralf


> I, for one, am excited about the
> prospect of writing kernels as simply as:
>
> >>> @jit_filter_function
> ... def fmin(values):
> ...     result = np.inf
> ...     for v in values:
> ...         if v < result:
> ...             result = v
> ...     return result
>
> >>> ndi.generic_filter(image, fmin, footprint=fp)
>
> Best regards
> St?fan
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/f3e8eafb/attachment-0001.html>

From matthew.brett at gmail.com  Thu Jul 13 04:10:24 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Thu, 13 Jul 2017 09:10:24 +0100
Subject: [scikit-image] Numba on pypi
In-Reply-To: <CABL7CQign+++EtqZqyJG88FEnG2w_dRps5WD3M_E5r=0YQMgGg@mail.gmail.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
 <CABL7CQign+++EtqZqyJG88FEnG2w_dRps5WD3M_E5r=0YQMgGg@mail.gmail.com>
Message-ID: <CAH6Pt5q0tGXmswEmv5XFL8DUXwMTR_bioab0PYud9eapTodgKw@mail.gmail.com>

Hi,

On Thu, Jul 13, 2017 at 8:50 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Thu, Jul 13, 2017 at 6:16 PM, Stefan van der Walt <stefanv at berkeley.edu>
> wrote:
>>
>> Hi everyone,
>>
>> As many of you know, speed has been a point of contention in
>> scikit-image for a long time.  We've made a very deliberate decision to
>> focus on writing high-level, understandable code (via Python and
>> Cython): both to lower the barrier to entry for newcomers, and to lessen
>> the burden on maintainers.  But execution time comparisons, vs OpenCV
>> e.g., left much to be desired.
>>
>> I think we have hit a turning point in the road.  Binary wheels for
>> Numba (actually, llvmlite) were recently uploaded to PyPi, making this
>> technology available to users on both pip and conda installations.  The
>> importance of this release on pypi should not be dismissed, and I am
>> grateful to the numba team and Continuum for making that decision.
>
>
> Agreed. Note that there are no Windows wheels up on PyPI (yet, or not
> coming?). Given that there are no SciPy wheels for Windows either I don't
> think that that changes your argument much - people should just use a binary
> distribution on Windows - but I thought I'd point it out anway.

We might be close to a working scipy wheel - discussion evolving over
at https://github.com/scipy/scipy/issues/7551#issuecomment-314922271

If we do succeed, that would make the lack of a numba wheel for
Windows much more significant.

Does anyone know Continuum's plans in this matter?   Is the numba
wheel recipe open-source?

Cheers,

Matthew

From Thomas.Edgar.Walter at googlemail.com  Thu Jul 13 04:04:53 2017
From: Thomas.Edgar.Walter at googlemail.com (Thomas Walter)
Date: Thu, 13 Jul 2017 10:04:53 +0200
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaIE51bWJhIG9uIHB5cGk=?=
In-Reply-To: <20170713064912.63C55B000D7@webmail.sinamail.sina.com.cn>
References: <20170713064912.63C55B000D7@webmail.sinamail.sina.com.cn>
Message-ID: <185229e7-59bd-37e7-4c1f-fd29d5f7b19e@googlemail.com>

Hi YXDragon,

just a word on some aspect you mention:

 > the local_max has not a tolerance, so the result is too massy, then 
do a watershed end with too many fragments...,

The definition that underlies this function is: "A local maximum is a 
region of constant grey level strictly greater than the grey levels of 
all pixels in direct neighborhood of the region." - There is no 
tolerance associated to this definition, and I am perfectly fine with 
this. There are definitely many cases where you want to extract all 
local maxima. Nevertheless, there are other functions for detection of 
maxima that allow one to also impose certain criteria (such as h_maxima 
or peak_local_max).

A question to St?fan: would this mean that you would remove all cython 
code from scikit-image or would numba just be another option?

Best,

Thomas.

On 7/13/17 8:49 AM, imagepy at sina.com wrote:
> Hi St?fan:
>
>   I appreciate Numba. for sometimes, we must do a 'for' in our python 
> code, but just a 'for' with a 'if', It is fussy to compile a so/dll or 
> write cython. Numba is very portable, and can run anywhere, just need 
> to install numba and llvmlite. (That means our package could be a 
> light-weight library, undepended any native so/dll)
>
>   As I metioned befor, many scikit-image's algrisms are not exquisite 
> enough(just my own opinion),
>       the mid_axi function results too many branch and sometimes with 
> hols,
>       the local_max has not a tolerance, so the result is too massy, 
> then do a watershed end with too many fragments...,
>       and how to build a graph from the skeleton, then do a network 
> analysis.
>
> I want to do a contribute to scikit-image, But after some effort, I 
> give up, I prefor to write a dynamic lib rather then Cython. In the 
> end, I wrote them in Numba. So I appreciate to use Numba.
>
> Best
> YXDragon
>
> ----- ???? -----
> ????Stefan van der Walt <stefanv at berkeley.edu>
> ????scikit-image at python.org
> ???[scikit-image] Numba on pypi
> ???2017?07?13? 14?17?
>
> Hi everyone,
> As many of you know, speed has been a point of contention in
> scikit-image for a long time. We've made a very deliberate decision to
> focus on writing high-level, understandable code (via Python and
> Cython): both to lower the barrier to entry for newcomers, and to lessen
> the burden on maintainers. But execution time comparisons, vs OpenCV
> e.g., left much to be desired.
> I think we have hit a turning point in the road. Binary wheels for
> Numba (actually, llvmlite) were recently uploaded to PyPi, making this
> technology available to users on both pip and conda installations. The
> importance of this release on pypi should not be dismissed, and I am
> grateful to the numba team and Continuum for making that decision.
> So, how does that impact scikit-image? Well, imagine we choose to
> optimize various procedures via numba (see Juan's blog post for exactly
> how impactful this can be:
> https://ilovesymposia.com/2017/03/15/prettier-lowlevelcallables-with-numba-jit-and-decorators/).
> The only question we have to answer (from a survival point of view)
> needs to be: if, somehow, something happens to numba, will an
> alternative will be available at that time? Looking at the Python JIT
> landscape (which is very active), and the current state of numba
> development, I think this is likely. And, if we choose to use numba, of
> course we'll help to keep it healthy, as far as we can.
> I'd love to hear your thoughts. I, for one, am excited about the
> prospect of writing kernels as simply as:
> >>> @jit_filter_function
> ... def fmin(values):
> ... result = np.inf
> ... for v in values:
> ... if v < result:
> ... result = v
> ... return result
> >>> ndi.generic_filter(image, fmin, footprint=fp)
> Best regards
> St?fan
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image


-- 
Thomas Walter
27 rue des Acacias
75017 Paris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/a796c9d5/attachment.html>

From stefanv at berkeley.edu  Thu Jul 13 04:30:09 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Thu, 13 Jul 2017 01:30:09 -0700
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaIE51bWJhIG9uIHB5cGk=?=
In-Reply-To: <185229e7-59bd-37e7-4c1f-fd29d5f7b19e@googlemail.com>
References: <20170713064912.63C55B000D7@webmail.sinamail.sina.com.cn>
 <185229e7-59bd-37e7-4c1f-fd29d5f7b19e@googlemail.com>
Message-ID: <1499934609.2841260.1039469008.7B9A4BD6@webmail.messagingengine.com>

Hi Thomas

On Thu, Jul 13, 2017, at 01:04, Thomas Walter via scikit-image wrote:
>  A question to St?fan: would this mean that you would remove all cython code from scikit-image or would numba just be another option? 
I think we'd probably keep both around; one of the advantages of this approach is that we can gradually explore new functionality with numba, while retaining all existing code until there's a strong need to modify for speed/simplicity.  Cython & numba also do not fulfill exactly the same roles.  E.g., Cython is great for wrapping C and C++  libraries.
Best regards
St?fan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/1e5f3cda/attachment.html>

From imagepy at sina.com  Thu Jul 13 06:04:17 2017
From: imagepy at sina.com (imagepy at sina.com)
Date: Thu, 13 Jul 2017 18:04:17 +0800
Subject: [scikit-image] =?gbk?b?u9i4tKO6UmU6ICC72Li0o7ogTnVtYmEgb24gcHlw?=
 =?gbk?q?i?=
Message-ID: <20170713100417.EF0F6B000D8@webmail.sinamail.sina.com.cn>

Hi ThomasI know local_max has no tolerance, And peak_local_max has some parameter based on distance and abs value,(which the watershed demo in gallery used, but when the area is complex, the result is very massy). the h_max is what I need, but the result is unexpected somewhere, and It is too slow (the same image h_max cost 2.6s, but I wrote in numba cost 0.1s)
May I start a new topic thread? with the test image, and my numba code.
BestYXDragon----- ???? -----
????Thomas Walter via scikit-image <scikit-image at python.org>
????scikit-image at python.org
????Thomas Walter <Thomas.Edgar.Walter at googlemail.com>
???Re: [scikit-image] ??? Numba on pypi
???2017?07?13? 16?25?


  
  
    Hi YXDragon, 

      

      just a word on some aspect you mention: 

      

      > the local_max has not a tolerance, so the result is too
      massy, then do a watershed end with too many fragments...,
      

      

      The definition that underlies this function is: "A local maximum
      is a region of constant grey level strictly greater than the grey
      levels of all pixels in direct neighborhood of the region." -
      There is no tolerance associated to this definition, and I am
      perfectly fine with this. There are definitely many cases where
      you want to extract all local maxima. Nevertheless, there are
      other functions for detection of maxima that allow one to also
      impose certain criteria (such as h_maxima or peak_local_max).

      

      A question to St?fan: would this mean that you would remove all
      cython code from scikit-image or would numba just be another
      option? 

      

      Best, 

      

      Thomas. 

      

      On 7/13/17 8:49 AM, imagepy at sina.com wrote:

    
    
      Hi St?fan:
      

      
        I appreciate Numba. for sometimes, we must do a 'for' in
        our python code, but just a 'for' with a 'if', It is fussy to
        compile a so/dll or write cython. Numba is very portable, and
        can run anywhere, just need to install numba and llvmlite. (That
        means our package could be a light-weight library, undepended
        any native so/dll)
      

      
        As I metioned befor, many scikit-image's algrisms are
        not exquisite enough(just my own opinion),
            the mid_axi function results too many branch and
        sometimes with hols,  
            the local_max has not a tolerance, so the result is too
        massy, then do a watershed end with too many fragments...,
            and how to build a graph from the skeleton, then do a
        network analysis.
      

      
      I want to do a contribute to scikit-image, But after some
        effort, I give up, I prefor to write a dynamic lib rather then
        Cython. In the end, I wrote them in Numba. So I appreciate to
        use Numba.
      

      
      Best
      YXDragon
      

      
      
        ----- ???? -----

          ????Stefan van der Walt <stefanv at berkeley.edu>

          ????scikit-image at python.org

          ???[scikit-image] Numba on pypi

          ???2017?07?13? 14?17?

        
        

        Hi everyone,

        As many of you know, speed has been a point of contention in

        scikit-image for a long time. We've made a very deliberate
        decision to

        focus on writing high-level, understandable code (via Python and

        Cython): both to lower the barrier to entry for newcomers, and
        to lessen

        the burden on maintainers. But execution time comparisons, vs
        OpenCV

        e.g., left much to be desired.

        I think we have hit a turning point in the road. Binary wheels
        for

        Numba (actually, llvmlite) were recently uploaded to PyPi,
        making this

        technology available to users on both pip and conda
        installations. The

        importance of this release on pypi should not be dismissed, and
        I am

        grateful to the numba team and Continuum for making that
        decision.

        So, how does that impact scikit-image? Well, imagine we choose
        to

        optimize various procedures via numba (see Juan's blog post for
        exactly

        how impactful this can be:

https://ilovesymposia.com/2017/03/15/prettier-lowlevelcallables-with-numba-jit-and-decorators/).

        The only question we have to answer (from a survival point of
        view)

        needs to be: if, somehow, something happens to numba, will an

        alternative will be available at that time? Looking at the
        Python JIT

        landscape (which is very active), and the current state of numba

        development, I think this is likely. And, if we choose to use
        numba, of

        course we'll help to keep it healthy, as far as we can.

        I'd love to hear your thoughts. I, for one, am excited about the

        prospect of writing kernels as simply as:

        >>> @jit_filter_function

        ... def fmin(values):

        ... result = np.inf

        ... for v in values:

        ... if v < result:

        ... result = v

        ... return result

        >>> ndi.generic_filter(image, fmin, footprint=fp)

        Best regards

        St?fan

        _______________________________________________

        scikit-image mailing list

        scikit-image at python.org

        https://mail.python.org/mailman/listinfo/scikit-image

      
      

      
      

      _______________________________________________
scikit-image mailing list
scikit-image at python.org
https://mail.python.org/mailman/listinfo/scikit-image

    
    

    

    
    -- 
Thomas Walter
27 rue des Acacias
75017 Paris
  


_______________________________________________
scikit-image mailing list
scikit-image at python.org
https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/5a28e3ef/attachment-0001.html>

From martin.fleck at uni-konstanz.de  Thu Jul 13 07:21:58 2017
From: martin.fleck at uni-konstanz.de (Martin Fleck)
Date: Thu, 13 Jul 2017 13:21:58 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
 <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
Message-ID: <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>

Hi Juan, hi Greg,

quoting Greg:
> I think the main reason for the increased memory usage is that the
output type of the label function is int64 while your input is most
likely uint8.

Indeed, this could be the complete problem already! For the analysis I
use a binary image - so only one bit per pixel.

Greg: Regarding your PR and my analysis: My analysis using a 1.2GB file
stops due to memory problems already in
skimage.morphology.remove_small_objects() even if the major memory
blowup happens with skimage.morphology.label().
So there are problems at multiple steps that hopefully can be improved.

Quoting Juan:
> For example, what are the data types of the outputs in Matlab?

the first steps of my analysis are to convert the 8 bit input image to a
meaningful binary image. The whole analysis is done on binary images. So
all inputs and outputs in Matlab are of Matlab Class "logical".

I will provide you with a minimal example script and data for the
skimage case.
I will try to create equivalent memory inofrmation in Matlab.

I'll both post it here as soon as I'm done with that.

Thanks so far!

Martin

On 07/13/2017 03:05 AM, Juan Nunez-Iglesias wrote:
> Hi Martin,
>
> No one on this list wants to push you to more Matlab usage, believe me. ;)
>
> Do you think you could provide a script and sample data that we can
> use for troubleshooting? As Greg pointed out, the optimization
> approach *might* have to be data-type dependent. We could, for
> example, provide a dtype= keyword argument that would force the output
> to be of a particular, more memory-efficient type, if you know in
> advance how many objects you expect.
>
> If you can provide something similar to a memory profile, and
> diagnostic information, for your equivalent Matlab script, that would
> be really useful, so we know what we are aiming for. For example, what
> are the data types of the outputs in Matlab?
>
> Juan.
>
> On 13 Jul 2017, 9:59 AM +1000, Gregory Lee <grlee77 at gmail.com>, wrote:
>> Hi Martin,
>>
>>     My problem my analysis uses much more memory than I expect.
>>     I attached output from the memory_profiler package, with which I
>>     tried
>>     to keep track of the memory consumption of my analysis.
>>     You can see that for an ~8MiB file that I used for testing,
>>     skimage.measure.label needs to use 56MiB of memory, which
>>     surprised me.
>>
>>
>> I haven't looked at it in much detail, but I did find what appear to
>> be some unnecessary copies in the top-level Cython routine called by
>> skimage.morphology.label.  I opened a PR to try and avoid this here:
>> https://github.com/scikit-image/scikit-image/pull/2701
>> <https://github.com/scikit-image/scikit-image/pull/2701>
>>
>> However, I think that PR is going to give a minor performance
>> improvement, but not help with memory use much if at all.  I think
>> the main reason for the increased memory usage is that the output
>> type of the label function is int64 while your input is most likely
>> uint8.  This means that the labels array requires 8 times the memory
>> usage of the uint8 input.  I don't think there is much way around
>> that without making a version of the routines that allows specifying
>> a smaller integer dtype.
>>
>> - Greg
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/66e603d8/attachment.html>

From martin.fleck at uni-konstanz.de  Thu Jul 13 08:09:38 2017
From: martin.fleck at uni-konstanz.de (Martin Fleck)
Date: Thu, 13 Jul 2017 14:09:38 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
 <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
 <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
Message-ID: <340079c0-c64a-f45d-1320-5975dc7b0012@uni-konstanz.de>

Hi again,

here you can download a minimal example:

https://drive.google.com/open?id=0BzmlODsuIIz0elpIcU1kdmpNTlE
(download button is the arrow on the top right)

In order to run it and get the memory_profiler output you have to
install memory_profiler
e.g. with

pip3 install memory_profiler

and run the file with

python3 -m memory_profiler minimal_test.py

If you just want to run the example without memory profiling and
installing memory_profiler, you have to comment out or remove line 8
"@profile"

Cheers,
Martin



On 07/13/2017 01:21 PM, Martin Fleck wrote:
>
> Hi Juan, hi Greg,
>
> quoting Greg:
> > I think the main reason for the increased memory usage is that the
> output type of the label function is int64 while your input is most
> likely uint8.
>
> Indeed, this could be the complete problem already! For the analysis I
> use a binary image - so only one bit per pixel.
>
> Greg: Regarding your PR and my analysis: My analysis using a 1.2GB
> file stops due to memory problems already in
> skimage.morphology.remove_small_objects() even if the major memory
> blowup happens with skimage.morphology.label().
> So there are problems at multiple steps that hopefully can be improved.
>
> Quoting Juan:
> > For example, what are the data types of the outputs in Matlab?
>
> the first steps of my analysis are to convert the 8 bit input image to
> a meaningful binary image. The whole analysis is done on binary
> images. So all inputs and outputs in Matlab are of Matlab Class "logical".
>
> I will provide you with a minimal example script and data for the
> skimage case.
> I will try to create equivalent memory inofrmation in Matlab.
>
> I'll both post it here as soon as I'm done with that.
>
> Thanks so far!
>
> Martin
>
> On 07/13/2017 03:05 AM, Juan Nunez-Iglesias wrote:
>> Hi Martin,
>>
>> No one on this list wants to push you to more Matlab usage, believe
>> me. ;)
>>
>> Do you think you could provide a script and sample data that we can
>> use for troubleshooting? As Greg pointed out, the optimization
>> approach *might* have to be data-type dependent. We could, for
>> example, provide a dtype= keyword argument that would force the
>> output to be of a particular, more memory-efficient type, if you know
>> in advance how many objects you expect.
>>
>> If you can provide something similar to a memory profile, and
>> diagnostic information, for your equivalent Matlab script, that would
>> be really useful, so we know what we are aiming for. For example,
>> what are the data types of the outputs in Matlab?
>>
>> Juan.
>>
>> On 13 Jul 2017, 9:59 AM +1000, Gregory Lee <grlee77 at gmail.com>, wrote:
>>> Hi Martin,
>>>
>>>     My problem my analysis uses much more memory than I expect.
>>>     I attached output from the memory_profiler package, with which I
>>>     tried
>>>     to keep track of the memory consumption of my analysis.
>>>     You can see that for an ~8MiB file that I used for testing,
>>>     skimage.measure.label needs to use 56MiB of memory, which
>>>     surprised me.
>>>
>>>
>>> I haven't looked at it in much detail, but I did find what appear to
>>> be some unnecessary copies in the top-level Cython routine called by
>>> skimage.morphology.label.  I opened a PR to try and avoid this here:
>>> https://github.com/scikit-image/scikit-image/pull/2701
>>> <https://github.com/scikit-image/scikit-image/pull/2701>
>>>
>>> However, I think that PR is going to give a minor performance
>>> improvement, but not help with memory use much if at all.  I think
>>> the main reason for the increased memory usage is that the output
>>> type of the label function is int64 while your input is most likely
>>> uint8.  This means that the labels array requires 8 times the memory
>>> usage of the uint8 input.  I don't think there is much way around
>>> that without making a version of the routines that allows specifying
>>> a smaller integer dtype.
>>>
>>> - Greg
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/c13e0ea8/attachment-0001.html>

From martin.fleck at uni-konstanz.de  Thu Jul 13 10:03:31 2017
From: martin.fleck at uni-konstanz.de (Martin Fleck)
Date: Thu, 13 Jul 2017 16:03:31 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <340079c0-c64a-f45d-1320-5975dc7b0012@uni-konstanz.de>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
 <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
 <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
 <340079c0-c64a-f45d-1320-5975dc7b0012@uni-konstanz.de>
Message-ID: <cd1c1d30-1776-dc06-6ca0-653b242a382f@uni-konstanz.de>

Hi again,

attached is a file "matlab_memory_info" and again the same
"skiamge_memory_profiler.out" that I showed before.
in the matlab_memory_info file, I added for every matlab call the
equivalent that I do in skimage.

I don't think it will be needed - the attached files should be enough -
but if someone wants to see the full memory report of matlab, you can
download it here:
https://drive.google.com/open?id=0BzmlODsuIIz0dVRsMk9sT3RuU0E
(it's an html file)

Cheers,
Martin


On 07/13/2017 02:09 PM, Martin Fleck wrote:
>
> Hi again,
>
> here you can download a minimal example:
>
> https://drive.google.com/open?id=0BzmlODsuIIz0elpIcU1kdmpNTlE
> (download button is the arrow on the top right)
>
> In order to run it and get the memory_profiler output you have to
> install memory_profiler
> e.g. with
>
> pip3 install memory_profiler
>
> and run the file with
>
> python3 -m memory_profiler minimal_test.py
>
> If you just want to run the example without memory profiling and
> installing memory_profiler, you have to comment out or remove line 8
> "@profile"
>
> Cheers,
> Martin
>
>
>
> On 07/13/2017 01:21 PM, Martin Fleck wrote:
>>
>> Hi Juan, hi Greg,
>>
>> quoting Greg:
>> > I think the main reason for the increased memory usage is that the
>> output type of the label function is int64 while your input is most
>> likely uint8.
>>
>> Indeed, this could be the complete problem already! For the analysis
>> I use a binary image - so only one bit per pixel.
>>
>> Greg: Regarding your PR and my analysis: My analysis using a 1.2GB
>> file stops due to memory problems already in
>> skimage.morphology.remove_small_objects() even if the major memory
>> blowup happens with skimage.morphology.label().
>> So there are problems at multiple steps that hopefully can be improved.
>>
>> Quoting Juan:
>> > For example, what are the data types of the outputs in Matlab?
>>
>> the first steps of my analysis are to convert the 8 bit input image
>> to a meaningful binary image. The whole analysis is done on binary
>> images. So all inputs and outputs in Matlab are of Matlab Class
>> "logical".
>>
>> I will provide you with a minimal example script and data for the
>> skimage case.
>> I will try to create equivalent memory inofrmation in Matlab.
>>
>> I'll both post it here as soon as I'm done with that.
>>
>> Thanks so far!
>>
>> Martin
>>
>> On 07/13/2017 03:05 AM, Juan Nunez-Iglesias wrote:
>>> Hi Martin,
>>>
>>> No one on this list wants to push you to more Matlab usage, believe
>>> me. ;)
>>>
>>> Do you think you could provide a script and sample data that we can
>>> use for troubleshooting? As Greg pointed out, the optimization
>>> approach *might* have to be data-type dependent. We could, for
>>> example, provide a dtype= keyword argument that would force the
>>> output to be of a particular, more memory-efficient type, if you
>>> know in advance how many objects you expect.
>>>
>>> If you can provide something similar to a memory profile, and
>>> diagnostic information, for your equivalent Matlab script, that
>>> would be really useful, so we know what we are aiming for. For
>>> example, what are the data types of the outputs in Matlab?
>>>
>>> Juan.
>>>
>>> On 13 Jul 2017, 9:59 AM +1000, Gregory Lee <grlee77 at gmail.com>, wrote:
>>>> Hi Martin,
>>>>
>>>>     My problem my analysis uses much more memory than I expect.
>>>>     I attached output from the memory_profiler package, with which
>>>>     I tried
>>>>     to keep track of the memory consumption of my analysis.
>>>>     You can see that for an ~8MiB file that I used for testing,
>>>>     skimage.measure.label needs to use 56MiB of memory, which
>>>>     surprised me.
>>>>
>>>>
>>>> I haven't looked at it in much detail, but I did find what appear
>>>> to be some unnecessary copies in the top-level Cython routine
>>>> called by skimage.morphology.label.  I opened a PR to try and avoid
>>>> this here:
>>>> https://github.com/scikit-image/scikit-image/pull/2701
>>>> <https://github.com/scikit-image/scikit-image/pull/2701>
>>>>
>>>> However, I think that PR is going to give a minor performance
>>>> improvement, but not help with memory use much if at all.  I think
>>>> the main reason for the increased memory usage is that the output
>>>> type of the label function is int64 while your input is most likely
>>>> uint8.  This means that the labels array requires 8 times the
>>>> memory usage of the uint8 input.  I don't think there is much way
>>>> around that without making a version of the routines that allows
>>>> specifying a smaller integer dtype.
>>>>
>>>> - Greg
>>>> _______________________________________________
>>>> scikit-image mailing list
>>>> scikit-image at python.org
>>>> https://mail.python.org/mailman/listinfo/scikit-image
>>>
>>>
>>> _______________________________________________
>>> scikit-image mailing list
>>> scikit-image at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/d09303c6/attachment.html>
-------------- next part --------------
Line #    Mem usage    Increment   Line Contents
================================================
   154  102.523 MiB    0.000 MiB   @profile
   155                             def run():
   156  110.500 MiB    7.977 MiB       image = data.imread(image_filename)
   157  110.500 MiB    0.000 MiB       image = color.rgb2gray(image)
   158                                 
   159                                 # make binary image with threshold using Li's method
   160  117.328 MiB    6.828 MiB       bwSource = image<filters.threshold_minimum(image)
   161                                 
   162                                 # Filter bwSource image -> remove small specks
   163  131.715 MiB   14.387 MiB       bwFiltered = morphology.remove_small_objects(bwSource, min_size=minSingleDislocationArea, connectivity=2)
   164                                 
   165                                 # analyze regions:
   166  187.551 MiB   55.836 MiB       label_img = label(bwFiltered)
   167  187.809 MiB    0.258 MiB       regions = regionprops(label_img)

-------------- next part --------------
FunctionName	EquivalentCallInSkimage					Calls	TotalTime	SelfTime*	AllocatedMemory	FreedMemory	SelfMemory	PeakMemory	TotalTimePlot
imread		skimage.data.imread()					1	0.178 s		0.003 s		8770.72 Kb	612.52 Kb	42.67 Kb	7598.20 Kb	
imbinarize	IMAGE<TRESHOLD_VALUE					1	0.035 s		0.001 s		6705.84 Kb	58.42 Kb	14.33 Kb	6472.17 Kb	
bwareaopen	skimage.morphology.remove_small_objects()		1	0.720 s		0.032 s		17420.83 Kb	7763.23 Kb	5011.38 Kb	7147.20 Kb	
regionprops	skimage.measure.regionprops(skiamge.measure.label)	1	19.697 s	0.018 s		93376.56 Kb	96408.75 Kb	-11470.08 Kb	1602.05 Kb

From stefanv at berkeley.edu  Thu Jul 13 14:23:40 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Thu, 13 Jul 2017 11:23:40 -0700
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaUmU6ICDlm57lpI3vvJogTnVt?=
 =?utf-8?q?ba_on_pypi?=
In-Reply-To: <20170713100417.EF0F6B000D8@webmail.sinamail.sina.com.cn>
References: <20170713100417.EF0F6B000D8@webmail.sinamail.sina.com.cn>
Message-ID: <1499970220.710173.1040057416.0C16B1CA@webmail.messagingengine.com>

On Thu, Jul 13, 2017, at 03:04, imagepy at sina.com wrote:
> May I start a new topic thread? with the test image, and my numba code.
Yes, please.

St?fan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/3220ce44/attachment.html>

From stefanv at berkeley.edu  Thu Jul 13 14:26:00 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Thu, 13 Jul 2017 11:26:00 -0700
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
 <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
 <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
Message-ID: <1499970360.710411.1040059024.2AB12C17@webmail.messagingengine.com>

On Thu, Jul 13, 2017, at 04:21, Martin Fleck wrote:
> Indeed, this could be the complete problem already! For the analysis I use a binary image - so only one bit per pixel.
FWIW, binary images are stored as ubyte, so 1 *byte* per pixel.

>  Greg: Regarding your PR and my analysis: My analysis using a 1.2GB file stops due to memory problems already in skimage.morphology.remove_small_objects() even if the major memory blowup happens with skimage.morphology.label().>  So there are problems at multiple steps that hopefully can be improved.
Thanks for helping us uncover and trace these!

St?fan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/aa05f4ca/attachment.html>

From silvertrumpet999 at gmail.com  Thu Jul 13 14:37:45 2017
From: silvertrumpet999 at gmail.com (Josh Warner)
Date: Thu, 13 Jul 2017 11:37:45 -0700
Subject: [scikit-image] =?utf-8?b?5Zue5aSN77yaUmU6IOWbnuWkje+8miBOdW1i?=
	=?utf-8?q?a_on_pypi?=
In-Reply-To: <CADnryEe923J9LH=7m5sh=nswDu=YmOgxp-uVB2mct3LF=amp3w@mail.gmail.com>
References: <20170713100417.EF0F6B000D8@webmail.sinamail.sina.com.cn>
 <1499970220.710173.1040057416.0C16B1CA@webmail.messagingengine.com>
 <CADnryEe923J9LH=7m5sh=nswDu=YmOgxp-uVB2mct3LF=amp3w@mail.gmail.com>
Message-ID: <CADnryEcmiZscy5D7jX5-fOgN4ks_YwqUyPRwnq_QgoueR86bhQ@mail.gmail.com>

What I find numba brings to the table is a significantly more expressive
and easier to maintain code base.  Essentially writing basic bare bones
loops often are easiest to JIT.  They are then easier to debug, and faster
to iterate on thanks to removing the compilation step.

I haven't checked recently if numba is happy with certain very useful
features like np.nditer - if so, actually a large chunk of our Cython could
be replaced by simpler JITted code while making it n-D.

That said, Cython isn't going anywhere for the reasons St?fan mentions
above... but I believe as soon as it is reasonable adding numba would yield
large benefits. Probably even more so than we fully appreciate.

Looks like we're nearing that point.

Josh


On Jul 13, 2017 11:23 AM, "Stefan van der Walt" <stefanv at berkeley.edu>
wrote:

On Thu, Jul 13, 2017, at 03:04, imagepy at sina.com wrote:

May I start a new topic thread? with the test image, and my numba code.


Yes, please.

St?fan


_______________________________________________
scikit-image mailing list
scikit-image at python.org
https://mail.python.org/mailman/listinfo/scikit-image
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/82de9096/attachment.html>

From martin.fleck at uni-konstanz.de  Thu Jul 13 15:56:07 2017
From: martin.fleck at uni-konstanz.de (Martin Fleck)
Date: Thu, 13 Jul 2017 21:56:07 +0200
Subject: [scikit-image] Memory consumption of measure.label (compared to
 matlab)
In-Reply-To: <1499970360.710411.1040059024.2AB12C17@webmail.messagingengine.com>
References: <edb35ef0-639a-a38e-6ce2-134a6dd39a89@uni-konstanz.de>
 <CAJR3sXfJQQ7VCYGLR_6xokLHFm-JaOjod42CF3YNK5FwNCBfAQ@mail.gmail.com>
 <270b8fba-ac6b-46f0-b4f0-f1b45e52b73f@Spark>
 <f4ca524e-b4e9-35d9-0cd0-b7e5977deb05@uni-konstanz.de>
 <1499970360.710411.1040059024.2AB12C17@webmail.messagingengine.com>
Message-ID: <bee24bea-5b2c-05c7-1456-7bff190d832c@uni-konstanz.de>

Oh one more small thing comes to my mind:

In the minimal example skimage code that I gave, I included two test
files: testImage.png and testImageTiny.png
The *Tiny.png strangely does not show the memory blowup at all. I don't
understand this behavior at all, but included anyway. Maybe it helps for
finding out more about the problem.

> FWIW, binary images are stored as ubyte, so 1 *byte* per pixel.

Sounds strange. Have to google about this. Thanks for the Info, anyway! :)


Cheers,
Martin



On 07/13/2017 08:26 PM, Stefan van der Walt wrote:
> On Thu, Jul 13, 2017, at 04:21, Martin Fleck wrote:
>>
>> Indeed, this could be the complete problem already! For the analysis
>> I use a binary image - so only one bit per pixel.
>>
>
> FWIW, binary images are stored as ubyte, so 1 *byte* per pixel.
>
>> Greg: Regarding your PR and my analysis: My analysis using a 1.2GB
>> file stops due to memory problems already in
>> skimage.morphology.remove_small_objects() even if the major memory
>> blowup happens with skimage.morphology.label().
>> So there are problems at multiple steps that hopefully can be improved.
>
> Thanks for helping us uncover and trace these!
>
> St?fan
>
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/195f1367/attachment-0001.html>

From grlee77 at gmail.com  Thu Jul 13 22:23:31 2017
From: grlee77 at gmail.com (Gregory Lee)
Date: Thu, 13 Jul 2017 22:23:31 -0400
Subject: [scikit-image] Numba on pypi
In-Reply-To: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
Message-ID: <CAJR3sXfkL4G_1awg8hLaHhgg_0=F7-zyO3-UYzjFs_QqOgrjNw@mail.gmail.com>

I am also +1 on allowing numba code in scikit-image.  I have tended to
prefer Cython in the past, but it has been a while since I looked at numba
and it seems it has come a long way in recent years.  Juan's blog post and
the simple example you provided are great examples of relevant use cases.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/7297bd63/attachment.html>

From nadavh.horesh at gmail.com  Thu Jul 13 23:29:37 2017
From: nadavh.horesh at gmail.com (Nadav Horesh)
Date: Fri, 14 Jul 2017 06:29:37 +0300
Subject: [scikit-image] Numba on pypi
In-Reply-To: <CAJR3sXfkL4G_1awg8hLaHhgg_0=F7-zyO3-UYzjFs_QqOgrjNw@mail.gmail.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
 <CAJR3sXfkL4G_1awg8hLaHhgg_0=F7-zyO3-UYzjFs_QqOgrjNw@mail.gmail.com>
Message-ID: <CA+oBP_s8=USf3c9dtsgu1hrQF0oxV_c8O+ZpnE8OsZwFYVrHBw@mail.gmail.com>

I'd make numba dependency optional, since keeping llvm and llvmlite
versions in sync requires special attention (at least with the Linux
distros I used), what makes numba availability below 100%.

  Nadav.

On Jul 14, 2017 5:23 AM, "Gregory Lee" <grlee77 at gmail.com> wrote:

> I am also +1 on allowing numba code in scikit-image.  I have tended to
> prefer Cython in the past, but it has been a while since I looked at numba
> and it seems it has come a long way in recent years.  Juan's blog post and
> the simple example you provided are great examples of relevant use cases.
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170714/9cbfb05d/attachment.html>

From stefanv at berkeley.edu  Fri Jul 14 01:20:18 2017
From: stefanv at berkeley.edu (Stefan van der Walt)
Date: Fri, 14 Jul 2017 00:20:18 -0500
Subject: [scikit-image] Numba on pypi
In-Reply-To: <CA+oBP_s8=USf3c9dtsgu1hrQF0oxV_c8O+ZpnE8OsZwFYVrHBw@mail.gmail.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
 <CAJR3sXfkL4G_1awg8hLaHhgg_0=F7-zyO3-UYzjFs_QqOgrjNw@mail.gmail.com>
 <CA+oBP_s8=USf3c9dtsgu1hrQF0oxV_c8O+ZpnE8OsZwFYVrHBw@mail.gmail.com>
Message-ID: <15d3f8a5a50.2815.acf34a9c767d7bb498a799333be0433e@fastmail.com>

On 13 July 2017 22:30:03 Nadav Horesh <nadavh.horesh at gmail.com> wrote:

> I'd make numba dependency optional, since keeping llvm and llvmlite
> versions in sync requires special attention (at least with the Linux
> distros I used), what makes numba availability below 100%.

I don't think we need to keep these things in sync (if llvmlite is 
available at all, we're fine). And for pip installs we can consider 
precompiled binary modules, I guess?

Best regards
St?fan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170714/0693fb82/attachment.html>

From cedric.espenel at gmail.com  Fri Jul 14 01:23:01 2017
From: cedric.espenel at gmail.com (Cedric Espenel)
Date: Thu, 13 Jul 2017 22:23:01 -0700
Subject: [scikit-image] Image intensity measure.regionprops
Message-ID: <CAJE86+F0kux23KYjxLbsRgKd9pjVVL0txu5e8E8frvHHCud-Zw@mail.gmail.com>

Hi scikit-image users/devs,

I'm using skimage.measure.regionprops and I'm little bit confused about a
result I get with it.

I'm working with a zstack of a microscopy image that I have segmented/label
and I'm trying to get the mean intensity of the labeled region:

region_props = measure.regionprops(label_image, intensity_image)
>

If now I do:

for prop in region_prop:

print('x', prop.mean_intensity)
>>
> print('y', np.mean(prop.intensity_image))
>>
>
prop.mean_intensity and np.mean(prop.intensity_image) give me different
value, which confuse me. Can someone help me understand why I'm getting
something different?

Thank you in advance for your help.

Sincerely,

Cedric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170713/a71e1fe4/attachment.html>

From matthew.brett at gmail.com  Fri Jul 14 04:15:16 2017
From: matthew.brett at gmail.com (Matthew Brett)
Date: Fri, 14 Jul 2017 09:15:16 +0100
Subject: [scikit-image] Numba on pypi
In-Reply-To: <CAH6Pt5q0tGXmswEmv5XFL8DUXwMTR_bioab0PYud9eapTodgKw@mail.gmail.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
 <CABL7CQign+++EtqZqyJG88FEnG2w_dRps5WD3M_E5r=0YQMgGg@mail.gmail.com>
 <CAH6Pt5q0tGXmswEmv5XFL8DUXwMTR_bioab0PYud9eapTodgKw@mail.gmail.com>
Message-ID: <CAH6Pt5rGEv1qDYrOoaU-oOcQvCATLwJNqa-92KYG3nDR-mRHgA@mail.gmail.com>

Hi,

On Thu, Jul 13, 2017 at 9:10 AM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Thu, Jul 13, 2017 at 8:50 AM, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>>
>>
>> On Thu, Jul 13, 2017 at 6:16 PM, Stefan van der Walt <stefanv at berkeley.edu>
>> wrote:
>>>
>>> Hi everyone,
>>>
>>> As many of you know, speed has been a point of contention in
>>> scikit-image for a long time.  We've made a very deliberate decision to
>>> focus on writing high-level, understandable code (via Python and
>>> Cython): both to lower the barrier to entry for newcomers, and to lessen
>>> the burden on maintainers.  But execution time comparisons, vs OpenCV
>>> e.g., left much to be desired.
>>>
>>> I think we have hit a turning point in the road.  Binary wheels for
>>> Numba (actually, llvmlite) were recently uploaded to PyPi, making this
>>> technology available to users on both pip and conda installations.  The
>>> importance of this release on pypi should not be dismissed, and I am
>>> grateful to the numba team and Continuum for making that decision.
>>
>>
>> Agreed. Note that there are no Windows wheels up on PyPI (yet, or not
>> coming?). Given that there are no SciPy wheels for Windows either I don't
>> think that that changes your argument much - people should just use a binary
>> distribution on Windows - but I thought I'd point it out anway.
>
> We might be close to a working scipy wheel - discussion evolving over
> at https://github.com/scipy/scipy/issues/7551#issuecomment-314922271

Following up on my own post - updates on progress for a scipy wheel here:

https://github.com/scipy/scipy/issues/759

> If we do succeed, that would make the lack of a numba wheel for
> Windows much more significant.
>
> Does anyone know Continuum's plans in this matter?  Is the numba
> wheel recipe open-source?

Can anyone comment here?

The basic question is - what would happen if Continuum stopped
supplying a pypi wheel?  If the answer is the standard open source
answer - someone else would take over pretty quickly - that's fine.
Otherwise, it's a problem.

Cheers,

Matthew

From ralf.gommers at gmail.com  Fri Jul 14 04:29:12 2017
From: ralf.gommers at gmail.com (Ralf Gommers)
Date: Fri, 14 Jul 2017 20:29:12 +1200
Subject: [scikit-image] Numba on pypi
In-Reply-To: <CAH6Pt5rGEv1qDYrOoaU-oOcQvCATLwJNqa-92KYG3nDR-mRHgA@mail.gmail.com>
References: <1499926597.787926.1039365608.5A29B309@webmail.messagingengine.com>
 <CABL7CQign+++EtqZqyJG88FEnG2w_dRps5WD3M_E5r=0YQMgGg@mail.gmail.com>
 <CAH6Pt5q0tGXmswEmv5XFL8DUXwMTR_bioab0PYud9eapTodgKw@mail.gmail.com>
 <CAH6Pt5rGEv1qDYrOoaU-oOcQvCATLwJNqa-92KYG3nDR-mRHgA@mail.gmail.com>
Message-ID: <CABL7CQiXJcQQ1kioHHXPOqAkdb8iKrGkTaz-97FdPDUfo_QayQ@mail.gmail.com>

On Fri, Jul 14, 2017 at 8:15 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Thu, Jul 13, 2017 at 9:10 AM, Matthew Brett <matthew.brett at gmail.com>
> wrote:
> > Hi,
> >
> > On Thu, Jul 13, 2017 at 8:50 AM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
> >>
> >>
> >> On Thu, Jul 13, 2017 at 6:16 PM, Stefan van der Walt <
> stefanv at berkeley.edu>
> >> wrote:
> >>>
> >>> Hi everyone,
> >>>
> >>> As many of you know, speed has been a point of contention in
> >>> scikit-image for a long time.  We've made a very deliberate decision to
> >>> focus on writing high-level, understandable code (via Python and
> >>> Cython): both to lower the barrier to entry for newcomers, and to
> lessen
> >>> the burden on maintainers.  But execution time comparisons, vs OpenCV
> >>> e.g., left much to be desired.
> >>>
> >>> I think we have hit a turning point in the road.  Binary wheels for
> >>> Numba (actually, llvmlite) were recently uploaded to PyPi, making this
> >>> technology available to users on both pip and conda installations.  The
> >>> importance of this release on pypi should not be dismissed, and I am
> >>> grateful to the numba team and Continuum for making that decision.
> >>
> >>
> >> Agreed. Note that there are no Windows wheels up on PyPI (yet, or not
> >> coming?). Given that there are no SciPy wheels for Windows either I
> don't
> >> think that that changes your argument much - people should just use a
> binary
> >> distribution on Windows - but I thought I'd point it out anway.
> >
> > We might be close to a working scipy wheel - discussion evolving over
> > at https://github.com/scipy/scipy/issues/7551#issuecomment-314922271
>
> Following up on my own post - updates on progress for a scipy wheel here:
>
> https://github.com/scipy/scipy/issues/759
>
> > If we do succeed, that would make the lack of a numba wheel for
> > Windows much more significant.
> >
> > Does anyone know Continuum's plans in this matter?  Is the numba
> > wheel recipe open-source?
>
> Can anyone comment here?
>
> The basic question is - what would happen if Continuum stopped
> supplying a pypi wheel?  If the answer is the standard open source
> answer - someone else would take over pretty quickly - that's fine.
> Otherwise, it's a problem.
>

Can't read their mind, but did look at the build instructions. Doesn't look
that hard to build and package, if the need arises (which is unlikely). And
the current wheels will not disappear. So I don't really see an issue.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170714/d4b70f12/attachment.html>

From chaco001 at umn.edu  Fri Jul 14 09:49:41 2017
From: chaco001 at umn.edu (Jeremy Chacon)
Date: Fri, 14 Jul 2017 08:49:41 -0500
Subject: [scikit-image] Image intensity measure.regionprops
In-Reply-To: <CAJE86+F0kux23KYjxLbsRgKd9pjVVL0txu5e8E8frvHHCud-Zw@mail.gmail.com>
References: <CAJE86+F0kux23KYjxLbsRgKd9pjVVL0txu5e8E8frvHHCud-Zw@mail.gmail.com>
Message-ID: <CAH65v73m5PBLrrXSMpb2M__bABX5RjJK79z_aTcHUYb1nKQBtw@mail.gmail.com>

Hi Cedric,

I'm just a user but my interpretation is that mean_intensity is the mean
intensity of the pixels in that labeled region, whereas intensity_image is
the region within its bounding box.  So intensity_image can contain zeros
where the label isn't present.

See this:

from skimage import measure
import numpy as np
a = np.array([[2,3,4,5],[6,5,4,3]])
b = np.array([[0,1,1,0],[0,1,0,0]])

rp = measure.regionprops(b, a)

rp[0].mean_intensity  # 4
rp[0].intensity_image  # contains a zero, making the mean of this < 4



On Fri, Jul 14, 2017 at 12:23 AM, Cedric Espenel <cedric.espenel at gmail.com>
wrote:

> Hi scikit-image users/devs,
>
> I'm using skimage.measure.regionprops and I'm little bit confused about a
> result I get with it.
>
> I'm working with a zstack of a microscopy image that I have
> segmented/label and I'm trying to get the mean intensity of the labeled
> region:
>
> region_props = measure.regionprops(label_image, intensity_image)
>>
>
> If now I do:
>
> for prop in region_prop:
>
> print('x', prop.mean_intensity)
>>>
>> print('y', np.mean(prop.intensity_image))
>>>
>>
> prop.mean_intensity and np.mean(prop.intensity_image) give me different
> value, which confuse me. Can someone help me understand why I'm getting
> something different?
>
> Thank you in advance for your help.
>
> Sincerely,
>
> Cedric
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>


-- 

*___________________________________________________________________________Jeremy
M. Chacon, Ph.D.*

*Post-Doctoral Associate, Harcombe Lab*
*University of Minnesota*
*Ecology, Evolution and Behavior*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170714/55be9705/attachment.html>

From cedric.espenel at gmail.com  Fri Jul 14 12:34:00 2017
From: cedric.espenel at gmail.com (Cedric Espenel)
Date: Fri, 14 Jul 2017 09:34:00 -0700
Subject: [scikit-image] Image intensity measure.regionprops
In-Reply-To: <CAH65v73m5PBLrrXSMpb2M__bABX5RjJK79z_aTcHUYb1nKQBtw@mail.gmail.com>
References: <CAJE86+F0kux23KYjxLbsRgKd9pjVVL0txu5e8E8frvHHCud-Zw@mail.gmail.com>
 <CAH65v73m5PBLrrXSMpb2M__bABX5RjJK79z_aTcHUYb1nKQBtw@mail.gmail.com>
Message-ID: <CAJE86+F5W8j+hPU5QEKcGfBx_Lj5EM-Qz=4PMjR6qGzz4e64iw@mail.gmail.com>

Hi Jeremy,

Thank you for your answer, it makes a lot of sense to mask the intensity
image with the label region before measuring the mean intensity, which is
exactly what is in the source code:

> np.mean(self.intensity_image[self.image])


I guess I should have check before sending an email but the thank you for
your help!

Best,

Cedric

2017-07-14 6:49 GMT-07:00 Jeremy Chacon <chaco001 at umn.edu>:

> Hi Cedric,
>
> I'm just a user but my interpretation is that mean_intensity is the mean
> intensity of the pixels in that labeled region, whereas intensity_image is
> the region within its bounding box.  So intensity_image can contain zeros
> where the label isn't present.
>
> See this:
>
> from skimage import measure
> import numpy as np
> a = np.array([[2,3,4,5],[6,5,4,3]])
> b = np.array([[0,1,1,0],[0,1,0,0]])
>
> rp = measure.regionprops(b, a)
>
> rp[0].mean_intensity  # 4
> rp[0].intensity_image  # contains a zero, making the mean of this < 4
>
>
>
> On Fri, Jul 14, 2017 at 12:23 AM, Cedric Espenel <cedric.espenel at gmail.com
> > wrote:
>
>> Hi scikit-image users/devs,
>>
>> I'm using skimage.measure.regionprops and I'm little bit confused about a
>> result I get with it.
>>
>> I'm working with a zstack of a microscopy image that I have
>> segmented/label and I'm trying to get the mean intensity of the labeled
>> region:
>>
>> region_props = measure.regionprops(label_image, intensity_image)
>>>
>>
>> If now I do:
>>
>> for prop in region_prop:
>>
>> print('x', prop.mean_intensity)
>>>>
>>> print('y', np.mean(prop.intensity_image))
>>>>
>>>
>> prop.mean_intensity and np.mean(prop.intensity_image) give me different
>> value, which confuse me. Can someone help me understand why I'm getting
>> something different?
>>
>> Thank you in advance for your help.
>>
>> Sincerely,
>>
>> Cedric
>>
>>
>> _______________________________________________
>> scikit-image mailing list
>> scikit-image at python.org
>> https://mail.python.org/mailman/listinfo/scikit-image
>>
>>
>
>
> --
>
> *___________________________________________________________________________Jeremy
> M. Chacon, Ph.D.*
>
> *Post-Doctoral Associate, Harcombe Lab*
> *University of Minnesota*
> *Ecology, Evolution and Behavior*
>
>
> _______________________________________________
> scikit-image mailing list
> scikit-image at python.org
> https://mail.python.org/mailman/listinfo/scikit-image
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170714/c695e2ca/attachment.html>

From nelle.varoquaux at gmail.com  Sat Jul 15 10:15:24 2017
From: nelle.varoquaux at gmail.com (Nelle Varoquaux)
Date: Sat, 15 Jul 2017 07:15:24 -0700
Subject: [scikit-image] Sprint @ scipy: ticket triage and gathering ideas
In-Reply-To: <CAE-UAvTQ=0AyAi48ZPxuckXCPPcK34Kw9wf_cDNzir9sEikpwQ@mail.gmail.com>
References: <CAE-UAvTQ=0AyAi48ZPxuckXCPPcK34Kw9wf_cDNzir9sEikpwQ@mail.gmail.com>
Message-ID: <CAE-UAvSsoVWK6y+B07HSoBprFT49+KjdaViQO=-+BivAJd1hUg@mail.gmail.com>

Hi everyone,

Quick email to remind that we are about to start a sprint at Scipy!
Expect a lot of contributions today!
Help reviewing would be appreciated.

Cheers,
Nelle

On 11 July 2017 at 10:13, Nelle Varoquaux <nelle.varoquaux at gmail.com> wrote:
> Hi folks,
>
> Scikit-image will be hosting a sprint this week-end at scipy. In order
> to welcome new contributors, it'd be nice if we could gather ideas and
> start triaging tickets. Contact me or St?fan if you have any
> questions.
>
> Also, expect an increase in pull request activity! This might be a
> good opportunity to make progress and close some old pull requests.
>
> Cheers,
> N

From william.myers at student.nmt.edu  Wed Jul 26 14:35:32 2017
From: william.myers at student.nmt.edu (Riley Myers)
Date: Wed, 26 Jul 2017 12:35:32 -0600
Subject: [scikit-image] plot_matches() error?
Message-ID: <CAEXBJUgxjTK9Ei0BamcmDkq+Mn-jKCZ_PLRh=QD498q21ZND7Q@mail.gmail.com>

Hello,

I am currently having a bit of trouble with using scikit-image to align two
images using ORB. At the moment, if I try to plot the images, keypoints and
matches using plot_matches, I get a plot with only the images and the
keypoints and then a large stack trace, attached below. I thought that my
code might be the problem, but the same thing occurs if I run one of the
ipython notebooks from the skimage-tutorials, specifically
/lectures/adv3_panorama-stitching.ipynb, using all of their images, etc.
This seems to imply that something is off with my setup. According to
`conda list`, I'm using version 0.13.0 of scikit-image. Do any of you have
any ideas?

Thanks!
Riley


stack trace: https://pastebin.com/DtGTmmft
conda list output: https://pastebin.com/fEzikbi2
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-image/attachments/20170726/0f2bb02e/attachment.html>