From gael.varoquaux at normalesup.org Fri Dec 1 00:48:33 2017 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Fri, 1 Dec 2017 06:48:33 +0100 Subject: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17? In-Reply-To: References: Message-ID: <20171201054833.GE3504478@phare.normalesup.org> > - There are major projects like scikit-learn that simply have no > alternative to using np.matrix, because of scipy.sparse. Well, we have no love for np.matrix, we just use scipy.sparse and np.ndarray. > - After there's an alternative to scipy.sparse: +1 for alternative to scipy.sparse before transition. Ga?l From chris.barker at noaa.gov Fri Dec 1 13:03:18 2017 From: chris.barker at noaa.gov (Chris Barker) Date: Fri, 1 Dec 2017 10:03:18 -0800 Subject: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17? In-Reply-To: References: Message-ID: On Thu, Nov 30, 2017 at 11:58 AM, Marten van Kerkwijk < m.h.vankerkwijk at gmail.com> wrote: > Your point about not doing things in the python 2->3 move makes sense; > But this is NOT the 2->3 move -- numpy as been py3 compatible for years. At some point, it is a really good idea to deprecate some things. Personally, I think Matrix should have been deprecated a good while ago -- it never really worked well, and folks have been advised not to use it for years. But anyway, once we can count on having @ then there really is no reason to have Matrix, so it happens that dropping py2 support is the first time we can count on that. But this is really deprecating something when we stop support for py < 3.5, not the py2 to py3 transition. Remember that deprecating is different than dropping. If we want to keep Matrix around for one release after py2 is dropped, so that people can use it once they are "forced" to move to py3, OK, but let's get clear deprecation plan in place. Also -- we aren't requiring people to move to py3 -- we are only requiring people to move to py3 if they want the latest numpy features. One last note: Guido's suggestion that libraries not take py3 as an opportunity to change APIs was a good one, but it was also predicated on the fact that py2+p3 support was going to be needed for a good while. So this is really a different case. It's really a regular old deprecation -- you aren't going to have this feature in future numpy releases -- py2/3 has nothing to do with it. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.h.vankerkwijk at gmail.com Fri Dec 1 14:38:22 2017 From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk) Date: Fri, 1 Dec 2017 14:38:22 -0500 Subject: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17? In-Reply-To: References: Message-ID: Hi Chris, I'm easily convinced - yes, your argument makes sense too. Fortunately, at some level it doesn't affect what we do now. For 1.15 it should at least have a PendingDeprecationWarning. Since putting that in place means that all tests involving matrices now fail by default, it also becomes a good excuse to put all those tests together in `libmatrix`, so that we can set an exception to the warning=exception rule in one place. Hence, we automatically move closer to making it possible to take matrix out. In the meantime, for everybody, help with this initial step most welcome! PRs to master or to mine. (I'm at end of term and it's only procrastination that made me start this...) Thanks, all the best, Marten From ralf.gommers at gmail.com Sun Dec 3 02:36:42 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 3 Dec 2017 20:36:42 +1300 Subject: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17? In-Reply-To: References: Message-ID: On Fri, Dec 1, 2017 at 8:39 AM, Charles R Harris wrote: > > > On Thu, Nov 30, 2017 at 11:43 AM, Ralf Gommers > wrote: > >> >> >> On Fri, Dec 1, 2017 at 7:17 AM, Matthew Rocklin >> wrote: >> >>> How would the community handle the scipy.sparse matrix subclasses? >>> These are still in common use. >>> >> >> They're not going anywhere for quite a while (until the sparse ndarrays >> materialize at least). Hence np.matrix needs to be moved, not deleted. We >> discussed this earlier this year: https://mail.python.org/piperm >> ail/numpy-discussion/2017-January/076332.html >> >> >>> Somewhat related: https://github.com/scipy/scipy/issues/8162 >>> >>> On Thu, Nov 30, 2017 at 1:13 PM, wrote: >>> >>>> >>>> >>>> On Thu, Nov 30, 2017 at 9:23 AM, Marten van Kerkwijk < >>>> m.h.vankerkwijk at gmail.com> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I wondered if the move to python3-only starting with numpy 1.17 would >>>>> be a good reason to act on what we all seem to agree: that the matrix >>>>> class was a bad idea, with its overriding of multiplication and lack >>>>> of support for stacks of matrices. >>>> >>>> >> I'd suggest any release in the next couple of years is fine,but the one >> where we drop Python 2 support is probably the worst choice. That's one of >> the few things the core Python devs got 100% right with the Python 3 move: >> advocate that in the 2->3 transition packages would not make any API >> changes in order to make porting the least painful. >> >> Ralf >> > > Agree, we don't want to pile in too many changes at once. I think the big > sticking point is the sparse matrices in SciPy, even issuing a > DeprecationWarning could be problematic as long as there are sparse > matrices. May I suggest that we put together an NEP for the NumPy side of > things? Ralf, does SciPy have a mechanism for proposing such changes? > Not an official one, but any NEP-like proposal can be considered. I imagine the NEP itself will have to say something about it anyway, so could as well put it all in there. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Dec 3 02:47:09 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 3 Dec 2017 20:47:09 +1300 Subject: [Numpy-discussion] Deprecate matrices in 1.15 and remove in 1.17? In-Reply-To: References: Message-ID: On Sat, Dec 2, 2017 at 7:03 AM, Chris Barker wrote: > On Thu, Nov 30, 2017 at 11:58 AM, Marten van Kerkwijk < > m.h.vankerkwijk at gmail.com> wrote: > >> Your point about not doing things in the python 2->3 move makes sense; >> > > But this is NOT the 2->3 move -- numpy as been py3 compatible for years. > At some point, it is a really good idea to deprecate some things. > Deprecating can be done whenever (at least from this point of view, not from the scipy.sparse situation), the original proposal and my reply were about removal in 1.17. Effective what will happen is that users want to upgrade to the release that drops py2 (for whatever reason), and at that point are forced to upgrade to py3. So it would be good if that release did not have other API breaks. > Personally, I think Matrix should have been deprecated a good while ago -- > it never really worked well, and folks have been advised not to use it for > years. But anyway, once we can count on having @ then there really is no > reason to have Matrix, so it happens that dropping py2 support is the first > time we can count on that. But this is really deprecating something when we > stop support for py < 3.5, not the py2 to py3 transition. > > Remember that deprecating is different than dropping. If we want to keep > Matrix around for one release after py2 is dropped, so that people can use > it once they are "forced" to move to py3, OK, but let's get clear > deprecation plan in place. > > Also -- we aren't requiring people to move to py3 -- we are only requiring > people to move to py3 if they want the latest numpy features. > > One last note: Guido's suggestion that libraries not take py3 as an > opportunity to change APIs was a good one, but it was also predicated on > the fact that py2+p3 support was going to be needed for a good while. So > this is really a different case. It's really a regular old deprecation -- > you aren't going to have this feature in future numpy releases -- py2/3 has > nothing to do with it. > I think you're massively underestimating how many people are still on py2, are going to have to upgrade when we drop it, and how painful that is. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From pengyu.ut at gmail.com Sun Dec 3 16:38:02 2017 From: pengyu.ut at gmail.com (Peng Yu) Date: Sun, 3 Dec 2017 15:38:02 -0600 Subject: [Numpy-discussion] How to subset a matrix according to some criterion? Message-ID: Hi, I'd like to select the rows of 'data' for which the corresponding elements in 'target' is not 2. Could anybody let know what is the best way (with the least code) to do conditional subsetting like this with numpy objects? Thanks. from sklearn import datasets data, target = datasets.load_iris() -- Regards, Peng From jni.soma at gmail.com Sun Dec 3 17:06:30 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Mon, 4 Dec 2017 09:06:30 +1100 Subject: [Numpy-discussion] How to subset a matrix according to some criterion? In-Reply-To: References: Message-ID: <3863945b-85cd-4f42-b8e9-ffecd2dde17a@Spark> selector = target != 2 data, target = data[selector], target[selector] On 4 Dec 2017, 8:38 AM +1100, Peng Yu , wrote: > Hi, I'd like to select the rows of 'data' for which the corresponding > elements in 'target' is not 2. Could anybody let know what is the best > way (with the least code) to do conditional subsetting like this with > numpy objects? Thanks. > > from sklearn import datasets > data, target = datasets.load_iris() > > -- > Regards, > Peng > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Tue Dec 5 13:04:54 2017 From: shoyer at gmail.com (Stephan Hoyer) Date: Tue, 05 Dec 2017 18:04:54 +0000 Subject: [Numpy-discussion] Type annotations for NumPy In-Reply-To: <-4087092425119403979@unknownmsgid> References: <70c5a8e1-6b57-4019-be7a-65081fecf646@Spark> <-4087092425119403979@unknownmsgid> Message-ID: This discussion has died down, but I don't want to lose momentum . It sounds like there is at least strong interest from a subset of our community in type annotations. Are there any objections to the first part of my plan, to start developing type stubs for NumPy in separate repository? We'll come back to the mailing list when we have concrete proposals for typing dtypes and shapes. On Tue, Nov 28, 2017 at 4:05 PM Chris Barker - NOAA Federal < chris.barker at noaa.gov> wrote: > > > (a) it would be good if NumPy type annotations could include an > ?array_like? type that allows lists, tuples, etc. > > > I think that would be a sequence ? already supported by the Typing system. > > (b) I?ve always thought (since PEP561) that it would be cool for type > annotations to replace compiler type annotations for e.g. Cython and Numba. > Is this in the realm of possibility for the future? > > > Well, this was brought up early in the Typing discussion, and it was made > clear that these kinds of truly static types, as needed by Cython, was a > non-goal of the project. > > That being said, perhaps it could be made to work with a bunch of > additional type objects. > > And we should lol lol to Cython for ideas about how to type numpy arrays. > > One note: in addition to shape (rank) and types, there is contiguous and C > or F order. That may want to be considered. > > -CHB > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Tue Dec 5 17:19:46 2017 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 5 Dec 2017 14:19:46 -0800 Subject: [Numpy-discussion] Type annotations for NumPy In-Reply-To: References: <70c5a8e1-6b57-4019-be7a-65081fecf646@Spark> <-4087092425119403979@unknownmsgid> Message-ID: On Tue, Dec 5, 2017 at 10:04 AM, Stephan Hoyer wrote: > This discussion has died down, but I don't want to lose momentum . > > It sounds like there is at least strong interest from a subset of our > community in type annotations. Are there any objections to the first part of > my plan, to start developing type stubs for NumPy in separate repository? I think there's been plenty of time for folks to object to this if they wanted, so we can assume consensus until we hear otherwise. -n -- Nathaniel J. Smith -- https://vorpus.org From fperez.net at gmail.com Tue Dec 5 17:42:20 2017 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 5 Dec 2017 14:42:20 -0800 Subject: [Numpy-discussion] Type annotations for NumPy In-Reply-To: References: <70c5a8e1-6b57-4019-be7a-65081fecf646@Spark> <-4087092425119403979@unknownmsgid> Message-ID: On Tue, Dec 5, 2017 at 2:19 PM, Nathaniel Smith wrote: > On Tue, Dec 5, 2017 at 10:04 AM, Stephan Hoyer wrote: > > This discussion has died down, but I don't want to lose momentum . > > > > It sounds like there is at least strong interest from a subset of our > > community in type annotations. Are there any objections to the first > part of > > my plan, to start developing type stubs for NumPy in separate repository? > > I think there's been plenty of time for folks to object to this if > they wanted, so we can assume consensus until we hear otherwise. > peanut_gallery.throw('+1', thanks=True) # very happy to see this moving forward -------------- next part -------------- An HTML attachment was scrubbed... URL: From millman at berkeley.edu Tue Dec 5 18:31:42 2017 From: millman at berkeley.edu (Jarrod Millman) Date: Tue, 5 Dec 2017 15:31:42 -0800 Subject: [Numpy-discussion] NEP process update Message-ID: Hi all, Since we expect to be writing some NEPs in the near future, Nathaniel and I were looking at how they're organized, and realized that the process is a bit underspecified and it's a hard to tell the status of things. So I'm thinking of putting together some better tools and documentation, and wanted to get some quick feedback before I runoff in the wrong direction. The goal is not to add a ton of new process, but to better document the current process. I would also like to add a little additional structure to make it easier to understand the process for new contributors and to make the status of NEPS easier to understand. After reviewing the existing system, looking at how other projects do this, and discussing this with Nathaniel, I tentatively plan to work on the following: 1. Standardize the NEP metadata (e.g., whether they're draft/accepted/implemented) 2. Write a NEP to explain the purpose and process (think PEP 1) 3. Create a NEP template (think PEP 12) 4. Add metadata to old NEPs 5. Automate updating the NEP website and autogenerating the index based on the NEP metadata. Nathaniel and I have already started discussing some of these items and would love to get some feedback soon. Assuming that sounds good, my tentative next steps are: - I'll draft a purpose and process NEP based on PEP 1 and a few other projects. - I'll also create a draft NEP template. - I'll move the NEPs into their own repo (something like numpy/neps), and set up an automated system (RTD or Github pages) to render and publish them with some useful index. Does this seem like a reasonable way to start? Best regards, Jarrod From shoyer at gmail.com Tue Dec 5 18:58:58 2017 From: shoyer at gmail.com (Stephan Hoyer) Date: Tue, 05 Dec 2017 23:58:58 +0000 Subject: [Numpy-discussion] Type annotations for NumPy In-Reply-To: References: <70c5a8e1-6b57-4019-be7a-65081fecf646@Spark> <-4087092425119403979@unknownmsgid> Message-ID: OK, in that case let's get to work over in https://github.com/numpy/numpy_stubs! On Tue, Dec 5, 2017 at 2:43 PM Fernando Perez wrote: > On Tue, Dec 5, 2017 at 2:19 PM, Nathaniel Smith wrote: > >> On Tue, Dec 5, 2017 at 10:04 AM, Stephan Hoyer wrote: >> > This discussion has died down, but I don't want to lose momentum . >> > >> > It sounds like there is at least strong interest from a subset of our >> > community in type annotations. Are there any objections to the first >> part of >> > my plan, to start developing type stubs for NumPy in separate >> repository? >> >> I think there's been plenty of time for folks to object to this if >> they wanted, so we can assume consensus until we hear otherwise. >> > > peanut_gallery.throw('+1', thanks=True) # very happy to see this moving > forward > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Tue Dec 5 19:12:27 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 6 Dec 2017 13:12:27 +1300 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman wrote: > Hi all, > > Since we expect to be writing some NEPs in the near future, Nathaniel > and I were looking at how they're organized, and realized that the > process is a bit underspecified and it's a hard to tell the status of > things. > > So I'm thinking of putting together some better tools and > documentation, and wanted to get some quick feedback before I > runoff in the wrong direction. > > The goal is not to add a ton of new process, but to better document the > current > process. I would also like to add a little additional structure to make it > easier to understand the process for new contributors and to make the > status > of NEPS easier to understand. > > After reviewing the existing system, looking at how other projects do > this, and discussing this with Nathaniel, I tentatively plan to work > on the following: > > 1. Standardize the NEP metadata (e.g., whether they're > draft/accepted/implemented) > 2. Write a NEP to explain the purpose and process (think PEP 1) > 3. Create a NEP template (think PEP 12) > 4. Add metadata to old NEPs > 5. Automate updating the NEP website and autogenerating the index > based on the NEP metadata. > That all sounds good to me. > Nathaniel and I have already started discussing some of these items > and would love to get some feedback soon. > > Assuming that sounds good, my tentative next steps are: > > - I'll draft a purpose and process NEP based on PEP 1 and a few other > projects. > - I'll also create a draft NEP template. > sounds good - I'll move the NEPs into their own repo (something like numpy/neps), > This doesn't sound ideal to me - NEPs are important pieces of documentation, so I'd rather keep them included in the main docs. and set up an automated system (RTD or Github pages) to > render and publish them with some useful index. > If you could copy over the scipy method to rebuild the docs on each merge into master, that would achieve the same purpose. Compare https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to http://scipy.github.io/devdocs/, always up-to-date). Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Tue Dec 5 19:49:48 2017 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 5 Dec 2017 16:49:48 -0800 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers wrote: > On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman > wrote: >> Assuming that sounds good, my tentative next steps are: >> >> - I'll draft a purpose and process NEP based on PEP 1 and a few other >> projects. >> - I'll also create a draft NEP template. > > > sounds good > >> - I'll move the NEPs into their own repo (something like numpy/neps), > > This doesn't sound ideal to me - NEPs are important pieces of documentation, > so I'd rather keep them included in the main docs. > >> and set up an automated system (RTD or Github pages) to >> render and publish them with some useful index. > > > If you could copy over the scipy method to rebuild the docs on each merge > into master, that would achieve the same purpose. Compare > https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs > https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to > http://scipy.github.io/devdocs/, always up-to-date). Yeah, we were debating back and forth on this -- I can see arguments either way. The reasons we were leaning towards splitting them out are: - it would be great to make our regular docs auto-generated, but we didn't necessarily want to block this on that - part of the idea is to auto-generate the NEP index out of the metadata inside each NEP file, which is going to involve writing some code and integrating it into the NEP build. This seems easier if we don't have to integrate it into the overall doc build process too, which already has a lot of custom code. - NEPs are really part of the development process, not an output for end-users -- they're certainly useful to have available as a reference, but if we're asking end-users to look at them on a regular basis then I think we've messed up and should improve our actual documentation :-) - NEPs have a different natural life-cycle than numpy itself. Right now, if I google "numpy neps", the first hit is the 1.13 version of the NEPs, and the third hit is someone else's copy of the 1.9 version of the NEPs. What you actually want in every case is the latest development version of the NEPs, and the idea of "numpy 1.13 NEPs" doesn't even make sense, because NEPs are not describing a specific numpy release. -n -- Nathaniel J. Smith -- https://vorpus.org From robert.kern at gmail.com Tue Dec 5 20:05:05 2017 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 6 Dec 2017 10:05:05 +0900 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 9:49 AM, Nathaniel Smith wrote: > - NEPs are really part of the development process, not an output for > end-users -- they're certainly useful to have available as a > reference, but if we're asking end-users to look at them on a regular > basis then I think we've messed up and should improve our actual > documentation :-) I agree. The usefulness of NEPs (and PEPs) as documentation degrades over time. The functionality matches the NEP at the time that it is accepted, but features evolve over time through the normal, non-NEP development process. Personally, I've frequently drove myself into corners reading a PEP to learn about a new feature only to "learn" things that did not make it into the final implementation or were quickly overtaken by later development. We should not rely on NEPs to provide documentation for users. The regular docs should be the authority. To the extent that the NEPs happen to provide useful documentation for the new feature (and represent a significant amount of sunk effort to draft that documentation), we should feel free to copy-paste that into the regular docs and evolve it from there. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Tue Dec 5 20:32:01 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 6 Dec 2017 14:32:01 +1300 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: > On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers > wrote: > > On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman > > wrote: > >> Assuming that sounds good, my tentative next steps are: > >> > >> - I'll draft a purpose and process NEP based on PEP 1 and a few other > >> projects. > >> - I'll also create a draft NEP template. > > > > > > sounds good > > > >> - I'll move the NEPs into their own repo (something like numpy/neps), > > > > This doesn't sound ideal to me - NEPs are important pieces of > documentation, > > so I'd rather keep them included in the main docs. > > > >> and set up an automated system (RTD or Github pages) to > >> render and publish them with some useful index. > > > > > > If you could copy over the scipy method to rebuild the docs on each merge > > into master, that would achieve the same purpose. Compare > > https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs > > https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to > > http://scipy.github.io/devdocs/, always up-to-date). > > Yeah, we were debating back and forth on this -- I can see arguments > either way. The reasons we were leaning towards splitting them out > are: > > - it would be great to make our regular docs auto-generated, but we > didn't necessarily want to block this on that > This is easy, it's just copying a trick from SciPy. (and that's work that anyway needs doing at some point, sooner rather than later) - part of the idea is to auto-generate the NEP index out of the > metadata inside each NEP file, which is going to involve writing some > code and integrating it into the NEP build. This seems easier if we > don't have to integrate it into the overall doc build process too, > which already has a lot of custom code. > This is easy too, if you have your script to auto-generate an index file, it's just calling that file from within `doc/Makefile`. - NEPs are really part of the development process, not an output for > end-users -- they're certainly useful to have available as a > reference, but if we're asking end-users to look at them on a regular > basis then I think we've messed up and should improve our actual > documentation :-) > - NEPs have a different natural life-cycle than numpy itself. Right > now, if I google "numpy neps", the first hit is the 1.13 version of > the NEPs, and the third hit is someone else's copy of the 1.9 version > of the NEPs. What you actually want in every case is the latest > development version of the NEPs, and the idea of "numpy 1.13 NEPs" > doesn't even make sense, because NEPs are not describing a specific > numpy release. > The last two points are good arguments, I agree that they shouldn't serve as documentation. A separate repo has downsides though (discoverability etc.), we also keep our dev docs within the numpy repo and you can make exactly the same argument about those as about NEPs. So I'd still suggest keeping them where they are. Or otherwise move all development related docs. It's probably less effort even to keep them where they are rather than creating a separate repo, setting up RTD for it, and linking to that from our main docs. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From nelle.varoquaux at gmail.com Tue Dec 5 20:36:41 2017 From: nelle.varoquaux at gmail.com (Nelle Varoquaux) Date: Tue, 5 Dec 2017 17:36:41 -0800 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On 5 December 2017 at 17:32, Ralf Gommers wrote: > > > On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: >> >> On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers >> wrote: >> > On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman >> > wrote: >> >> Assuming that sounds good, my tentative next steps are: >> >> >> >> - I'll draft a purpose and process NEP based on PEP 1 and a few other >> >> projects. >> >> - I'll also create a draft NEP template. >> > >> > >> > sounds good >> > >> >> - I'll move the NEPs into their own repo (something like numpy/neps), >> > >> > This doesn't sound ideal to me - NEPs are important pieces of >> > documentation, >> > so I'd rather keep them included in the main docs. >> > >> >> and set up an automated system (RTD or Github pages) to >> >> render and publish them with some useful index. >> > >> > >> > If you could copy over the scipy method to rebuild the docs on each >> > merge >> > into master, that would achieve the same purpose. Compare >> > https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs >> > https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to >> > http://scipy.github.io/devdocs/, always up-to-date). >> >> Yeah, we were debating back and forth on this -- I can see arguments >> either way. The reasons we were leaning towards splitting them out >> are: >> >> - it would be great to make our regular docs auto-generated, but we >> didn't necessarily want to block this on that > > > This is easy, it's just copying a trick from SciPy. (and that's work that > anyway needs doing at some point, sooner rather than later) > I can probably help out with that. It's very straightforward once you've done it a couple of time. I'm also a strong believer in automating as much as possible. >> - part of the idea is to auto-generate the NEP index out of the >> metadata inside each NEP file, which is going to involve writing some >> code and integrating it into the NEP build. This seems easier if we >> don't have to integrate it into the overall doc build process too, >> which already has a lot of custom code. > > > This is easy too, if you have your script to auto-generate an index file, > it's just calling that file from within `doc/Makefile`. > >> - NEPs are really part of the development process, not an output for >> end-users -- they're certainly useful to have available as a >> reference, but if we're asking end-users to look at them on a regular >> basis then I think we've messed up and should improve our actual >> documentation :-) >> - NEPs have a different natural life-cycle than numpy itself. Right >> now, if I google "numpy neps", the first hit is the 1.13 version of >> the NEPs, and the third hit is someone else's copy of the 1.9 version >> of the NEPs. What you actually want in every case is the latest >> development version of the NEPs, and the idea of "numpy 1.13 NEPs" >> doesn't even make sense, because NEPs are not describing a specific >> numpy release. > > > The last two points are good arguments, I agree that they shouldn't serve as > documentation. A separate repo has downsides though (discoverability etc.), > we also keep our dev docs within the numpy repo and you can make exactly the > same argument about those as about NEPs. So I'd still suggest keeping them > where they are. Or otherwise move all development related docs. > > It's probably less effort even to keep them where they are rather than > creating a separate repo, setting up RTD for it, and linking to that from > our main docs. > > Ralf > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > From millman at berkeley.edu Tue Dec 5 20:58:41 2017 From: millman at berkeley.edu (Jarrod Millman) Date: Tue, 5 Dec 2017 17:58:41 -0800 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: I was planning on looking at/working on the main doc generating system and the main webpage (for numpy and scipy) soon (over the winter break), but I didn't want to get too many things in the discussion right now. My immediate interest is getting agreement on the first two items: - A purpose and process NEP based on PEP 1 and a few other projects. - A draft NEP template. I am ambivalent about whether we need a separate repo. My immediate plan is start drafting text as a PR to numpy/doc/neps on Thursday (assuming no major objections arise before then). That way we can discuss the NEP purpose/process and template docs, while I look into the build system. I was imaging doing something similar to what I did for for networkx this summer: https://networkx.github.io/ The latest docs are autogenerated whenever a PR is merged into master: https://networkx.github.io/documentation/latest/ I will look into the scipy system as well and make sure we don't have an explosion of systems. Jarrod On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers wrote: > > > On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: >> >> On Tue, Dec 5, 2017 at 4:12 PM, Ralf Gommers >> wrote: >> > On Wed, Dec 6, 2017 at 12:31 PM, Jarrod Millman >> > wrote: >> >> Assuming that sounds good, my tentative next steps are: >> >> >> >> - I'll draft a purpose and process NEP based on PEP 1 and a few other >> >> projects. >> >> - I'll also create a draft NEP template. >> > >> > >> > sounds good >> > >> >> - I'll move the NEPs into their own repo (something like numpy/neps), >> > >> > This doesn't sound ideal to me - NEPs are important pieces of >> > documentation, >> > so I'd rather keep them included in the main docs. >> > >> >> and set up an automated system (RTD or Github pages) to >> >> render and publish them with some useful index. >> > >> > >> > If you could copy over the scipy method to rebuild the docs on each >> > merge >> > into master, that would achieve the same purpose. Compare >> > https://docs.scipy.org/doc/numpy-dev/reference/ (outdated) vs >> > https://docs.scipy.org/doc/scipy-dev/reference/ (redirects to >> > http://scipy.github.io/devdocs/, always up-to-date). >> >> Yeah, we were debating back and forth on this -- I can see arguments >> either way. The reasons we were leaning towards splitting them out >> are: >> >> - it would be great to make our regular docs auto-generated, but we >> didn't necessarily want to block this on that > > > This is easy, it's just copying a trick from SciPy. (and that's work that > anyway needs doing at some point, sooner rather than later) > >> - part of the idea is to auto-generate the NEP index out of the >> metadata inside each NEP file, which is going to involve writing some >> code and integrating it into the NEP build. This seems easier if we >> don't have to integrate it into the overall doc build process too, >> which already has a lot of custom code. > > > This is easy too, if you have your script to auto-generate an index file, > it's just calling that file from within `doc/Makefile`. > >> - NEPs are really part of the development process, not an output for >> end-users -- they're certainly useful to have available as a >> reference, but if we're asking end-users to look at them on a regular >> basis then I think we've messed up and should improve our actual >> documentation :-) >> - NEPs have a different natural life-cycle than numpy itself. Right >> now, if I google "numpy neps", the first hit is the 1.13 version of >> the NEPs, and the third hit is someone else's copy of the 1.9 version >> of the NEPs. What you actually want in every case is the latest >> development version of the NEPs, and the idea of "numpy 1.13 NEPs" >> doesn't even make sense, because NEPs are not describing a specific >> numpy release. > > > The last two points are good arguments, I agree that they shouldn't serve as > documentation. A separate repo has downsides though (discoverability etc.), > we also keep our dev docs within the numpy repo and you can make exactly the > same argument about those as about NEPs. So I'd still suggest keeping them > where they are. Or otherwise move all development related docs. > > It's probably less effort even to keep them where they are rather than > creating a separate repo, setting up RTD for it, and linking to that from > our main docs. > > Ralf > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > From njs at pobox.com Tue Dec 5 22:44:23 2017 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 5 Dec 2017 19:44:23 -0800 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers wrote: > > > On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: >> - NEPs are really part of the development process, not an output for >> end-users -- they're certainly useful to have available as a >> reference, but if we're asking end-users to look at them on a regular >> basis then I think we've messed up and should improve our actual >> documentation :-) >> - NEPs have a different natural life-cycle than numpy itself. Right >> now, if I google "numpy neps", the first hit is the 1.13 version of >> the NEPs, and the third hit is someone else's copy of the 1.9 version >> of the NEPs. What you actually want in every case is the latest >> development version of the NEPs, and the idea of "numpy 1.13 NEPs" >> doesn't even make sense, because NEPs are not describing a specific >> numpy release. > > > The last two points are good arguments, I agree that they shouldn't serve as > documentation. A separate repo has downsides though (discoverability etc.), > we also keep our dev docs within the numpy repo and you can make exactly the > same argument about those as about NEPs. So I'd still suggest keeping them > where they are. Or otherwise move all development related docs. Are these the dev docs you're thinking of? https://docs.scipy.org/doc/numpy-dev/dev/index.html Regarding discoverability, right now it looks like the only way to find the latest NEPs on google is by searching for something like "numpy-dev neps", which is pretty obscure. (It took me 4 tries to find something that worked. "numpy neps" seemed to work, but actually sent me to an out-of-date snapshot.) In Python, the PEP web pages are rebuilt on something like a 6 hour timer, and it's actually super annoying, because it means that when someone posts to the list like "hey, I just pushed a new version, tell me what you think", everyone goes and finds the old stale version, sometimes people start critiquing it, ... it's just confusing all around. So I do think we want to make sure there's some simple way to find them, and that it leads to the latest version, not a stale build or an old snapshot. Moving NEPs + development docs to their own dedicated repo would resolve this and seems like a plausible option to me. We could probably do better than we are now with the regular docs too. Though the experience with PEPs does make me a bit nervous about having versioned snapshots of the NEPs in all our old versioned manuals (which have tons of google-juice). -n -- Nathaniel J. Smith -- https://vorpus.org From josef.pktd at gmail.com Tue Dec 5 23:39:04 2017 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 5 Dec 2017 23:39:04 -0500 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Tue, Dec 5, 2017 at 10:44 PM, Nathaniel Smith wrote: > On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers > wrote: > > > > > > On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: > >> - NEPs are really part of the development process, not an output for > >> end-users -- they're certainly useful to have available as a > >> reference, but if we're asking end-users to look at them on a regular > >> basis then I think we've messed up and should improve our actual > >> documentation :-) > >> - NEPs have a different natural life-cycle than numpy itself. Right > >> now, if I google "numpy neps", the first hit is the 1.13 version of > >> the NEPs, and the third hit is someone else's copy of the 1.9 version > >> of the NEPs. What you actually want in every case is the latest > >> development version of the NEPs, and the idea of "numpy 1.13 NEPs" > >> doesn't even make sense, because NEPs are not describing a specific > >> numpy release. > > > > > > The last two points are good arguments, I agree that they shouldn't > serve as > > documentation. A separate repo has downsides though (discoverability > etc.), > > we also keep our dev docs within the numpy repo and you can make exactly > the > > same argument about those as about NEPs. So I'd still suggest keeping > them > > where they are. Or otherwise move all development related docs. > > Are these the dev docs you're thinking of? > https://docs.scipy.org/doc/numpy-dev/dev/index.html > > Regarding discoverability, right now it looks like the only way to > find the latest NEPs on google is by searching for something like > "numpy-dev neps", which is pretty obscure. (It took me 4 tries to find > something that worked. "numpy neps" seemed to work, but actually sent > me to an out-of-date snapshot.) In Python, the PEP web pages are > rebuilt on something like a 6 hour timer, and it's actually super > annoying, because it means that when someone posts to the list like > "hey, I just pushed a new version, tell me what you think", everyone > goes and finds the old stale version, sometimes people start > critiquing it, ... it's just confusing all around. So I do think we > want to make sure there's some simple way to find them, and that it > leads to the latest version, not a stale build or an old snapshot. > > Moving NEPs + development docs to their own dedicated repo would > resolve this and seems like a plausible option to me. We could > probably do better than we are now with the regular docs too. Though > the experience with PEPs does make me a bit nervous about having > versioned snapshots of the NEPs in all our old versioned manuals > (which have tons of google-juice). > maybe I have a different google, but the first search result for me for "numpy nep" is https://docs.scipy.org/doc/numpy-dev/neps/index.html developer guides are on top here https://docs.scipy.org/doc/ using the link for "Complete Numpy Manual" on that page has dev docs and neps listed. But I don't see "numpy doc standard" which I usually need to google and the search leads to github page for it. Searching for specific items leads often to older versions in google and needs manual interventions, e.g. edit the version number in link Aside: My main occasion for reading Peps (besides quoting ZEN) is to follow the links in the python what's new pages, e.g. https://docs.python.org/3/whatsnew/3.6.html Josef > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Wed Dec 6 03:41:38 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 6 Dec 2017 21:41:38 +1300 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 2:58 PM, Jarrod Millman wrote: > I was planning on looking at/working on the main doc generating system > and the main webpage (for numpy and scipy) soon (over the winter > break), but I didn't want to get too many things in the discussion > right now. My immediate interest is getting agreement on the first > two items: > > - A purpose and process NEP based on PEP 1 and a few other projects. > - A draft NEP template. > +1 thanks Jarrod. And thanks Nelle for offering to work on the automation! Ralf > I am ambivalent about whether we need a separate repo. My immediate > plan is start drafting text as a PR to numpy/doc/neps on Thursday > (assuming no major objections arise before then). That way we can > discuss the NEP purpose/process and template docs, while I look into > the build system. I was imaging doing something similar to what I did > for for networkx this summer: > https://networkx.github.io/ > The latest docs are autogenerated whenever a PR is merged into master: > https://networkx.github.io/documentation/latest/ > > I will look into the scipy system as well and make sure we don't have > an explosion of systems. > Note that the build/doc/release systems of numpy and scipy are ~95% common (because they're mostly maintained by the same people), so syncing improvements between numpy and scipy will almost always require the least effort. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Wed Dec 6 03:43:37 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 6 Dec 2017 21:43:37 +1300 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 5:39 PM, wrote: > > > On Tue, Dec 5, 2017 at 10:44 PM, Nathaniel Smith wrote: > >> On Tue, Dec 5, 2017 at 5:32 PM, Ralf Gommers >> wrote: >> > >> > >> > On Wed, Dec 6, 2017 at 1:49 PM, Nathaniel Smith wrote: >> >> - NEPs are really part of the development process, not an output for >> >> end-users -- they're certainly useful to have available as a >> >> reference, but if we're asking end-users to look at them on a regular >> >> basis then I think we've messed up and should improve our actual >> >> documentation :-) >> >> - NEPs have a different natural life-cycle than numpy itself. Right >> >> now, if I google "numpy neps", the first hit is the 1.13 version of >> >> the NEPs, and the third hit is someone else's copy of the 1.9 version >> >> of the NEPs. What you actually want in every case is the latest >> >> development version of the NEPs, and the idea of "numpy 1.13 NEPs" >> >> doesn't even make sense, because NEPs are not describing a specific >> >> numpy release. >> > >> > >> > The last two points are good arguments, I agree that they shouldn't >> serve as >> > documentation. A separate repo has downsides though (discoverability >> etc.), >> > we also keep our dev docs within the numpy repo and you can make >> exactly the >> > same argument about those as about NEPs. So I'd still suggest keeping >> them >> > where they are. Or otherwise move all development related docs. >> >> Are these the dev docs you're thinking of? >> https://docs.scipy.org/doc/numpy-dev/dev/index.html > > indeed. possibly governance as well. >> >> Regarding discoverability, right now it looks like the only way to >> find the latest NEPs on google is by searching for something like >> "numpy-dev neps", which is pretty obscure. (It took me 4 tries to find >> something that worked. "numpy neps" seemed to work, but actually sent >> me to an out-of-date snapshot.) In Python, the PEP web pages are >> rebuilt on something like a 6 hour timer, and it's actually super >> annoying, because it means that when someone posts to the list like >> "hey, I just pushed a new version, tell me what you think", everyone >> goes and finds the old stale version, sometimes people start >> critiquing it, ... it's just confusing all around. So I do think we >> want to make sure there's some simple way to find them, and that it >> leads to the latest version, not a stale build or an old snapshot. >> >> Moving NEPs + development docs to their own dedicated repo would >> resolve this and seems like a plausible option to me. We could >> probably do better than we are now with the regular docs too. Though >> the experience with PEPs does make me a bit nervous about having >> versioned snapshots of the NEPs in all our old versioned manuals >> (which have tons of google-juice). >> > > > maybe I have a different google, but the first search result for me for > "numpy nep" > is https://docs.scipy.org/doc/numpy-dev/neps/index.html > > developer guides are on top here > https://docs.scipy.org/doc/ > > using the link for "Complete Numpy Manual" on that page has > dev docs and neps listed. > > But I don't see "numpy doc standard" which I usually need to google > and the search leads to github page for it. > These have recently been moved to https://numpydoc.readthedocs.io/en/latest/format.html Ralf > Searching for specific items leads often to older versions in google and > needs > manual interventions, e.g. edit the version number in link > > Aside: My main occasion for reading Peps (besides quoting ZEN) is to follow > the links in the python what's new pages, e.g. https://docs.python.org/ > 3/whatsnew/3.6.html > > > Josef > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.h.vankerkwijk at gmail.com Wed Dec 6 09:23:01 2017 From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk) Date: Wed, 6 Dec 2017 09:23:01 -0500 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: Would be great to have structure, and especially a template - ideally, the latter is enough for someone to create a NEP, i.e., has lots of in-template documentation. One thing I'd recommend thinking a little about is to what extend a NEP is "frozen" after acceptance. In astropy we've seen situations where it helps to clarify details later, and it may be good to think beforehand what one wants. In my opinion, one should allow clarifications of accepted NEPs, and major editing of ones still pending (as happened for __[numpy|array]_ufunc__). I think the location is secondary, but for what it is worth, I'm not fond of the astropy APEs being in a separate repository, mostly because I like detailed discussion of everything related in the project to happen in one place on github. Also, having to clone a repository is yet another hurdle for doing stuff. So, I'd suggest to keep the NEPs in the main repository. -- Marten From charlesr.harris at gmail.com Wed Dec 6 13:43:42 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 6 Dec 2017 11:43:42 -0700 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: On Wed, Dec 6, 2017 at 7:23 AM, Marten van Kerkwijk < m.h.vankerkwijk at gmail.com> wrote: > Would be great to have structure, and especially a template - ideally, > the latter is enough for someone to create a NEP, i.e., has lots of > in-template documentation. > > One thing I'd recommend thinking a little about is to what extend a > NEP is "frozen" after acceptance. In astropy we've seen situations > where it helps to clarify details later, and it may be good to think > beforehand what one wants. In my opinion, one should allow > clarifications of accepted NEPs, and major editing of ones still > pending (as happened for __[numpy|array]_ufunc__). > > I think the location is secondary, but for what it is worth, I'm not > fond of the astropy APEs being in a separate repository, mostly > because I like detailed discussion of everything related in the > project to happen in one place on github. Also, having to clone a > repository is yet another hurdle for doing stuff. So, I'd suggest to > keep the NEPs in the main repository. +1 Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Thu Dec 7 10:51:36 2017 From: chris.barker at noaa.gov (Chris Barker) Date: Thu, 7 Dec 2017 07:51:36 -0800 Subject: [Numpy-discussion] NEP process update In-Reply-To: References: Message-ID: Great idea -- thanks for pushing this forward all. In the end, you can have the NEPs in a separate repo, and still publish them closely with the main docs (intersphinx is pretty cool), or have them in the same repo and publish them separately. So I say let the folks doing the work decide what workflow works best for them. Comments on a couple other points: I find myself going back to PEPs quite a bit -- mostly to understand the hows an whys of a feature, rather than the how-to-use its. And yes -- we should keep NEPs updated -- they certainly should be edited for typos and minor clarifications, but It's particularly important if the implementation ends up differing a bit from what was expected when the NEP was written. I'm not sure what the PEP policy is about this, but they are certainly maintained with regard to typos and the like. -CHB On Wed, Dec 6, 2017 at 10:43 AM, Charles R Harris wrote: > > > On Wed, Dec 6, 2017 at 7:23 AM, Marten van Kerkwijk < > m.h.vankerkwijk at gmail.com> wrote: > >> Would be great to have structure, and especially a template - ideally, >> the latter is enough for someone to create a NEP, i.e., has lots of >> in-template documentation. >> >> One thing I'd recommend thinking a little about is to what extend a >> NEP is "frozen" after acceptance. In astropy we've seen situations >> where it helps to clarify details later, and it may be good to think >> beforehand what one wants. In my opinion, one should allow >> clarifications of accepted NEPs, and major editing of ones still >> pending (as happened for __[numpy|array]_ufunc__). >> >> I think the location is secondary, but for what it is worth, I'm not >> fond of the astropy APEs being in a separate repository, mostly >> because I like detailed discussion of everything related in the >> project to happen in one place on github. Also, having to clone a >> repository is yet another hurdle for doing stuff. So, I'd suggest to >> keep the NEPs in the main repository. > > > +1 > > Chuck > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Dec 7 15:28:19 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Thu, 7 Dec 2017 13:28:19 -0700 Subject: [Numpy-discussion] NumPy 1.14 branch Message-ID: Hi All, I plan to branch NumPy 1.14.x tomorrow, 08-12-1017. If anyone feels that there is a current PR that needs to be in, please comment. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Fri Dec 8 15:28:18 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 8 Dec 2017 13:28:18 -0700 Subject: [Numpy-discussion] Numpy 1.14.x has been branch, master is open for 1.15 development. Message-ID: Hi All, Just a heads up that NumPy 1.14.x has been branched. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Fri Dec 8 17:00:03 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 9 Dec 2017 11:00:03 +1300 Subject: [Numpy-discussion] Numpy 1.14.x has been branch, master is open for 1.15 development. In-Reply-To: References: Message-ID: On Sat, Dec 9, 2017 at 9:28 AM, Charles R Harris wrote: > Hi All, > > Just a heads up that NumPy 1.14.x has been branched. > Thanks Chuck! Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From pengyu.ut at gmail.com Mon Dec 11 22:39:24 2017 From: pengyu.ut at gmail.com (Peng Yu) Date: Mon, 11 Dec 2017 21:39:24 -0600 Subject: [Numpy-discussion] The meaning of threshold in numpy.set_printoptions()? Message-ID: Hi, The help says. threshold : int, optional Total number of array elements which trigger summarization rather than full repr (default 1000). But the following code shows that 7 triggers summarization instead of 5. Does anybody know how to interpret the meaning of `threshold`? Thanks. In [1]: import numpy as np ...: np.set_printoptions(threshold=5) ...: print(np.arange(6)) ...: print(np.arange(7)) ...: [0 1 2 3 4 5] [0 1 2 ..., 4 5 6] -- Regards, Peng From wieser.eric+numpy at gmail.com Mon Dec 11 22:44:06 2017 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Tue, 12 Dec 2017 03:44:06 +0000 Subject: [Numpy-discussion] The meaning of threshold in numpy.set_printoptions()? In-Reply-To: References: Message-ID: Whether summarization happens is also determined by edgeitems (defaults to 3), which is the number of items to print on each edge when summarizing. It wouldn?t make sense for summarization to cause the output to be longer than if no summarization occured! Feel free to submit an improvement to the documentation there on github. Eric ? On Mon, 11 Dec 2017 at 19:40 Peng Yu wrote: > Hi, > > The help says. > > threshold : int, optional > Total number of array elements which trigger summarization > rather than full repr (default 1000). > > But the following code shows that 7 triggers summarization instead of > 5. Does anybody know how to interpret the meaning of `threshold`? > Thanks. > > In [1]: import numpy as np > ...: np.set_printoptions(threshold=5) > ...: print(np.arange(6)) > ...: print(np.arange(7)) > ...: > [0 1 2 3 4 5] > [0 1 2 ..., 4 5 6] > > -- > Regards, > Peng > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcaswell at gmail.com Mon Dec 11 22:49:06 2017 From: tcaswell at gmail.com (Thomas Caswell) Date: Tue, 12 Dec 2017 03:49:06 +0000 Subject: [Numpy-discussion] [REL] Matplotlib 2.1.1 Message-ID: Folks, Happy to announce Matplotlib 2.1.1 This is primarily a bug fix release, see https://github.com/matplotlib/matplotlib/releases/tag/v2.1.1 for details. The next planned release is a 2.2 feature release in January/February 2018. Thank you to everyone who worked on this release! Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From solarjoe at posteo.org Tue Dec 12 03:02:16 2017 From: solarjoe at posteo.org (Joe) Date: Tue, 12 Dec 2017 09:02:16 +0100 Subject: [Numpy-discussion] =?utf-8?q?Which_rule_makes_x=5Bnp=2Enewaxis?= =?utf-8?b?LDpdIGFuZCB4W25wLm5ld2F4aXNdIGVxdWl2YWxlbnQ/?= In-Reply-To: References: Message-ID: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> Hi, question says it all. I looked through the basic and advanced indexing, but I could not find the rule that is applied to make x[np.newaxis,:] and x[np.newaxis] the same. Kind regards, Joe From njs at pobox.com Tue Dec 12 03:09:19 2017 From: njs at pobox.com (Nathaniel Smith) Date: Tue, 12 Dec 2017 00:09:19 -0800 Subject: [Numpy-discussion] Which rule makes x[np.newaxis, :] and x[np.newaxis] equivalent? In-Reply-To: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> Message-ID: On Tue, Dec 12, 2017 at 12:02 AM, Joe wrote: > Hi, > > question says it all. I looked through the basic and advanced indexing, > but I could not find the rule that is applied to make > x[np.newaxis,:] and x[np.newaxis] the same. I think it's the general rule that all indexing expressions have an invisible "..." on the right edge. For example, x[i][j][k] is an inefficient and IMO somewhat confusing way to write x[i, j, k], because x[i][j][k] is interpreted as: -> x[i, ...][j, ...][k, ...] -> x[i, :, :][j, :][k] That this also applies to newaxis is a little surprising, but I guess consistent. -n -- Nathaniel J. Smith -- https://vorpus.org From solarjoe at posteo.org Tue Dec 12 08:19:36 2017 From: solarjoe at posteo.org (Joe) Date: Tue, 12 Dec 2017 14:19:36 +0100 Subject: [Numpy-discussion] =?utf-8?q?Which_rule_makes_x=5Bnp=2Enewaxis?= =?utf-8?q?=2C_=3A=5D_and_x=5Bnp=2Enewaxis=5D_equivalent=3F?= In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> Message-ID: <3711a746661e352f5507e8fc2bc4e97b@posteo.de> Ah, ok, now that I knew what to look for I guess I found it: "If the number of objects in the selection tuple is less than N , then : is assumed for any subsequent dimensions." https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.indexing.html This is the one, right? Am 12.12.2017 09:09 schrieb Nathaniel Smith: > On Tue, Dec 12, 2017 at 12:02 AM, Joe wrote: >> Hi, >> >> question says it all. I looked through the basic and advanced >> indexing, >> but I could not find the rule that is applied to make >> x[np.newaxis,:] and x[np.newaxis] the same. > > I think it's the general rule that all indexing expressions have an > invisible "..." on the right edge. For example, x[i][j][k] is an > inefficient and IMO somewhat confusing way to write x[i, j, k], > because x[i][j][k] is interpreted as: > > -> x[i, ...][j, ...][k, ...] > -> x[i, :, :][j, :][k] > > That this also applies to newaxis is a little surprising, but I guess > consistent. > > -n From sebastian at sipsolutions.net Tue Dec 12 08:32:58 2017 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Tue, 12 Dec 2017 14:32:58 +0100 Subject: [Numpy-discussion] Which rule makes x[np.newaxis, :] and x[np.newaxis] equivalent? In-Reply-To: <3711a746661e352f5507e8fc2bc4e97b@posteo.de> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> Message-ID: <1513085578.14576.3.camel@sipsolutions.net> On Tue, 2017-12-12 at 14:19 +0100, Joe wrote: > Ah, ok, now that I knew what to look for I guess I found it: > > "If the number of objects in the selection tuple is less than N , > then :? > is assumed for any subsequent dimensions." > > https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.indexing.htm > l > > This is the one, right? > Yeah, plus if it is not a tuple, it actually behaves the same as a tuple, e.g. `arr[obj]` is identical to `arr[obj,]` (or `arr[(obj,)]` which is the same). There are some weird exception when obj is a list a sequence but not an array. Note also that while everything has an implicit `, ...` at the end of indexing, if you have exactly as many integers to index as dimensions you get a scalar, if you would add the Ellipsis you would get an array back. Anyway, too many weird details for day to day stuff :). And all of that should be covered in the docs? - Sebastian > > Am 12.12.2017 09:09 schrieb Nathaniel Smith: > > On Tue, Dec 12, 2017 at 12:02 AM, Joe wrote: > > > Hi, > > > > > > question says it all. I looked through the basic and advanced? > > > indexing, > > > but I could not find the rule that is applied to make > > > x[np.newaxis,:] and x[np.newaxis] the same. > > > > I think it's the general rule that all indexing expressions have an > > invisible "..." on the right edge. For example, x[i][j][k] is an > > inefficient and IMO somewhat confusing way to write x[i, j, k], > > because x[i][j][k] is interpreted as: > > > > -> x[i, ...][j, ...][k, ...] > > -> x[i, :, :][j, :][k] > > > > That this also applies to newaxis is a little surprising, but I > > guess > > consistent. > > > > -n > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: This is a digitally signed message part URL: From solarjoe at posteo.org Tue Dec 12 14:50:14 2017 From: solarjoe at posteo.org (Joe) Date: Tue, 12 Dec 2017 20:50:14 +0100 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: <1513085578.14576.3.camel@sipsolutions.net> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> Message-ID: <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Hi, the best example I found was this one: https://stackoverflow.com/a/29319864/7919597 def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True # The magic happens here if scalar_input: return np.squeeze(ret) return ret Is this as good as it gets or do you have other suggestions? Cheers, Joe From robert.kern at gmail.com Tue Dec 12 14:57:01 2017 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 13 Dec 2017 04:57:01 +0900 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On Wed, Dec 13, 2017 at 4:50 AM, Joe wrote: > > Hi, > > the best example I found was this one: > > https://stackoverflow.com/a/29319864/7919597 > > def func_for_scalars_or_vectors(x): > x = np.asarray(x) > scalar_input = False > if x.ndim == 0: > x = x[None] # Makes x 1D > scalar_input = True > > # The magic happens here > > if scalar_input: > return np.squeeze(ret) > return ret > > Is this as good as it gets or do you have other suggestions? In general structure, yeah. I'd probably use `np.isscalar(x)` for the test and `x = np.atleast_1d(x)` for the coercion for readability, but otherwise, that's it. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirillbalunov at gmail.com Tue Dec 12 15:00:40 2017 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 12 Dec 2017 23:00:40 +0300 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On minor thing that instead of 'ret' there should be 'x'. With kind regards, -gdg On Dec 12, 2017 22:51, "Joe" wrote: Hi, the best example I found was this one: https://stackoverflow.com/a/29319864/7919597 def func_for_scalars_or_vectors(x): x = np.asarray(x) scalar_input = False if x.ndim == 0: x = x[None] # Makes x 1D scalar_input = True # The magic happens here if scalar_input: return np.squeeze(ret) return ret Is this as good as it gets or do you have other suggestions? Cheers, Joe _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Tue Dec 12 15:03:58 2017 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 13 Dec 2017 05:03:58 +0900 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On Wed, Dec 13, 2017 at 5:00 AM, Kirill Balunov wrote: > > On minor thing that instead of 'ret' there should be 'x'. No, `x` is the input. The code that actually does the computation (elided here by the `# The magic happens here` comment) would have assigned to `ret`. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From kirillbalunov at gmail.com Tue Dec 12 15:06:58 2017 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Tue, 12 Dec 2017 23:06:58 +0300 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: Oh, sorry for noise... With kind regards, -gdg On Dec 12, 2017 23:05, "Robert Kern" wrote: > On Wed, Dec 13, 2017 at 5:00 AM, Kirill Balunov > wrote: > > > > On minor thing that instead of 'ret' there should be 'x'. > > No, `x` is the input. The code that actually does the computation (elided > here by the `# The magic happens here` comment) would have assigned to > `ret`. > > -- > Robert Kern > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wieser.eric+numpy at gmail.com Tue Dec 12 15:52:52 2017 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Tue, 12 Dec 2017 20:52:52 +0000 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the better option there. I?d maybe suggest not special-casing 0d arrays though, and using: def func_for_scalars_or_vectors(x): x = np.asanyarray(x) # convert scalars to 0d arrays # The magic happens here return ret[()] # convert 0d arrays to scalars Eric On Tue, 12 Dec 2017 at 11:58 Robert Kern wrote: On Wed, Dec 13, 2017 at 4:50 AM, Joe wrote: > > > > Hi, > > > > the best example I found was this one: > > > > https://stackoverflow.com/a/29319864/7919597 > > > > def func_for_scalars_or_vectors(x): > > x = np.asarray(x) > > scalar_input = False > > if x.ndim == 0: > > x = x[None] # Makes x 1D > > scalar_input = True > > > > # The magic happens here > > > > if scalar_input: > > return np.squeeze(ret) > > return ret > > > > Is this as good as it gets or do you have other suggestions? > > In general structure, yeah. I'd probably use `np.isscalar(x)` for the test > and `x = np.atleast_1d(x)` for the coercion for readability, but otherwise, > that's it. > > -- > Robert Kern > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Tue Dec 12 16:03:52 2017 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 13 Dec 2017 06:03:52 +0900 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On Wed, Dec 13, 2017 at 5:52 AM, Eric Wieser wrote: > Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the > better option there. > > I?d maybe suggest not special-casing 0d arrays though, and using: > > def func_for_scalars_or_vectors(x): > x = np.asanyarray(x) # convert scalars to 0d arrays > > # The magic happens here > > return ret[()] # convert 0d arrays to scalars > > Good call. I didn't think that the empty tuple was valid for anything but indexing into 0d arrays, but of course, following the rules of indexing, it works as required for other arrays too. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From millman at berkeley.edu Tue Dec 12 17:45:30 2017 From: millman at berkeley.edu (Jarrod Millman) Date: Tue, 12 Dec 2017 14:45:30 -0800 Subject: [Numpy-discussion] NEP process PR Message-ID: Hi all, I've started working on the proposal discussed in this thread: https://mail.python.org/pipermail/numpy-discussion/2017-December/077481.html here: https://github.com/numpy/numpy/pull/10213 You can see how I modified PEP 1 here: https://github.com/numpy/numpy/pull/10213/commits/eaf788940dee7d0f1c7922fac70a87144de89656 I used numbers (i.e., ``nep-0000.rst``) in the file names, since it seems more standard and numbers are easier to refer to in discussions. I don't have a strong preference, so I am happy to change it. If using numbers seems reasonable, I will number the existing NEPs. Moreover, if the preamble seems reasonable, I will go through the existing NEPs and make sure they all have compliant headers. For now, I think auto-generating the index is unnecessary. Once we are happy with the purpose and template NEPs as well as automatically publish to GH pages, we can always go back and write a little script to autogenerate the index using the preamble information. Finally, I started preparing to host the NEPs online at: http://numpy.github.io/neps If that seems reasonable, I will need someone to create a neps repo. We also need to decide whether to move ``numpy/doc/neps`` to the master branch of the new neps repo or whether to leave it where it is. I don't have a strong opinion either way. However, if no one else minds leaving it where it is, not moving it is slightly less work. Either way, the work is trivial. Regardless of where the source resides, we can host the generated web page in the same location (i.e., http://numpy.github.io/neps). It is probably also worth having https://docs.scipy.org/doc/numpy/neps/ redirect to http://numpy.github.io/neps Best regards, Jarrod From mark.campanelli at gmail.com Tue Dec 12 20:07:53 2017 From: mark.campanelli at gmail.com (Mark Campanelli) Date: Tue, 12 Dec 2017 18:07:53 -0700 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: The different approaches and corresponding "code bloat" here is one of the most annoying things I have found about using numpy+scipy. Furthermore, the flip side to the handling of the inputs, including both type and shape, is getting the output to match the input, including both type and shape. Getting the output correct is exacerbated by different intermediate functions not following any standards whatsoever around returning an output that "matches" the input. This makes me long for strong typing in Python and even tempts me to port my production code to something "safe" like Rust (including proper generic types and concurrency-safety guaranteed by the compiler). I think I saw some other discussion recently about numpy joining forces with Python 3's gradual type system. Is there any draft formal proposal for this yet? If numpy+scipy wants to scale to "bigger" projects, I think it behooves the community to get rid of this messiness. On Tue, Dec 12, 2017 at 2:03 PM, Robert Kern wrote: > On Wed, Dec 13, 2017 at 5:52 AM, Eric Wieser > wrote: > >> Using np.iscalar is a bad idea, as it fails for 0d arrays. x.ndim is the >> better option there. >> >> I?d maybe suggest not special-casing 0d arrays though, and using: >> >> def func_for_scalars_or_vectors(x): >> x = np.asanyarray(x) # convert scalars to 0d arrays >> >> # The magic happens here >> >> return ret[()] # convert 0d arrays to scalars >> >> Good call. I didn't think that the empty tuple was valid for anything but > indexing into 0d arrays, but of course, following the rules of indexing, it > works as required for other arrays too. > > -- > Robert Kern > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.h.vankerkwijk at gmail.com Tue Dec 12 21:20:35 2017 From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk) Date: Tue, 12 Dec 2017 21:20:35 -0500 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: The real magic happens when you ducktype, and ensure your function works both for arrays and scalars on its own. This is more often possible than you might think! If you really need, e.g., the shape, you can do `getattr(input, 'shape', ())` and things may well work for scalars, and also for objects that behave somewhat like arrays ("duck arrays"). If you do go for coercion, I agree with Eric' comment about `np.asanyarray` - it may now just work for `MaskedArray` (but not for, e.g., SparseArray; that needs real duck-typing). All the best, Marten From shoyer at gmail.com Tue Dec 12 20:49:14 2017 From: shoyer at gmail.com (Stephan Hoyer) Date: Wed, 13 Dec 2017 01:49:14 +0000 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On Tue, Dec 12, 2017 at 5:07 PM Mark Campanelli wrote: > I think I saw some other discussion recently about numpy joining forces > with Python 3's gradual type system. Is there any draft formal proposal for > this yet? If numpy+scipy wants to scale to "bigger" projects, I think it > behooves the community to get rid of this messiness. > We're still figuring this out, but if you're interested please follow along (and pitch in!): https://github.com/numpy/numpy_stubs https://github.com/numpy/numpy/issues/7370 https://github.com/python/typing/issues/516 Cheers, Stephan -------------- next part -------------- An HTML attachment was scrubbed... URL: From shoyer at gmail.com Wed Dec 13 01:19:48 2017 From: shoyer at gmail.com (Stephan Hoyer) Date: Wed, 13 Dec 2017 06:19:48 +0000 Subject: [Numpy-discussion] What is the pythonic way to write a function that handles arrays and scalars? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: On Tue, Dec 12, 2017 at 6:20 PM Marten van Kerkwijk < m.h.vankerkwijk at gmail.com> wrote: > The real magic happens when you ducktype, and ensure your function > works both for arrays and scalars on its own. This is more often > possible than you might think! Sadly, this still doesn't work in a type-stable way. NumPy's ufuncs convert 0-dimensional arrays into scalars. The typing rules for functions like np.sin() look like: - scalar or 0d array -> scalar - 1d or higher dimensional array -> array I'm not entirely sure, but I suspect this was a practical rather than principled choice. NumPy scalars are "duck arrays" of sorts (e.g., with shape and dtype attributes) which helps to some extent, but the bugs that slip through are even harder to understand. This wart reminds me of how mixed basic/advanced indexing reorders sliced dimensions to make the result more "intuitive", which only works in some cases. I usually favor coercing all arguments to my functions to numpy arrays with np.asarray(), but to guarantee the return type you would also need to coerce it with np.asarray(), too. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solarjoe at posteo.org Wed Dec 13 15:43:57 2017 From: solarjoe at posteo.org (Joe) Date: Wed, 13 Dec 2017 21:43:57 +0100 Subject: [Numpy-discussion] Does x[True] trigger basic or advanced indexing? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: Hi, yet another question. I looked through the indexing rules in the documentation but I count not find which one applies to x[True] and x[False] that might e.g result from import numpy as np x = np.array(3) x[x>5] x[x<1] x[True] x[False] x = np.random.rand(2,3) x[x>5] x[x<1] x[True] x[False] I understood that they are equivalent to x[(False,)] I tested it and it looks like advanced indexing, but I try to unterstand the logic behind this, if there is any :) In x[x<1] the x<1 is a mask and thus I guess it is a "tuple with at least one sequence object or ndarray (of data type integer or bool)", right? Or will x[True] trigger basic indexing as it is "a tuple of integers" because True will be converted to Int? Cheers, Joe From wieser.eric+numpy at gmail.com Wed Dec 13 15:49:56 2017 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Wed, 13 Dec 2017 20:49:56 +0000 Subject: [Numpy-discussion] Does x[True] trigger basic or advanced indexing? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: Increasingly, NumPy does not considers booleans to be integer types, and indexing is one of these cases. So no, it will not be treated as a tuple of integers, but as a 0d mask Eric On Wed, 13 Dec 2017 at 12:44 Joe wrote: > Hi, > > yet another question. > > I looked through the indexing rules in the > documentation but I count not find which one > applies to x[True] and x[False] > > that might e.g result from > > import numpy as np > x = np.array(3) > x[x>5] > x[x<1] > x[True] > x[False] > > x = np.random.rand(2,3) > x[x>5] > x[x<1] > x[True] > x[False] > > I understood that they are equivalent to > > x[(False,)] > > I tested it and it looks like advanced indexing, > but I try to unterstand the logic behind this, > if there is any :) > > In x[x<1] the x<1 is a mask and thus I guess it is a > "tuple with at least one sequence object or ndarray (of data type > integer or bool)", right? > > Or will x[True] trigger basic indexing as it is "a tuple of integers" > because True will be converted to Int? > > Cheers, > Joe > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Dec 13 17:54:17 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 13 Dec 2017 15:54:17 -0700 Subject: [Numpy-discussion] NumPy 1.14.0rc1 release Message-ID: Hi All, On behalf of the NumPy team, I am pleased to announce NumPy 1.14.0rc1. Numpy 1.14.0rc1 is the result of seven months of work and contains a large number of bug fixes and new features, along with several changes with potential compatibility issues. The major change that users will notice are the stylistic changes in the way numpy arrays and scalars are printed, a change that will affect doctests. See the release notes for details on how to preserve the old style printing when needed. A major decision affecting future development concerns the schedule for dropping Python 2.7 support in the runup to 2020. The decision has been made to support 2.7 for all releases made in 2018, with the last release being designated a long term release with support for bug fixes extending through the end of 2019. Starting from January, 2019 support for 2.7 will be dropped in all new releases. More details can be found in the relevant NEP . This release supports Python 2.7 and 3.4 - 3.6. Wheels for the pre-release are available on PyPI. Source tarballs, zipfiles, release notes, and the changelog are available on github . *Highlights* - The ``np.einsum`` function uses BLAS when possible - ``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle files with arbitrary Python supported encoding. - Major improvements to printing of NumPy arrays and scalars. *New functions* - ``parametrize``: decorator added to numpy.testing - ``chebinterpolate``: Interpolate function at Chebyshev points. - ``format_float_positional`` and ``format_float_scientific`` : format floating-point scalars unambiguously with control of rounding and padding. - ``PyArray_ResolveWritebackIfCopy`` and ``PyArray_SetWritebackIfCopyBase``, new C-API functions useful in achieving PyPy compatibity. *Contributors* A total of 101 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Alexey Brodkin + - Allan Haldane - Andras Deak + - Andrew Lawson + - Antoine Pitrou - Bernhard M. Wiedemann + - Bob Eldering + - Brandon Carter - CJ Carey - Charles Harris - Chris Lamb - Christoph Boeddeker + - Christoph Gohlke - Daniel Hrisca + - Daniel Smith - Danny Hermes - David Freese - David Hagen - David Linke + - David Schaefer + - Dillon Niederhut + - Egor Panfilov + - Emilien Kofman - Eric Wieser - Erik Bray + - Erik Quaeghebeur + - Garry Polley + - Gunjan + - Henke Adolfsson + - Hidehiro NAGAOKA + - Hong Xu + - Iryna Shcherbina + - Jaime Fernandez - James Bourbeau + - Jamie Townsend + - Jarrod Millman - Jean Helie + - Jeroen Demeyer + - John Goetz + - John Kirkham - John Zwinck - Jonathan Helmus - Joseph Fox-Rabinovitz - Joseph Paul Cohen + - Joshua Leahy + - Julian Taylor - J?rg D?pfert + - Keno Goertz + - Kevin Sheppard + - Kexuan Sun + - Konrad Kapp + - Kristofor Maynard + - Licht Takeuchi + - Lo?c Est?ve - Lukas Mericle + - Marten van Kerkwijk - Matheus Portela + - Matthew Brett - Matti Picus - Michael Lamparski + - Michael Odintsov + - Michael Schnaitter + - Michael Seifert - Mike Nolta - Nathaniel J. Smith - Nelle Varoquaux + - Nicholas Del Grosso + - Nico Schl?mer + - Oleg Zabluda + - Oleksandr Pavlyk - Pauli Virtanen - Pim de Haan + - Ralf Gommers - Robert T. McGibbon + - Roland Kaufmann - Sebastian Berg - Serhiy Storchaka + - Shitian Ni + - Spencer Hill + - Srinivas Reddy Thatiparthy + - Stefan Winkler + - Stephan Hoyer - Steven Maude + - SuperBo + - Thomas K?ppe + - Toon Verstraelen - Vedant Misra + - Warren Weckesser - Wirawan Purwanto + - Yang Li + - Ziyan + - chaoyu3 + - hemildesai + - k_kapp at yahoo.com + - nickdg + - orbit-stabilizer + - schnaitterm + - solarjoe - wufangjie + - xoviat + - ?lie Gouzien + Enjoy, Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From solarjoe at posteo.org Thu Dec 14 07:27:08 2017 From: solarjoe at posteo.org (Joe) Date: Thu, 14 Dec 2017 13:27:08 +0100 Subject: [Numpy-discussion] =?utf-8?q?Does_x=5BTrue=5D_trigger_basic_or_a?= =?utf-8?q?dvanced_indexing=3F?= In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: Hello, thanks for you feedback. Sorry, if thie question is stupid and the case below does not make sense. I am just trying to understand the logic. For x = np.random.rand(2,3) x[True] x[(True,)] or x[False] x[(False,)] where True and False are not arrays, it will pick the first or second row. Is this basic indexing then with one the rules - obj is an integer - obj is a tuple of slice objects and integers. ? Am 13.12.2017 21:49 schrieb Eric Wieser: > Increasingly, NumPy does not considers booleans to be integer types, > and indexing is one of these cases. > > So no, it will not be treated as a tuple of integers, but as a 0d mask > > Eric > > On Wed, 13 Dec 2017 at 12:44 Joe wrote: > >> Hi, >> >> yet another question. >> >> I looked through the indexing rules in the >> documentation but I count not find which one >> applies to x[True] and x[False] >> >> that might e.g result from >> >> import numpy as np >> x = np.array(3) >> x[x>5] >> x[x<1] >> x[True] >> x[False] >> >> x = np.random.rand(2,3) >> x[x>5] >> x[x<1] >> x[True] >> x[False] >> >> I understood that they are equivalent to >> >> x[(False,)] >> >> I tested it and it looks like advanced indexing, >> but I try to unterstand the logic behind this, >> if there is any :) >> >> In x[x<1] the x<1 is a mask and thus I guess it is a >> "tuple with at least one sequence object or ndarray (of data type >> integer or bool)", right? >> >> Or will x[True] trigger basic indexing as it is "a tuple of >> integers" >> because True will be converted to Int? >> >> Cheers, >> Joe >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion [1] > > > Links: > ------ > [1] https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From wieser.eric+numpy at gmail.com Thu Dec 14 11:24:09 2017 From: wieser.eric+numpy at gmail.com (Eric Wieser) Date: Thu, 14 Dec 2017 16:24:09 +0000 Subject: [Numpy-discussion] Does x[True] trigger basic or advanced indexing? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: It sounds like you're using an old version of numpy, where boolean scalars were interpreted as integers. What version are you using? Eric On Thu, Dec 14, 2017, 04:27 Joe wrote: > Hello, > thanks for you feedback. > > Sorry, if thie question is stupid and the case below does not make > sense. > I am just trying to understand the logic. > For > > x = np.random.rand(2,3) > > x[True] > x[(True,)] > > or > > x[False] > x[(False,)] > > where True and False are not arrays, > it will pick the first or second row. > > Is this basic indexing then with one the rules > - obj is an integer > - obj is a tuple of slice objects and integers. > ? > > > Am 13.12.2017 21:49 schrieb Eric Wieser: > > Increasingly, NumPy does not considers booleans to be integer types, > > and indexing is one of these cases. > > > > So no, it will not be treated as a tuple of integers, but as a 0d mask > > > > Eric > > > > On Wed, 13 Dec 2017 at 12:44 Joe wrote: > > > >> Hi, > >> > >> yet another question. > >> > >> I looked through the indexing rules in the > >> documentation but I count not find which one > >> applies to x[True] and x[False] > >> > >> that might e.g result from > >> > >> import numpy as np > >> x = np.array(3) > >> x[x>5] > >> x[x<1] > >> x[True] > >> x[False] > >> > >> x = np.random.rand(2,3) > >> x[x>5] > >> x[x<1] > >> x[True] > >> x[False] > >> > >> I understood that they are equivalent to > >> > >> x[(False,)] > >> > >> I tested it and it looks like advanced indexing, > >> but I try to unterstand the logic behind this, > >> if there is any :) > >> > >> In x[x<1] the x<1 is a mask and thus I guess it is a > >> "tuple with at least one sequence object or ndarray (of data type > >> integer or bool)", right? > >> > >> Or will x[True] trigger basic indexing as it is "a tuple of > >> integers" > >> because True will be converted to Int? > >> > >> Cheers, > >> Joe > >> _______________________________________________ > >> NumPy-Discussion mailing list > >> NumPy-Discussion at python.org > >> https://mail.python.org/mailman/listinfo/numpy-discussion [1] > > > > > > Links: > > ------ > > [1] https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.barker at noaa.gov Thu Dec 14 12:15:58 2017 From: chris.barker at noaa.gov (Chris Barker - NOAA Federal) Date: Thu, 14 Dec 2017 09:15:58 -0800 Subject: [Numpy-discussion] NumPy 1.14.0rc1 release In-Reply-To: References: Message-ID: <-5551096436323765362@unknownmsgid> Thanks Chuck! And a huge thanks to that awesome list of contributors!!! -Chris Sent from my iPhone On Dec 13, 2017, at 2:55 PM, Charles R Harris wrote: Hi All, On behalf of the NumPy team, I am pleased to announce NumPy 1.14.0rc1. Numpy 1.14.0rc1 is the result of seven months of work and contains a large number of bug fixes and new features, along with several changes with potential compatibility issues. The major change that users will notice are the stylistic changes in the way numpy arrays and scalars are printed, a change that will affect doctests. See the release notes for details on how to preserve the old style printing when needed. A major decision affecting future development concerns the schedule for dropping Python 2.7 support in the runup to 2020. The decision has been made to support 2.7 for all releases made in 2018, with the last release being designated a long term release with support for bug fixes extending through the end of 2019. Starting from January, 2019 support for 2.7 will be dropped in all new releases. More details can be found in the relevant NEP . This release supports Python 2.7 and 3.4 - 3.6. Wheels for the pre-release are available on PyPI. Source tarballs, zipfiles, release notes, and the changelog are available on github . *Highlights* - The ``np.einsum`` function uses BLAS when possible - ``genfromtxt``, ``loadtxt``, ``fromregex`` and ``savetxt`` can now handle files with arbitrary Python supported encoding. - Major improvements to printing of NumPy arrays and scalars. *New functions* - ``parametrize``: decorator added to numpy.testing - ``chebinterpolate``: Interpolate function at Chebyshev points. - ``format_float_positional`` and ``format_float_scientific`` : format floating-point scalars unambiguously with control of rounding and padding. - ``PyArray_ResolveWritebackIfCopy`` and ``PyArray_SetWritebackIfCopyBase``, new C-API functions useful in achieving PyPy compatibity. *Contributors* A total of 101 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Alexey Brodkin + - Allan Haldane - Andras Deak + - Andrew Lawson + - Antoine Pitrou - Bernhard M. Wiedemann + - Bob Eldering + - Brandon Carter - CJ Carey - Charles Harris - Chris Lamb - Christoph Boeddeker + - Christoph Gohlke - Daniel Hrisca + - Daniel Smith - Danny Hermes - David Freese - David Hagen - David Linke + - David Schaefer + - Dillon Niederhut + - Egor Panfilov + - Emilien Kofman - Eric Wieser - Erik Bray + - Erik Quaeghebeur + - Garry Polley + - Gunjan + - Henke Adolfsson + - Hidehiro NAGAOKA + - Hong Xu + - Iryna Shcherbina + - Jaime Fernandez - James Bourbeau + - Jamie Townsend + - Jarrod Millman - Jean Helie + - Jeroen Demeyer + - John Goetz + - John Kirkham - John Zwinck - Jonathan Helmus - Joseph Fox-Rabinovitz - Joseph Paul Cohen + - Joshua Leahy + - Julian Taylor - J?rg D?pfert + - Keno Goertz + - Kevin Sheppard + - Kexuan Sun + - Konrad Kapp + - Kristofor Maynard + - Licht Takeuchi + - Lo?c Est?ve - Lukas Mericle + - Marten van Kerkwijk - Matheus Portela + - Matthew Brett - Matti Picus - Michael Lamparski + - Michael Odintsov + - Michael Schnaitter + - Michael Seifert - Mike Nolta - Nathaniel J. Smith - Nelle Varoquaux + - Nicholas Del Grosso + - Nico Schl?mer + - Oleg Zabluda + - Oleksandr Pavlyk - Pauli Virtanen - Pim de Haan + - Ralf Gommers - Robert T. McGibbon + - Roland Kaufmann - Sebastian Berg - Serhiy Storchaka + - Shitian Ni + - Spencer Hill + - Srinivas Reddy Thatiparthy + - Stefan Winkler + - Stephan Hoyer - Steven Maude + - SuperBo + - Thomas K?ppe + - Toon Verstraelen - Vedant Misra + - Warren Weckesser - Wirawan Purwanto + - Yang Li + - Ziyan + - chaoyu3 + - hemildesai + - k_kapp at yahoo.com + - nickdg + - orbit-stabilizer + - schnaitterm + - solarjoe - wufangjie + - xoviat + - ?lie Gouzien + Enjoy, Chuck _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Thu Dec 14 15:37:38 2017 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 14 Dec 2017 21:37:38 +0100 Subject: [Numpy-discussion] Does x[True] trigger basic or advanced indexing? In-Reply-To: References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> Message-ID: <1513283858.26462.1.camel@sipsolutions.net> On Thu, 2017-12-14 at 16:24 +0000, Eric Wieser wrote: > It sounds like you're using an old version of numpy, where boolean > scalars were interpreted as integers. > What version are you using? > Eric > Indeed, you are maybe using a pre 1.9 version (post 1.9 should at least have a DeprecationWarning or some such, though you might not notice it IIRC). For newer versions you should get boolean indexing, the result of it may be a bit confusing. It is advanced indexing, basically with False giving you an empty array (with an extra dimension of size 0) and True being much like an `np.newaxis`. It all makes perfect sense if you think of it of a 0-d array picking.... The same thing is true for example for lists of booleans. - Sebastian > On Thu, Dec 14, 2017, 04:27 Joe wrote: > > Hello, > > thanks for you feedback. > > > > Sorry, if thie question is stupid and the case below does not make > > sense. > > I am just trying to understand the logic. > > For > > > > x = np.random.rand(2,3) > > > > x[True] > > x[(True,)] > > > > or > > > > x[False] > > x[(False,)] > > > > where True and False are not arrays, > > it will pick the first or second row. > > > > Is this basic indexing then with one the rules > > - obj is an integer > > - obj is a tuple of slice objects and integers. > > ? > > > > > > Am 13.12.2017 21:49 schrieb Eric Wieser: > > > Increasingly, NumPy does not considers booleans to be integer > > types, > > > and indexing is one of these cases. > > > > > > So no, it will not be treated as a tuple of integers, but as a 0d > > mask > > > > > > Eric > > > > > > On Wed, 13 Dec 2017 at 12:44 Joe wrote: > > > > > >> Hi, > > >> > > >> yet another question. > > >> > > >> I looked through the indexing rules in the > > >> documentation but I count not find which one > > >> applies to x[True] and x[False] > > >> > > >> that might e.g result from > > >> > > >> import numpy as np > > >> x = np.array(3) > > >> x[x>5] > > >> x[x<1] > > >> x[True] > > >> x[False] > > >> > > >> x = np.random.rand(2,3) > > >> x[x>5] > > >> x[x<1] > > >> x[True] > > >> x[False] > > >> > > >> I understood that they are equivalent to > > >> > > >> x[(False,)] > > >> > > >> I tested it and it looks like advanced indexing, > > >> but I try to unterstand the logic behind this, > > >> if there is any :) > > >> > > >> In x[x<1] the x<1 is a mask and thus I guess it is a > > >> "tuple with at least one sequence object or ndarray (of data > > type > > >> integer or bool)", right? > > >> > > >> Or will x[True] trigger basic indexing as it is "a tuple of > > >> integers" > > >> because True will be converted to Int? > > >> > > >> Cheers, > > >> Joe > > >> _______________________________________________ > > >> NumPy-Discussion mailing list > > >> NumPy-Discussion at python.org > > >> https://mail.python.org/mailman/listinfo/numpy-discussion [1] > > > > > > > > > Links: > > > ------ > > > [1] https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion at python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion at python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: This is a digitally signed message part URL: From solarjoe at posteo.org Fri Dec 15 01:55:29 2017 From: solarjoe at posteo.org (Joe) Date: Fri, 15 Dec 2017 07:55:29 +0100 Subject: [Numpy-discussion] =?utf-8?q?Does_x=5BTrue=5D_trigger_basic_or_a?= =?utf-8?q?dvanced_indexing=3F?= In-Reply-To: <1513283858.26462.1.camel@sipsolutions.net> References: <8a758fbcb74ba9537034dec9fe90d9e7@posteo.de> <3711a746661e352f5507e8fc2bc4e97b@posteo.de> <1513085578.14576.3.camel@sipsolutions.net> <9c7c4200-3571-f28f-6394-ce2abcf5e558@posteo.org> <1513283858.26462.1.camel@sipsolutions.net> Message-ID: <3099ab9a41fd25c03e0edf530e9aa6bb@posteo.de> Hi, you are right. I am using two different versions, Numpy 1.10.4 and 1.9.0 Both show this behavior. Numpy 1.11.1 also does, but now raises VisibleDeprecationWarning: using a boolean instead of an integer will result in an error in the future Thanks! Am 14.12.2017 21:37 schrieb Sebastian Berg: > On Thu, 2017-12-14 at 16:24 +0000, Eric Wieser wrote: >> It sounds like you're using an old version of numpy, where boolean >> scalars were interpreted as integers. >> What version are you using? >> Eric >> > > Indeed, you are maybe using a pre 1.9 version (post 1.9 should at least > have a DeprecationWarning or some such, though you might not notice it > IIRC). > For newer versions you should get boolean indexing, the result of it > may be a bit confusing. It is advanced indexing, basically with False > giving you an empty array (with an extra dimension of size 0) and True > being much like an `np.newaxis`. > It all makes perfect sense if you think of it of a 0-d array > picking.... > > The same thing is true for example for lists of booleans. > > - Sebastian > > > >> On Thu, Dec 14, 2017, 04:27 Joe wrote: >> > Hello, >> > thanks for you feedback. >> > >> > Sorry, if thie question is stupid and the case below does not make >> > sense. >> > I am just trying to understand the logic. >> > For >> > >> > x = np.random.rand(2,3) >> > >> > x[True] >> > x[(True,)] >> > >> > or >> > >> > x[False] >> > x[(False,)] >> > >> > where True and False are not arrays, >> > it will pick the first or second row. >> > >> > Is this basic indexing then with one the rules >> > - obj is an integer >> > - obj is a tuple of slice objects and integers. >> > ? >> > >> > >> > Am 13.12.2017 21:49 schrieb Eric Wieser: >> > > Increasingly, NumPy does not considers booleans to be integer >> > types, >> > > and indexing is one of these cases. >> > > >> > > So no, it will not be treated as a tuple of integers, but as a 0d >> > mask >> > > >> > > Eric >> > > >> > > On Wed, 13 Dec 2017 at 12:44 Joe wrote: >> > > >> > >> Hi, >> > >> >> > >> yet another question. >> > >> >> > >> I looked through the indexing rules in the >> > >> documentation but I count not find which one >> > >> applies to x[True] and x[False] >> > >> >> > >> that might e.g result from >> > >> >> > >> import numpy as np >> > >> x = np.array(3) >> > >> x[x>5] >> > >> x[x<1] >> > >> x[True] >> > >> x[False] >> > >> >> > >> x = np.random.rand(2,3) >> > >> x[x>5] >> > >> x[x<1] >> > >> x[True] >> > >> x[False] >> > >> >> > >> I understood that they are equivalent to >> > >> >> > >> x[(False,)] >> > >> >> > >> I tested it and it looks like advanced indexing, >> > >> but I try to unterstand the logic behind this, >> > >> if there is any :) >> > >> >> > >> In x[x<1] the x<1 is a mask and thus I guess it is a >> > >> "tuple with at least one sequence object or ndarray (of data >> > type >> > >> integer or bool)", right? >> > >> >> > >> Or will x[True] trigger basic indexing as it is "a tuple of >> > >> integers" >> > >> because True will be converted to Int? >> > >> >> > >> Cheers, >> > >> Joe >> > >> _______________________________________________ >> > >> NumPy-Discussion mailing list >> > >> NumPy-Discussion at python.org >> > >> https://mail.python.org/mailman/listinfo/numpy-discussion [1] >> > > >> > > >> > > Links: >> > > ------ >> > > [1] https://mail.python.org/mailman/listinfo/numpy-discussion >> > > >> > > _______________________________________________ >> > > NumPy-Discussion mailing list >> > > NumPy-Discussion at python.org >> > > https://mail.python.org/mailman/listinfo/numpy-discussion >> > _______________________________________________ >> > NumPy-Discussion mailing list >> > NumPy-Discussion at python.org >> > https://mail.python.org/mailman/listinfo/numpy-discussion >> > >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion at python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion From hannes at saao.ac.za Fri Dec 15 05:11:01 2017 From: hannes at saao.ac.za (Hannes Breytenbach) Date: Fri, 15 Dec 2017 12:11:01 +0200 (SAST) Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> Message-ID: <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> Hi devs! This is my first post to the discussion list! Has anyone tried to build numpy with python3.7.0a3? I get the following gcc errors during compile: . . . compiling C sources C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC compile options: '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c' gcc: numpy/random/mtrand/mtrand.c numpy/random/mtrand/mtrand.c: In function ?__Pyx__ExceptionSave?: numpy/random/mtrand/mtrand.c:40970:19: error: ?PyThreadState {aka struct _ts}? has no member named ?exc_type?; did you mean ?curexc_type?? *type = tstate->exc_type; ^~ . . . error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c numpy/random/mtrand/mtrand.c -o build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o -MMD -MF build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o.d" failed with exit status 1 Version info: ------------- gcc --version: gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 2017040 numpy version: 1.15.0.dev0+d233e1f The same error comes up when building via pip. I don't know enough about the underlying C code to know how to debug this. Any help would be greatly appreciated! Cheers, -- Hannes Breytenbach PhD Candidate South African Astronomical Observatory +27 82 726 9311 From njs at pobox.com Fri Dec 15 05:30:48 2017 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 15 Dec 2017 02:30:48 -0800 Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> Message-ID: Try upgrading cython? On Fri, Dec 15, 2017 at 2:11 AM, Hannes Breytenbach wrote: > Hi devs! > > This is my first post to the discussion list! > > Has anyone tried to build numpy with python3.7.0a3? > > I get the following gcc errors during compile: > > . > . > . > > compiling C sources > C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC > > compile options: '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c' > gcc: numpy/random/mtrand/mtrand.c > numpy/random/mtrand/mtrand.c: In function ?__Pyx__ExceptionSave?: > numpy/random/mtrand/mtrand.c:40970:19: error: ?PyThreadState {aka struct _ts}? has no member named ?exc_type?; did you mean ?curexc_type?? > *type = tstate->exc_type; > ^~ > . > . > . > > error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c numpy/random/mtrand/mtrand.c -o build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o -MMD -MF build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o.d" failed with exit status 1 > > > > Version info: > ------------- > gcc --version: gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 2017040 > numpy version: 1.15.0.dev0+d233e1f > > > The same error comes up when building via pip. I don't know enough about the underlying C code to know how to debug this. Any help would be greatly appreciated! > > Cheers, > > -- > Hannes Breytenbach > > PhD Candidate > South African Astronomical Observatory > +27 82 726 9311 > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -- Nathaniel J. Smith -- https://vorpus.org From hannes at saao.ac.za Fri Dec 15 05:42:25 2017 From: hannes at saao.ac.za (Hannes Breytenbach) Date: Fri, 15 Dec 2017 12:42:25 +0200 (SAST) Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> Message-ID: <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> I don't think this is a cython version issue - cloned the latest version from git yesterday... python3.7 -c "import cython; print(cython.__version__)" 0.28a0 ----- Original Message ----- From: "Nathaniel Smith" To: "Discussion of Numerical Python" Sent: Friday, December 15, 2017 12:30:48 PM Subject: Re: [Numpy-discussion] building numpy with python3.7 Try upgrading cython? On Fri, Dec 15, 2017 at 2:11 AM, Hannes Breytenbach wrote: > Hi devs! > > This is my first post to the discussion list! > > Has anyone tried to build numpy with python3.7.0a3? > > I get the following gcc errors during compile: > > . > . > . > > compiling C sources > C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC > > compile options: '-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c' > gcc: numpy/random/mtrand/mtrand.c > numpy/random/mtrand/mtrand.c: In function ?__Pyx__ExceptionSave?: > numpy/random/mtrand/mtrand.c:40970:19: error: ?PyThreadState {aka struct _ts}? has no member named ?exc_type?; did you mean ?curexc_type?? > *type = tstate->exc_type; > ^~ > . > . > . > > error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Inumpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/include/python3.7m -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.7/numpy/core/src/private -Ibuild/src.linux-x86_64-3.7/numpy/core/src/npymath -c numpy/random/mtrand/mtrand.c -o build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o -MMD -MF build/temp.linux-x86_64-3.7/numpy/random/mtrand/mtrand.o.d" failed with exit status 1 > > > > Version info: > ------------- > gcc --version: gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 2017040 > numpy version: 1.15.0.dev0+d233e1f > > > The same error comes up when building via pip. I don't know enough about the underlying C code to know how to debug this. Any help would be greatly appreciated! > > Cheers, > > -- > Hannes Breytenbach > > PhD Candidate > South African Astronomical Observatory > +27 82 726 9311 > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion !DSPAM:5a33a46a14261384311406! From njs at pobox.com Fri Dec 15 05:53:39 2017 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 15 Dec 2017 02:53:39 -0800 Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> Message-ID: On Fri, Dec 15, 2017 at 2:42 AM, Hannes Breytenbach wrote: > > I don't think this is a cython version issue - cloned the latest version from git yesterday... > > python3.7 -c "import cython; print(cython.__version__)" > 0.28a0 It is a cython version issue: https://github.com/cython/cython/issues/1955 It's supposed to be fixed though, so I don't know why you it isn't working for you. Are you sure that cython is installed in the same virtualenv as you're using to build numpy? If you were using a numpy sdist then it would make sense because we include the pre-generated .c files in the sdists instead of running cython, but given that you're building from a numpy checkout I dunno. -n -- Nathaniel J. Smith -- https://vorpus.org From tcaswell at gmail.com Fri Dec 15 08:25:39 2017 From: tcaswell at gmail.com (Thomas Caswell) Date: Fri, 15 Dec 2017 13:25:39 +0000 Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> Message-ID: I have been building numpy master with CPython master but am also using cython master. My not-pretty script for doing this is: #! /usr/bin/bash set -e TARGET_ENV=bleeding OSPATH=~/source/other_source/ pushd $OSPATH/cpython/ git pull git clean -xfd ./configure make -j 9 ./python -m venv --copies --clear ~/.virtualenvs/$TARGET_ENV popd source ~/.virtualenvs/$TARGET_ENV/bin/activate master_build () { git checkout master git pull git clean -xfd pip install -v . } pushd $OSPATH/cython master_build popd pushd $OSPATH/numpy master_build popd Tom On Fri, Dec 15, 2017 at 5:54 AM Nathaniel Smith wrote: > On Fri, Dec 15, 2017 at 2:42 AM, Hannes Breytenbach > wrote: > > > > I don't think this is a cython version issue - cloned the latest version > from git yesterday... > > > > python3.7 -c "import cython; print(cython.__version__)" > > 0.28a0 > > It is a cython version issue: https://github.com/cython/cython/issues/1955 > > It's supposed to be fixed though, so I don't know why you it isn't > working for you. Are you sure that cython is installed in the same > virtualenv as you're using to build numpy? If you were using a numpy > sdist then it would make sense because we include the pre-generated .c > files in the sdists instead of running cython, but given that you're > building from a numpy checkout I dunno. > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gina at numfocus.org Fri Dec 15 12:18:55 2017 From: gina at numfocus.org (Gina Helfrich) Date: Fri, 15 Dec 2017 11:18:55 -0600 Subject: [Numpy-discussion] NumFOCUS End-of-Year Fundraising Drive (NumPy fiscal sponsor) Message-ID: Hi folks, NumFOCUS, the nonprofit that provides fiscal sponsorship for NumPy, kicks off our end of year fundraising drive officially on Monday and would really appreciate your support! Andy Terrel, NumFOCUS Board President, has generously put up the first $5,000 donation as a matching challenge to the community. For every dollar the community donates, Andy will donate a dollar, up to $5k. Please spread the word and use this link to donate: https://www.flipcause.com/secure/cause_pdetails/Mjc2NTg= Thanks so much for your support of NumPy and NumFOCUS! Cheers, Gina -- Dr. Gina Helfrich Communications Director, NumFOCUS gina at numfocus.org 512-222-5449 <(512)%20222-5449> -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.webmail at gmail.com Fri Dec 15 16:40:42 2017 From: jesper.webmail at gmail.com (Jesper Larsen) Date: Fri, 15 Dec 2017 22:40:42 +0100 Subject: [Numpy-discussion] masked_values behaviour Message-ID: Hi numpy people, I was just wondering whether this behaviour is intended: >>> import numpy as np >>> np.ma.masked_values(np.array([-32768.0]), np.int16(-32768)) masked_array(data = [-32768.], mask = False, fill_value = -32768.0) So the resulting masked array is not masked. On the other hand it is masked in the three cases below: >>> np.ma.masked_values(np.array([-32767.0]), np.int16(-32767)) masked_array(data = [--], mask = [ True], fill_value = -32767.0) >>> np.ma.masked_values(np.array([-32768.0]), -32768.0) masked_array(data = [--], mask = [ True], fill_value = -32768.0) >>> np.ma.masked_values(np.array([-32768.0]), -32768) masked_array(data = [--], mask = [ True], fill_value = -32768.0) Best regards, Jesper -------------- next part -------------- An HTML attachment was scrubbed... URL: From pmhobson at gmail.com Fri Dec 15 17:54:27 2017 From: pmhobson at gmail.com (Paul Hobson) Date: Fri, 15 Dec 2017 14:54:27 -0800 Subject: [Numpy-discussion] masked_values behaviour In-Reply-To: References: Message-ID: I think this is a floating point precision issue. https://docs.python.org/3.6/tutorial/floatingpoint.html On Fri, Dec 15, 2017 at 1:40 PM, Jesper Larsen wrote: > Hi numpy people, > > I was just wondering whether this behaviour is intended: > > >>> import numpy as np > >>> np.ma.masked_values(np.array([-32768.0]), np.int16(-32768)) > masked_array(data = [-32768.], > mask = False, > fill_value = -32768.0) > > So the resulting masked array is not masked. On the other hand it is > masked in the three cases below: > > >>> np.ma.masked_values(np.array([-32767.0]), np.int16(-32767)) > masked_array(data = [--], > mask = [ True], > fill_value = -32767.0) > > >>> np.ma.masked_values(np.array([-32768.0]), -32768.0) > masked_array(data = [--], > mask = [ True], > fill_value = -32768.0) > > >>> np.ma.masked_values(np.array([-32768.0]), -32768) > masked_array(data = [--], > mask = [ True], > fill_value = -32768.0) > > Best regards, > Jesper > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.h.vankerkwijk at gmail.com Sat Dec 16 16:48:09 2017 From: m.h.vankerkwijk at gmail.com (Marten van Kerkwijk) Date: Sat, 16 Dec 2017 16:48:09 -0500 Subject: [Numpy-discussion] masked_values behaviour In-Reply-To: References: Message-ID: Definitely a big! The underlying problem is: ``` In [23]: np.abs(np.int16(-32768)) Out[23]: -32768 ``` This is not great, but perhaps consistent with the logic that abs should return a value of the same dtype. It could be solved inside `masked_values` by using `np.abs(value, dtype=xnew.dtype)` Do you want to have a go at fixing this? Or raise an issue at https://github.com/numpy/numpy -- Marten From jesper.webmail at gmail.com Mon Dec 18 04:57:18 2017 From: jesper.webmail at gmail.com (Jesper Larsen) Date: Mon, 18 Dec 2017 10:57:18 +0100 Subject: [Numpy-discussion] masked_values behaviour In-Reply-To: References: Message-ID: Thanks for the responses Paul and Marten, I have raised an issue for the issue at: https://github.com/numpy/numpy/issues/10227 Best regards, Jesper 2017-12-16 22:48 GMT+01:00 Marten van Kerkwijk : > Definitely a big! The underlying problem is: > ``` > In [23]: np.abs(np.int16(-32768)) > Out[23]: -32768 > ``` > This is not great, but perhaps consistent with the logic that abs > should return a value of the same dtype. > > It could be solved inside `masked_values` by using `np.abs(value, > dtype=xnew.dtype)` > > Do you want to have a go at fixing this? Or raise an issue at > https://github.com/numpy/numpy > > -- Marten > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hannes at saao.ac.za Mon Dec 18 05:20:08 2017 From: hannes at saao.ac.za (Hannes Breytenbach) Date: Mon, 18 Dec 2017 12:20:08 +0200 (SAST) Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> Message-ID: <1524113059.6201087.1513592408374.JavaMail.zimbra@saao.ac.za> OK, thanks for the link to the issue. I'm not using virtual environments - I built python3.7 (and 2.7) with the `make altinstall` method. I managed to get the numpy build working on 3.7 by removing the `random/mtrand/mtrand.c` file so that it gets (re-)generated during the build using the latest cython version. Thanks for the help! ----- Original Message ----- From: "Nathaniel Smith" To: "Discussion of Numerical Python" Sent: Friday, December 15, 2017 12:53:39 PM Subject: Re: [Numpy-discussion] building numpy with python3.7 On Fri, Dec 15, 2017 at 2:42 AM, Hannes Breytenbach wrote: > > I don't think this is a cython version issue - cloned the latest version from git yesterday... > > python3.7 -c "import cython; print(cython.__version__)" > 0.28a0 It is a cython version issue: https://github.com/cython/cython/issues/1955 It's supposed to be fixed though, so I don't know why you it isn't working for you. Are you sure that cython is installed in the same virtualenv as you're using to build numpy? If you were using a numpy sdist then it would make sense because we include the pre-generated .c files in the sdists instead of running cython, but given that you're building from a numpy checkout I dunno. -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion !DSPAM:5a33a9d014269019821239! From hanno_lists at gmx.net Mon Dec 18 06:02:11 2017 From: hanno_lists at gmx.net (hanno_lists at gmx.net) Date: Mon, 18 Dec 2017 12:02:11 +0100 Subject: [Numpy-discussion] Applying logical operations along an axis of a boolean array? Message-ID: An HTML attachment was scrubbed... URL: From jni.soma at gmail.com Mon Dec 18 06:23:51 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Mon, 18 Dec 2017 22:23:51 +1100 Subject: [Numpy-discussion] Applying logical operations along an axis of a boolean array? In-Reply-To: References: Message-ID: <5d160b78-9440-47a8-903d-ca75ca31e807@Spark> Use ?np.all? or ?np.any?. On 18 Dec 2017, 10:02 PM +1100, hanno_lists at gmx.net, wrote: > Hi, > > is it possible, to apply a logical operation, such as AND or OR along a particular axis of a numpy array? > > Let's say I have an (n,m) array and I want to AND along the first axis, such that I get a (1,m) (or just (m,) dimensional array back. I looked at the documentation for np.logical_and and friends but couldn't find an axis keyword on the logical_xyz operations and nothing else seemed to fit. > > Thanks, and best regards, > Hanno > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Mon Dec 18 09:58:58 2017 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Mon, 18 Dec 2017 15:58:58 +0100 Subject: [Numpy-discussion] Applying logical operations along an axis of a boolean array? In-Reply-To: References: Message-ID: <1513609138.22540.1.camel@sipsolutions.net> On Mon, 2017-12-18 at 12:02 +0100, hanno_lists at gmx.net wrote: > Hi, > ? > is it possible, to apply a logical operation, such as AND or OR along > a particular axis of a numpy array? > As mentioned, `np.any` and `np.all` work. However, what is more/also interesting to you is maybe that: `np.logical_or.reduce` works. All binary ufuncs (most elementwise functions such as addition, subtraction, multiplication, etc. support this `reduce` (and some other, please find out yourself ;)) methods. So that thing like `any`, `sum`, or `cumsum` are actually just thin wrappers around those. - Sebastian > ? > Let's say I have an (n,m) array and I want to AND along the first > axis, such that I get a (1,m) (or just (m,) dimensional array back. I > looked at the documentation for np.logical_and and friends but > couldn't find an axis keyword on the logical_xyz operations and > nothing else seemed to fit. > ? > Thanks, and best regards, > Hanno > ? > ? > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: This is a digitally signed message part URL: From charlesr.harris at gmail.com Tue Dec 19 15:01:40 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 19 Dec 2017 13:01:40 -0700 Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: <1524113059.6201087.1513592408374.JavaMail.zimbra@saao.ac.za> References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> <1524113059.6201087.1513592408374.JavaMail.zimbra@saao.ac.za> Message-ID: On Mon, Dec 18, 2017 at 3:20 AM, Hannes Breytenbach wrote: > OK, thanks for the link to the issue. I'm not using virtual environments > - I built python3.7 (and 2.7) with the `make altinstall` method. I managed > to get the numpy build working on 3.7 by removing the > `random/mtrand/mtrand.c` file so that it gets (re-)generated during the > build using the latest cython version. > > Thanks for the help! > Just to be sure, which Cython version is that? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From hannes at saao.ac.za Wed Dec 20 03:15:52 2017 From: hannes at saao.ac.za (Hannes Breytenbach) Date: Wed, 20 Dec 2017 10:15:52 +0200 (SAST) Subject: [Numpy-discussion] building numpy with python3.7 In-Reply-To: References: <1237981928.6063979.1513267195600.JavaMail.zimbra@saao.ac.za> <280195652.6103380.1513332661356.JavaMail.zimbra@saao.ac.za> <389896254.6106478.1513334545873.JavaMail.zimbra@saao.ac.za> <1524113059.6201087.1513592408374.JavaMail.zimbra@saao.ac.za> Message-ID: <1125979235.6298287.1513757752706.JavaMail.zimbra@saao.ac.za> Hi Chuck I'm using Cython 0.28a0. Hannes ----- Original Message ----- From: "Charles R Harris" To: "Discussion of Numerical Python" Sent: Tuesday, December 19, 2017 10:01:40 PM Subject: Re: [Numpy-discussion] building numpy with python3.7 On Mon, Dec 18, 2017 at 3:20 AM, Hannes Breytenbach < hannes at saao.ac.za > wrote: OK, thanks for the link to the issue. I'm not using virtual environments - I built python3.7 (and 2.7) with the `make altinstall` method. I managed to get the numpy build working on 3.7 by removing the `random/mtrand/mtrand.c` file so that it gets (re-)generated during the build using the latest cython version. Thanks for the help! Just to be sure, which Cython version is that? Chuck !DSPAM:5a39704114261779167816! _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion at python.org https://mail.python.org/mailman/listinfo/numpy-discussion !DSPAM:5a39704114261779167816! -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Thu Dec 21 20:38:12 2017 From: njs at pobox.com (Nathaniel Smith) Date: Thu, 21 Dec 2017 17:38:12 -0800 Subject: [Numpy-discussion] Another grant update, & numpy job ad is up! Message-ID: Hi all, Two exciting bits of news: 1) We just posted the announcement of a second grant to BIDS for NumPy, this time from the Sloan Foundation: https://bids.berkeley.edu/news/bids-receives-sloan-foundation-grant-contribute-numpy-development This is for $659,359 over two years, very similar to the previously-announced Moore Foundation grant. These two grants were originally written together as one, then split in half between the two foundations, then the schedules drifted apart... I'm excited to finally have this all sorted out so we can move ahead with the original plan! 2) We have successfully navigated UC Berkeley's administrative systems and posted an actual job opening, which you can apply to *right now*: https://jobsprod.is.berkeley.edu/psp/jobsprod/EMPLOYEE/HRMS/c/HRS_HRAM.HRS_CE.GBL?Page=HRS_CE_JOB_DTL&Action=A&JobOpeningId=24142&SiteId=1&PostingSeq=1 We're hoping to hire 2-3 people under that job description, so please tell your friends, relatives, enemies, whoever. This is a fully open search and we want our candidate pool to be as large and diverse (in all senses) as possible, so if you're on the fence about applying, why not give it a shot? Or if you want to know more, feel free to send any questions to me (njs at berkeley.edu) and/or St?fan van der Walt (stefanv at berkeley.edu) -- we'd love to chat. We'll also do another round of publicity/reminders in the new year after everyone's had a chance to recover from the holidays, but wanted to at least get this out now so those who are interested can start putting their application together... -n -- Nathaniel J. Smith -- https://vorpus.org From charlesr.harris at gmail.com Fri Dec 22 11:34:54 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 22 Dec 2017 09:34:54 -0700 Subject: [Numpy-discussion] Another grant update, & numpy job ad is up! In-Reply-To: References: Message-ID: On Thu, Dec 21, 2017 at 6:38 PM, Nathaniel Smith wrote: > Hi all, > > Two exciting bits of news: > > 1) We just posted the announcement of a second grant to BIDS for > NumPy, this time from the Sloan Foundation: > > https://bids.berkeley.edu/news/bids-receives-sloan- > foundation-grant-contribute-numpy-development > > This is for $659,359 over two years, very similar to the > previously-announced Moore Foundation grant. These two grants were > originally written together as one, then split in half between the two > foundations, then the schedules drifted apart... I'm excited to > finally have this all sorted out so we can move ahead with the > original plan! > Do the two grants concurrent, overlapping, or consecutive? Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From njs at pobox.com Fri Dec 22 14:57:36 2017 From: njs at pobox.com (Nathaniel Smith) Date: Fri, 22 Dec 2017 11:57:36 -0800 Subject: [Numpy-discussion] Another grant update, & numpy job ad is up! In-Reply-To: References: Message-ID: On Dec 22, 2017 8:35 AM, "Charles R Harris" wrote: On Thu, Dec 21, 2017 at 6:38 PM, Nathaniel Smith wrote: > Hi all, > > Two exciting bits of news: > > 1) We just posted the announcement of a second grant to BIDS for > NumPy, this time from the Sloan Foundation: > > https://bids.berkeley.edu/news/bids-receives-sloan-foundatio > n-grant-contribute-numpy-development > > This is for $659,359 over two years, very similar to the > previously-announced Moore Foundation grant. These two grants were > originally written together as one, then split in half between the two > foundations, then the schedules drifted apart... I'm excited to > finally have this all sorted out so we can move ahead with the > original plan! > Do the two grants concurrent, overlapping, or consecutive? Concurrent. We're in the process now of sorting out the dates so that they match, even though we got them at different times. -n -------------- next part -------------- An HTML attachment was scrubbed... URL: From garcia.marc at gmail.com Sun Dec 24 13:51:20 2017 From: garcia.marc at gmail.com (Marc Garcia) Date: Sun, 24 Dec 2017 18:51:20 +0000 Subject: [Numpy-discussion] Directives in numpy docstring convention Message-ID: Hi there, Is there a reason why the numpy docstring convention doesn't use the sphinx directives .. deprecated:: [1] and .. seealso:: [2]? I see that the numpy convention [3] uses the .. note:: directive for the deprecation notes, and for the "See also" it uses a section in this form: See also ------------- I guess those directives were added to the sphinx after the numpy docstring convention was created. And in this case, while is probably not worth to update numpy docstrings, I think they should be used for new projects that want to follow the numpy convention. Is that the case, or is there a reason why new projects shouldn't use them? Thanks! 1. http://www.sphinx-doc.org/en/stable/markup/para.html#directive-deprecated 2. http://www.sphinx-doc.org/en/stable/markup/para.html#directive-seealso 3. https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Sun Dec 24 14:16:30 2017 From: pav at iki.fi (Pauli Virtanen) Date: Sun, 24 Dec 2017 21:16:30 +0200 Subject: [Numpy-discussion] Directives in numpy docstring convention In-Reply-To: References: Message-ID: <1514142990.2290.2.camel@iki.fi> su, 2017-12-24 kello 18:51 +0000, Marc Garcia kirjoitti: [clip] > I guess those directives were added to the sphinx after the numpy > docstring convention was created. Just a comment on the historical point: the Numpy docstring standard existed before first Sphinx release. -- Pauli Virtanen From ralf.gommers at gmail.com Mon Dec 25 20:48:34 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 26 Dec 2017 14:48:34 +1300 Subject: [Numpy-discussion] Directives in numpy docstring convention In-Reply-To: References: Message-ID: On Mon, Dec 25, 2017 at 7:51 AM, Marc Garcia wrote: > Hi there, > > Is there a reason why the numpy docstring convention doesn't use the > sphinx directives .. deprecated:: [1] and .. seealso:: [2]? > > I see that the numpy convention [3] uses the .. note:: directive for the > deprecation notes, and for the "See also" it uses a section in this form: > > See also > ------------- > > I guess those directives were added to the sphinx after the numpy > docstring convention was created. And in this case, while is probably not > worth to update numpy docstrings, I think they should be used for new > projects that want to follow the numpy convention. Is that the case, or is > there a reason why new projects shouldn't use them? > If you prefer the way those Sphinx directives are rendered, then I don't see a problem with using them. I think our recommendation would be to keep to the NumPy docstring standard however; that'll get rendered like the docs for all core packages in the ecosystem. Ralf > > Thanks! > > 1. http://www.sphinx-doc.org/en/stable/markup/para.html# > directive-deprecated > 2. http://www.sphinx-doc.org/en/stable/markup/para.html#directive-seealso > 3. https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Dec 25 21:12:19 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 26 Dec 2017 15:12:19 +1300 Subject: [Numpy-discussion] GSoC'18 participation Message-ID: Hi all, It's the time of the year again where projects start preparing for GSoC. So I wanted to bring it up here. Last year I wrote: "in practice working on NumPy is just far too hard for most GSoC students. Previous years we've registered and generated ideas, but not gotten any students. We're also short on maintainer capacity. So I propose to not participate this year." I think that's still the case, so I won't be mentoring or organizing. In case anyone is interested to do one of those things, please speak up! Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Tue Dec 26 10:19:14 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Tue, 26 Dec 2017 08:19:14 -0700 Subject: [Numpy-discussion] GSoC'18 participation In-Reply-To: References: Message-ID: On Mon, Dec 25, 2017 at 7:12 PM, Ralf Gommers wrote: > Hi all, > > It's the time of the year again where projects start preparing for GSoC. > So I wanted to bring it up here. Last year I wrote: "in practice working on > NumPy is just far too hard for most GSoC students. Previous years we've > registered and generated ideas, but not gotten any students. We're also > short on maintainer capacity. So I propose to not participate this year." > > I think that's still the case, so I won't be mentoring or organizing. In > case anyone is interested to do one of those things, please speak up! > > Sounds realistic. I thought some of the ideas last year were doable, but no bites. Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From sebastian at sipsolutions.net Wed Dec 27 18:26:44 2017 From: sebastian at sipsolutions.net (Sebastian Berg) Date: Thu, 28 Dec 2017 00:26:44 +0100 Subject: [Numpy-discussion] GSoC'18 participation In-Reply-To: References: Message-ID: <1514417204.1215.0.camel@sipsolutions.net> On Tue, 2017-12-26 at 08:19 -0700, Charles R Harris wrote: > > > On Mon, Dec 25, 2017 at 7:12 PM, Ralf Gommers > wrote: > > Hi all, > > > > It's the time of the year again where projects start preparing for > > GSoC. So I wanted to bring it up here. Last year I wrote: "in > > practice working on NumPy is just far too hard for most GSoC > > students. Previous years we've registered and generated ideas, but > > not gotten any students. We're also short on maintainer capacity. > > So I propose to not participate this year." > > > > I think that's still the case, so I won't be mentoring or > > organizing. In case anyone is interested to do one of those things, > > please speak up! > > > > > > Sounds realistic. I thought some of the ideas last year were doable, > but no bites. > A bit unfortunate, but yeah, realistic. I do not have time to help out in any case. - Sebastian > Chuck > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: This is a digitally signed message part URL: From cimrman3 at ntc.zcu.cz Fri Dec 29 07:47:17 2017 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Fri, 29 Dec 2017 13:47:17 +0100 Subject: [Numpy-discussion] ANN: SfePy 2017.4 Message-ID: I am pleased to announce release 2017.4 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - basic support for penalty-based contacts - support for user-defined contexts in all solvers and preconditioners - new example: dispersion analysis of heterogeneous periodic materials For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Robert Cimrman Jan Heczko Lubos Kejzlar Jan Kopacka Vladimir Lukes Matyas Novak From me.vinob at gmail.com Sat Dec 30 05:37:09 2017 From: me.vinob at gmail.com (Vinodhini Balusamy) Date: Sat, 30 Dec 2017 21:37:09 +1100 Subject: [Numpy-discussion] array - dimension size of 1-D and 2-D examples Message-ID: Hi, I am new to bumpy and started working on basics. I need clarification for the below behaviour. case 1: >>> x12 = np.array([[1,2,3]]) >>> print(x12) [[1 2 3]] >>> x12.ndim 2 >>> Tried creating a 2-D array. also, >>> x12 = np.array([[1,2,3],[0,0,0]]) >>> print(x12) [[1 2 3] [0 0 0]] >>> x12.ndim 2 >>> Case 2: >>> >>> x12 = np.array([[1,2,3],[]]) >>> x12.ndim 1 >>> print(x12) [list([1, 2, 3]) list([])] >>> In case 2, I am trying to understand why it becomes 1 dimentional ?!?! Case 3: >>> >>> x12 = np.array([1,2,3]) >>> x12.ndim 1 >>> print(x12) [1 2 3] >>> This seems reasonable to me to be considered as 1 dimensional. Would like to understand case 2 a bit more to get to know if i am missing something. Will be much appreciated if someone to explain me a bit. Thanks in advance. Regards, Vinodhini B -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Sat Dec 30 07:36:32 2017 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Sat, 30 Dec 2017 13:36:32 +0100 Subject: [Numpy-discussion] array - dimension size of 1-D and 2-D examples In-Reply-To: References: Message-ID: On 30 Dec 2017, at 11:37 am, Vinodhini Balusamy wrote: > > Case 2: > >>> > >>> x12 = np.array([[1,2,3],[]]) > >>> x12.ndim > 1 > >>> print(x12) > [list([1, 2, 3]) list([])] > >>> > In case 2, I am trying to understand why it becomes 1 dimentional ?!?! > > > Case 3: > >>> > >>> x12 = np.array([1,2,3]) > >>> x12.ndim > 1 > >>> print(x12) > [1 2 3] > >>> > This seems reasonable to me to be considered as 1 dimensional. > > Would like to understand case 2 a bit more to get to know if i am missing something. > Will be much appreciated if someone to explain me a bit. > Welcome to the crowd! You cannot create a regular 2-dimensional integer array from one row of length 3 and a second one of length 0. Thus np.array chooses the next most basic type of array it can fit your input data in - you will notice in case 2 the array actually has two elements of type ?list?, and you can verify that In [1]: x12 = np.array([[1,2,3],[]]) In [2]: x12.dtype Out[2]: dtype('O') In [3]: x12.shape Out[3]: (2,) i.e. it has created an array of dtype ?object?, which is probably not what you expected (and nothing you could perform standard arithmetic operations on: In [4]: x12+1 TypeError: can only concatenate list (not "int") to list HTH Derek From me.vinob at gmail.com Sat Dec 30 11:38:05 2017 From: me.vinob at gmail.com (Vinodhini Balusamy) Date: Sun, 31 Dec 2017 03:38:05 +1100 Subject: [Numpy-discussion] array - dimension size of 1-D and 2-D examples In-Reply-To: References: Message-ID: <449981B3-DE05-4595-9A13-C129CFFBC51F@gmail.com> Thanks Derek for quick clarification. Just one more question from the details you have provided which from my understanding strongly seems to be Design [DEREK] You cannot create a regular 2-dimensional integer array from one row of length 3 > > and a second one of length 0. Thus np.array chooses the next most basic type of > array it can fit your input data in Which is the case, only if an second one of length 0 is given. What about the case 1 : >>> x12 = np.array([[1,2,3]]) >>> x12 array([[1, 2, 3]]) >>> print(x12) [[1 2 3]] >>> x12.ndim 2 >>> >>> This seems to take 2 dimension. I presumed the above case and the case where length 0 is provided to be treated same(I mean same behaviour). Correct me if I am wrong. Also, could u please point out any documentation to understand the logic behind creating elements of type list in case 2(with second grid of length 0) ? If possible. I am curious to understand. Kind Rgds, Vinodhini B > On 30 Dec 2017, at 11:36 PM, Derek Homeier wrote: > > On 30 Dec 2017, at 11:37 am, Vinodhini Balusamy wrote: >> >> Case 2: >>>>> >>>>> x12 = np.array([[1,2,3],[]]) >>>>> x12.ndim >> 1 >>>>> print(x12) >> [list([1, 2, 3]) list([])] >>>>> >> In case 2, I am trying to understand why it becomes 1 dimentional ?!?! >> >> >> Case 3: >>>>> >>>>> x12 = np.array([1,2,3]) >>>>> x12.ndim >> 1 >>>>> print(x12) >> [1 2 3] >>>>> >> This seems reasonable to me to be considered as 1 dimensional. >> >> Would like to understand case 2 a bit more to get to know if i am missing something. >> Will be much appreciated if someone to explain me a bit. >> > Welcome to the crowd! > You cannot create a regular 2-dimensional integer array from one row of length 3 > and a second one of length 0. Thus np.array chooses the next most basic type of > array it can fit your input data in - you will notice in case 2 the array actually has two > elements of type ?list?, and you can verify that > > In [1]: x12 = np.array([[1,2,3],[]]) > In [2]: x12.dtype > Out[2]: dtype('O') > In [3]: x12.shape > Out[3]: (2,) > > i.e. it has created an array of dtype ?object?, which is probably not what you expected > (and nothing you could perform standard arithmetic operations on: > > In [4]: x12+1 > TypeError: can only concatenate list (not "int") to list > > HTH > Derek > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion at python.org > https://mail.python.org/mailman/listinfo/numpy-discussion -------------- next part -------------- An HTML attachment was scrubbed... URL: From derek at astro.physik.uni-goettingen.de Sat Dec 30 18:11:48 2017 From: derek at astro.physik.uni-goettingen.de (Derek Homeier) Date: Sun, 31 Dec 2017 00:11:48 +0100 Subject: [Numpy-discussion] array - dimension size of 1-D and 2-D examples In-Reply-To: <449981B3-DE05-4595-9A13-C129CFFBC51F@gmail.com> References: <449981B3-DE05-4595-9A13-C129CFFBC51F@gmail.com> Message-ID: On 30 Dec 2017, at 5:38 pm, Vinodhini Balusamy wrote: > > Just one more question from the details you have provided which from my understanding strongly seems to be Design > [DEREK] You cannot create a regular 2-dimensional integer array from one row of length 3 >> and a second one of length 0. Thus np.array chooses the next most basic type of >> array it can fit your input data in > Indeed, the general philosophy is to preserve the structure and type of your input data as far as possible, i.e. a list is turned into a 1d-array, a list of lists (or tuples etc?) into a 2d-array,_ if_ the sequences are of equal length (even if length 1). As long as there is an unambiguous way to convert the data into an array (see below). > Which is the case, only if an second one of length 0 is given. > What about the case 1 : > >>> x12 = np.array([[1,2,3]]) > >>> x12 > array([[1, 2, 3]]) > >>> print(x12) > [[1 2 3]] > >>> x12.ndim > 2 > >>> > >>> > This seems to take 2 dimension. Yes, structurally this is equivalent to your second example > also, >>> x12 = np.array([[1,2,3],[0,0,0]]) >>> print(x12) [[1 2 3] [0 0 0]] >>> x12.ndim 2 > I presumed the above case and the case where length 0 is provided to be treated same(I mean same behaviour). > Correct me if I am wrong. > In this case there is no unambiguous way to construct the array - you would need a shape (2, 3) array to store the two lists with 3 elements in the first list. Obviously x12[0] would be np.array([1,2,3]), but what should be the value of x12[1], if the second list is empty - it could be zeros, or repeating x12[0], or simply undefined. np.array([1, 2, 3], [4]]) would be even less clearly defined. These cases where there is no obvious ?right? way to create the array have usually been discussed at some length, but I don?t know if this is fully documented in some place. For the essentials, see https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html note also the upcasting rules if you have e.g. a mix of integers and reals or complex numbers, and also how to control shape or data type explicitly with the respective keywords. Derek