From bruno at ft.unicamp.br Mon Jul 1 08:55:01 2013 From: bruno at ft.unicamp.br (Bruno Luciano Amadio Caires) Date: Mon, 01 Jul 2013 09:55:01 -0300 Subject: [SciPy-Dev] Compiling scipy on AIX 6.1 In-Reply-To: References: <134a6cef654b04dd1739bda90a76f554@ft.unicamp.br> Message-ID: <90151d9066835d705fff73fa83dfbc68@ft.unicamp.br> Ok, I'll try Tks Ralf On Sun, 30 Jun 2013 21:48:57 +0200, Ralf Gommers wrote: > On Thu, Jun 27, 2013 at 9:58 PM, Bruno Luciano Amadio Caires wrote: > Hi, > I am trying install scipy on AIX 6.1 power7, but I'm getting errors. > I > am using XL compiler, and scipy-0.12.0/scipy-0.8.0 > I have numpy-1.7.0 working that was compiling here with XL. > Has someone a HOWTO? > > Did you see this: https://github.com/scipy/scipy/issues/1825?source=c > [2] ? > > Ralf > > > > Links: > ------ > [1] mailto:bruno at ft.unicamp.br > [2] https://github.com/scipy/scipy/issues/1825?source=c -- Bruno L. Amadio Caires Inform?tica - FT Universidade Estadual de Campinas Fone:(19)2113-3470 From blake.a.griffith at gmail.com Mon Jul 1 23:40:09 2013 From: blake.a.griffith at gmail.com (Blake Griffith) Date: Mon, 1 Jul 2013 22:40:09 -0500 Subject: [SciPy-Dev] GSoC -- numpy interactions with sparse matrices Message-ID: Hello scipy, Today marks the first day of the next phase of my GSoC proposal, numpy interactions and overriding ufuncs for sparse matrices. I've been thinking about how I will handle the ufunc interactions. I'm still considering how to implement a ufunc override mechanism. However I think the existing functionality in numpy, (__array_prepare__, __array_finalize__, etc) would be enough for a few cases: * unary element-wise ufuncs. Because these could be applied to the .data attribute using array prepare. * binary element-wise ufuncs that correspond to python operators (__add__, __gt__, __mul__, etc.) because there is already functionality in place to override these. This of course leave out: logaddexp logaddexp2 ones_like arctan2 hypot maximum minimum copysign nextafter ldexp (However I think it would be possible to override these using the existing functionality for overriding python overrides but this would be a bit hackish, and numpy might not like the changes.) None of these left over ufuncs seem to be the most widely used (I could be wrong). So going about this without adding an ufunc override attribute would still add a lot of functionality. So I think the best course would be to explore adding a override attribute but if it is extremely difficult I can resort to this. comment, suggestions, criticisms? -------------- next part -------------- An HTML attachment was scrubbed... URL: From saullogiovani at gmail.com Tue Jul 2 16:14:14 2013 From: saullogiovani at gmail.com (Saullo Castro) Date: Tue, 2 Jul 2013 22:14:14 +0200 Subject: [SciPy-Dev] Quadrature of a vector-valued function, implementation changing the quadpack package Message-ID: Dear all, I am implementing a vectorized version of: `dagse.f` --> `dagsev.f` and `dqk21.f` --> `dqk21v.f` Which handles the vectorized integration in "low level" inside the Fortran routines. I would like to discuss with the SciPy dev group if there is any interest to make this feature part of SciPy. Also, I need some advice about how to wrap this with quadpack. Currently I am changing: `_quadpackmodule.c` `_quadpack.h` to make `dagsev.f` work similarly to `dagse.f`. Thank you! Saullo -------------- next part -------------- An HTML attachment was scrubbed... URL: From pav at iki.fi Tue Jul 2 16:15:18 2013 From: pav at iki.fi (Pauli Virtanen) Date: Tue, 02 Jul 2013 23:15:18 +0300 Subject: [SciPy-Dev] GSoC -- numpy interactions with sparse matrices In-Reply-To: References: Message-ID: (cc'd Numpy list --- the discussion is about making np.multiply(A, B) et al. do sensible rather than non-sensible things if A or B is a sparse matrix) Hi Blake, 02.07.2013 06:40, Blake Griffith kirjoitti: > Today marks the first day of the next phase of my GSoC proposal, numpy > interactions and overriding ufuncs for sparse matrices. > > I've been thinking about how I will handle the ufunc interactions. I'm > still considering how to implement a ufunc override mechanism. However I > think the existing functionality in numpy, (__array_prepare__, > __array_finalize__, etc) would be enough for a few cases: I think using __array_prepare__ and __array_wrap__ et al. for this will run into difficulties: - array_prepare requires that the strides and dimensions stay the same as in the "original array". - Sparse matrices are not subclasses of ndarrays, so I think they are considered 0-dim (or NxN?) object arrays by the ufuncs. This then wreaks all sorts of havoc. The issue is sort of that the whole wrapping mechanism is written under the assumption that the data is still contained in an ndarray stored in memory in a strided layout with the correct shape etc., which is not true for sparse matrices. You can perhaps check the above for yourself, in case I'm mistaken. However, as far as I know, the facilities in Numpy don't currently allow getting sensible behavior out. *** One option is to try to make array_prepare to allow returning arbitrary arrays. I think this approach has a number of problems: - It leads to an unnatural way to write the code --- you are trying to trick the chosen ufunc to do what you want, rather than coding up straightforwardly the operation that should give the result. - It seems somewhat a nasty job to do: the ufunc loop selection (which IIRC can do stuff like optimizing iteration order etc) runs *before* array_prepare, so you cannot change the memory layout any more... So I think extending this mechanism does not sound very promising... *** The simplest approach seems to me to add a way to completely override the ufuncs immediately on call. The logic could be of the style: On entry to ufunc: - if an input has __array_priority__, check if it also has __ufunc_override__ - if some of the inputs have __ufunc_override__, choose the one with the maximum __array_priority__ - call it, passing it the arguments of the ufuncs, and any keywords (in a dict?) - if it returns NotImplemented, try the next-highest priority one (or just give up) It's perhaps easiest to prototype the solution first in pure Python, following Nathaniel's example: https://github.com/njsmith/numpyNEP/blob/master/numpyNEP.py I.e., monkeypatch all ufuncs in Numpy, and insert them also to the numerical operations using set_numeric_ops. This should allow you to try out different solutions before diving deeply into the Numpy source code. Another things to consider: does this add significantly to the runtime cost? I think not, because the ufunc machinery makes many checks for __array_priority__, so adding one more attribute check will probably not significantly hurt performance. But I think this needs to be measured at the end. So I would suggest the following starting procedure: re-check if the array_prepare/array_wrap machinery can be used (I think not), and then we need to spec for the ufunc override. Here, prototyping it out will help, so I suggest you take a simple scheme how the override should work (eg. as above), and then try to use it in scipy.sparse with a pure-Python prototype monkeypatch implementation. This then hopefully shows how workable the scheme is, and whether we need to consider some other option. The drawback here is that this would add yet another knob for the ufunc vs. custom classes interaction for the users to fiddle with. On the other, it seems that currently there is no way to do it. *** The option C would be to add __array__ to sparse matrices, which would make np.multiply and other routines just cast the sparse matrices to dense ones. Or perhaps __array__ should raise an error? This can also be considered if the ufunc override ends up being too radical. -- Pauli Virtanen From pav at iki.fi Tue Jul 2 16:21:55 2013 From: pav at iki.fi (Pauli Virtanen) Date: Tue, 02 Jul 2013 23:21:55 +0300 Subject: [SciPy-Dev] Quadrature of a vector-valued function, implementation changing the quadpack package In-Reply-To: References: Message-ID: Hi, 02.07.2013 23:14, Saullo Castro kirjoitti: > I am implementing a vectorized version of: > > `dagse.f` --> `dagsev.f` > > and > > `dqk21.f` --> `dqk21v.f` > > Which handles the vectorized integration in "low level" inside the > Fortran routines. > > I would like to discuss with the SciPy dev group if there is any > interest to make this feature part of SciPy. This feature has been requested several times by users, and would be appreciated. > Also, I need some advice about how to wrap this with quadpack. Currently > I am changing: > > `_quadpackmodule.c` > `_quadpack.h` > > to make `dagsev.f` work similarly to `dagse.f`. This may be the path of least resistance. However, it should be noted that the _*.[ch] are rather old (pre-2001?) and not written in an optimal way --- it would be best to get rid of them if possible, or at least to not write more code in the same style. Nowadays, one of the easier ways to wrap Fortran code is to use `f2py`, see the documentation here: http://cens.ioc.ee/projects/f2py2e/ There are a number of examples of f2py usage in the Scipy code base, look for *.pyf files. Several of them also have examples on how to deal with callback routines. -- Pauli Virtanen From ben.root at ou.edu Tue Jul 2 16:25:44 2013 From: ben.root at ou.edu (Benjamin Root) Date: Tue, 2 Jul 2013 16:25:44 -0400 Subject: [SciPy-Dev] GSoC -- numpy interactions with sparse matrices In-Reply-To: References: Message-ID: Comments below: On Mon, Jul 1, 2013 at 11:40 PM, Blake Griffith wrote: > > This of course leave out: > logaddexp > logaddexp2 > ones_like > arctan2 > hypot > maximum > minimum > copysign > nextafter > ldexp > > (However I think it would be possible to override these using the existing > functionality for overriding python overrides but this would be a bit > hackish, and numpy might not like the changes.) > > None of these left over ufuncs seem to be the most widely used (I could be > wrong). So going about this without adding an ufunc override attribute > would still add a lot of functionality. > > And that would be a bad assumption. While I don't use sparse matrices, I do use some of those functions, particularly hypot, arctan2, maximum, and minimum (why is ones_like() a ufunc?). If I had sparse matrices, I suspect I certainly would be using these functions just like I do for regular numpy arrays. Cheers! Ben Root -------------- next part -------------- An HTML attachment was scrubbed... URL: From suryak at ieee.org Thu Jul 4 04:30:42 2013 From: suryak at ieee.org (Surya Kasturi) Date: Thu, 4 Jul 2013 10:30:42 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC Message-ID: Hi all, its just an idea and appreciate your comments. I am thinking about migrating text markup that spc currently supports from reStructured Text to Markdown.. Reasons: 1. reST has some security issues - Someone can put some raw HTML in it and thus creating vulnerabilities.. Better explained at https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text 2. currently reST to HTML conversion is done on the server side! It could be done on client side enabling better text editing and previewing.. Markdown seems to be a decent option since there are some opensource tools (by Stackoverflow) and we can start integrating them with our site http://code.google.com/p/pagedown/wiki/PageDown (link previously referred by Sebastian during discussion - thanks!) Eventhough we might not move to Markdown, there seems to be some tools django offers for handling text markup https://docs.djangoproject.com/en/1.4/ref/contrib/markup/ And we dont have to use convert_rest_to_html() at all!! Probably, we might need to convert everytime the user loads the page (have to look into the exact docs.. don't know) but it offers us another feature Rightnow, the image uploads embedded in description as hardcoded during conversion i.e., the URL looks something like " scipy-central.org/media/blah-blah/122.jpg" If for some reasons we change the domain name or want to work on localhost, these images don't work! https://github.com/scipy/SciPyCentral/issues/123 It would be nice to have your views.. its not part of GSoC but probably, we can put it in..or if I got some extra time, I can do without mentioning in GSoC proposal. Thanks Surya -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at hilboll.de Thu Jul 4 04:38:32 2013 From: lists at hilboll.de (Andreas Hilboll) Date: Thu, 04 Jul 2013 10:38:32 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: References: Message-ID: <51D53488.3060103@hilboll.de> On 04.07.2013 10:30, Surya Kasturi wrote: > Hi all, its just an idea and appreciate your comments. > > I am thinking about migrating text markup that spc currently supports > from reStructured Text to Markdown.. > > Reasons: > > 1. reST has some security issues - Someone can put some raw HTML in it > and thus creating vulnerabilities.. Better explained at > > https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text > > 2. currently reST to HTML conversion is done on the server side! It > could be done on client side enabling better text editing and previewing.. Using markdown might also make the notebook export easier, right? -- Andreas. From suryak at ieee.org Thu Jul 4 04:45:47 2013 From: suryak at ieee.org (Surya Kasturi) Date: Thu, 4 Jul 2013 10:45:47 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: <51D53488.3060103@hilboll.de> References: <51D53488.3060103@hilboll.de> Message-ID: > > Using markdown might also make the notebook export easier, right? > > Yeah.. I didn't know that before, just went into IPython Notebook and it supports Markdown syntax.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Thu Jul 4 05:23:25 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Thu, 4 Jul 2013 11:23:25 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: References: Message-ID: <89D4B9FC-3100-4DA0-9CC1-CF7BA573E004@gmail.com> Hi all, Le 4 juil. 2013 ? 10:30, Surya Kasturi a ?crit : > 1. reST has some security issues - Someone can put some raw HTML in it and thus creating vulnerabilities.. Better explained at > > https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text But markdown is a superset of HTML? so you implicitly allow to write html if you switch to MD? but you can sanitize.. (just sayin') > 2. currently reST to HTML conversion is done on the server side! It could be done on client side enabling better text editing and previewing.. I hope you cache the redered pages? > Markdown seems to be a decent option since there are some opensource tools (by Stackoverflow) and we can start integrating them with our site > > http://code.google.com/p/pagedown/wiki/PageDown > (link previously referred by Sebastian during discussion - thanks!) IPython just switched from PageDown to Marked? it support GFM, and user are really used to it. Le 4 juil. 2013 ? 10:38, Andreas Hilboll a ?crit : > On 04.07.2013 10:30, Surya Kasturi wrote: > Using markdown might also make the notebook export easier, right? That's funny, a few weeks ago we were saying that markdown was too restrictive and that it might be worth fixing rst problem than trying to extend markdown and switching back to rst one day? (don't worry, not soon), discussion should have been recored on youtube. -- Matthias -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at hilboll.de Thu Jul 4 05:29:50 2013 From: lists at hilboll.de (Andreas Hilboll) Date: Thu, 04 Jul 2013 11:29:50 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: <89D4B9FC-3100-4DA0-9CC1-CF7BA573E004@gmail.com> References: <89D4B9FC-3100-4DA0-9CC1-CF7BA573E004@gmail.com> Message-ID: <51D5408E.1030700@hilboll.de> On 04.07.2013 11:23, Matthias BUSSONNIER wrote: > Hi all, > > Le 4 juil. 2013 ? 10:30, Surya Kasturi a ?crit : > >> 1. reST has some security issues - Someone can put some raw HTML in it >> and thus creating vulnerabilities.. Better explained at >> >> https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text > > But markdown is a superset of HTML? so you implicitly allow to write > html if you switch to MD? but you can sanitize.. (just sayin') > >> 2. currently reST to HTML conversion is done on the server side! It >> could be done on client side enabling better text editing and >> previewing.. > > I hope you cache the redered pages? > >> Markdown seems to be a decent option since there are some opensource >> tools (by Stackoverflow) and we can start integrating them with our site >> >> http://code.google.com/p/pagedown/wiki/PageDown >> (link previously referred by Sebastian during discussion - thanks!) > > IPython just switched from PageDown to Marked? it support GFM, and user > are really used to it. > > > Le 4 juil. 2013 ? 10:38, Andreas Hilboll a ?crit : > >> On 04.07.2013 10:30, Surya Kasturi wrote: > >> Using markdown might also make the notebook export easier, right? > > That's funny, a few weeks ago we were saying that markdown was too > restrictive and that > it might be worth fixing rst problem than trying to extend markdown and > switching back to > rst one day? (don't worry, not soon), discussion should have been > recored on youtube. Well, I personally do prefer reST. So if the reST problems were fixed and the notebooks would support reST, then reST would be my favourite SPyCe solution ... But until then, using MD in SPyCe would make the notebook export easier. -- Andreas. From josef.pktd at gmail.com Thu Jul 4 10:43:31 2013 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Thu, 4 Jul 2013 10:43:31 -0400 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: <51D5408E.1030700@hilboll.de> References: <89D4B9FC-3100-4DA0-9CC1-CF7BA573E004@gmail.com> <51D5408E.1030700@hilboll.de> Message-ID: On Thu, Jul 4, 2013 at 5:29 AM, Andreas Hilboll wrote: > On 04.07.2013 11:23, Matthias BUSSONNIER wrote: >> Hi all, >> >> Le 4 juil. 2013 ? 10:30, Surya Kasturi a ?crit : >> >>> 1. reST has some security issues - Someone can put some raw HTML in it >>> and thus creating vulnerabilities.. Better explained at >>> >>> https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text >> >> But markdown is a superset of HTML? so you implicitly allow to write >> html if you switch to MD? but you can sanitize.. (just sayin') >> >>> 2. currently reST to HTML conversion is done on the server side! It >>> could be done on client side enabling better text editing and >>> previewing.. >> >> I hope you cache the redered pages? >> >>> Markdown seems to be a decent option since there are some opensource >>> tools (by Stackoverflow) and we can start integrating them with our site >>> >>> http://code.google.com/p/pagedown/wiki/PageDown >>> (link previously referred by Sebastian during discussion - thanks!) >> >> IPython just switched from PageDown to Marked? it support GFM, and user >> are really used to it. >> >> >> Le 4 juil. 2013 ? 10:38, Andreas Hilboll a ?crit : >> >>> On 04.07.2013 10:30, Surya Kasturi wrote: >> >>> Using markdown might also make the notebook export easier, right? >> >> That's funny, a few weeks ago we were saying that markdown was too >> restrictive and that >> it might be worth fixing rst problem than trying to extend markdown and >> switching back to >> rst one day? (don't worry, not soon), discussion should have been >> recored on youtube. > > Well, I personally do prefer reST. So if the reST problems were fixed > and the notebooks would support reST, then reST would be my favourite > SPyCe solution ... But until then, using MD in SPyCe would make the > notebook export easier. Similar here, I'm writing almost only reST these days. github allows reSt in the Wiki (without extensions and maybe reduced directives), so it should be possible to solve any security issues. (github offers a choice of markup) Josef > > -- Andreas. > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev From Thomas.Haslwanter at fh-linz.at Fri Jul 5 03:45:33 2013 From: Thomas.Haslwanter at fh-linz.at (Haslwanter Thomas) Date: Fri, 5 Jul 2013 09:45:33 +0200 Subject: [SciPy-Dev] Missing statistical Tests for scipy.stats Message-ID: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> Hi, in preparing a statistics lecture using Python (http://work.thaslwanter.at/Stats/html/ ), I realized that two important statistical tests are currently not available in scipy: ? McNemar Test (http://en.wikipedia.org/wiki/McNemar%27s_test ) ? Cochran's Q-Test (http://en.wikipedia.org/wiki/Cochran%27s_Q_test) I have found the equations for the McNemar test, and implemented a version in Python. Would there be an interest in incorporating this module in scipy.stats? If I'd get the formulas for Cochran's Q-test, I would also be keen on implementing that one. What do you think? thomas --- Prof. (FH) PD Dr. Thomas Haslwanter School of Applied Health and Social Sciences University of Applied Sciences Upper Austria FH O? Studienbetriebs GmbH Garnisonstra?e 21 4020 Linz/Austria Tel.: +43 (0)5 0804 -52170 Fax: +43 (0)5 0804 -52171 E-Mail: Thomas.Haslwanter at fh-linz.at Web: me-research.fh-linz.at or work.thaslwanter.at -------------- next part -------------- An HTML attachment was scrubbed... URL: From Thomas.Haslwanter at fh-linz.at Fri Jul 5 03:47:37 2013 From: Thomas.Haslwanter at fh-linz.at (Haslwanter Thomas) Date: Fri, 5 Jul 2013 09:47:37 +0200 Subject: [SciPy-Dev] Missing statistical Tests for scipy.stats In-Reply-To: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> References: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> Message-ID: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB014@LNZEXCHANGE001.linz.fhooe.at> Small update: I have the equations for Cochran's test, and already implemented a rudimentary version. -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Fri Jul 5 07:47:21 2013 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Fri, 5 Jul 2013 07:47:21 -0400 Subject: [SciPy-Dev] Missing statistical Tests for scipy.stats In-Reply-To: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> References: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> Message-ID: On Fri, Jul 5, 2013 at 3:45 AM, Haslwanter Thomas wrote: > Hi, > > > > in preparing a statistics lecture using Python > (http://work.thaslwanter.at/Stats/html/ ), I realized that two important > statistical tests are currently not available in scipy: > > > > ? McNemar Test (http://en.wikipedia.org/wiki/McNemar%27s_test ) > > ? Cochran?s Q-Test (http://en.wikipedia.org/wiki/Cochran%27s_Q_test) > http://statsmodels.sourceforge.net/devel/stats.html#non-parametric-tests I didn't get to those in my last round of statsmodels.stats cleanups. Josef > > > I have found the equations for the McNemar test, and implemented a version > in Python. Would there be an interest in incorporating this module in > scipy.stats? > > > > If I?d get the formulas for Cochran?s Q-test, I would also be keen on > implementing that one. > > > > What do you think? > > > > thomas > > > > --- > Prof. (FH) PD Dr. Thomas Haslwanter > School of Applied Health and Social Sciences > > University of Applied Sciences Upper Austria > FH O? Studienbetriebs GmbH > Garnisonstra?e 21 > 4020 Linz/Austria > Tel.: +43 (0)5 0804 -52170 > Fax: +43 (0)5 0804 -52171 > E-Mail: Thomas.Haslwanter at fh-linz.at > Web: me-research.fh-linz.at > or work.thaslwanter.at > > > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From suryak at ieee.org Fri Jul 5 09:55:08 2013 From: suryak at ieee.org (Surya Kasturi) Date: Fri, 5 Jul 2013 15:55:08 +0200 Subject: [SciPy-Dev] Migrating to Markdown Syntax from reStructured Text in SPC In-Reply-To: References: <89D4B9FC-3100-4DA0-9CC1-CF7BA573E004@gmail.com> <51D5408E.1030700@hilboll.de> Message-ID: On Thu, Jul 4, 2013 at 4:43 PM, wrote: > On Thu, Jul 4, 2013 at 5:29 AM, Andreas Hilboll wrote: > > On 04.07.2013 11:23, Matthias BUSSONNIER wrote: > >> Hi all, > >> > >> Le 4 juil. 2013 ? 10:30, Surya Kasturi a ?crit : > >> > >>> 1. reST has some security issues - Someone can put some raw HTML in it > >>> and thus creating vulnerabilities.. Better explained at > >>> > >>> > https://docs.djangoproject.com/en/1.4/ref/contrib/markup/#restructured-text > >> > >> But markdown is a superset of HTML? so you implicitly allow to write > >> html if you switch to MD? but you can sanitize.. (just sayin') > >> > >>> 2. currently reST to HTML conversion is done on the server side! It > >>> could be done on client side enabling better text editing and > >>> previewing.. > >> > >> I hope you cache the redered pages? > >> > >>> Markdown seems to be a decent option since there are some opensource > >>> tools (by Stackoverflow) and we can start integrating them with our > site > >>> > >>> http://code.google.com/p/pagedown/wiki/PageDown > >>> (link previously referred by Sebastian during discussion - thanks!) > >> > >> IPython just switched from PageDown to Marked? it support GFM, and user > >> are really used to it. > >> > >> > >> Le 4 juil. 2013 ? 10:38, Andreas Hilboll a ?crit : > >> > >>> On 04.07.2013 10:30, Surya Kasturi wrote: > >> > >>> Using markdown might also make the notebook export easier, right? > >> > >> That's funny, a few weeks ago we were saying that markdown was too > >> restrictive and that > >> it might be worth fixing rst problem than trying to extend markdown and > >> switching back to > >> rst one day? (don't worry, not soon), discussion should have been > >> recored on youtube. > > > > Well, I personally do prefer reST. So if the reST problems were fixed > > and the notebooks would support reST, then reST would be my favourite > > SPyCe solution ... But until then, using MD in SPyCe would make the > > notebook export easier. > > Similar here, I'm writing almost only reST these days. > github allows reSt in the Wiki (without extensions and maybe reduced > directives), so it should be possible to solve any security issues. > (github offers a choice of markup) > > Josef > > > > > -- Andreas. > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > Okay.. looks like there are lots of ReST fans here :) No problem, I will try to find a nice JavaScript Editor for ReSt and integrate it. And one more thing -- do we need photos' attachment in comments? -------------- next part -------------- An HTML attachment was scrubbed... URL: From warren.weckesser at gmail.com Fri Jul 5 11:50:00 2013 From: warren.weckesser at gmail.com (Warren Weckesser) Date: Fri, 5 Jul 2013 11:50:00 -0400 Subject: [SciPy-Dev] Missing statistical Tests for scipy.stats In-Reply-To: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> References: <1CFD8CBC30E0454B9DFAADEB36AD17396D0F3FB013@LNZEXCHANGE001.linz.fhooe.at> Message-ID: On Fri, Jul 5, 2013 at 3:45 AM, Haslwanter Thomas < Thomas.Haslwanter at fh-linz.at> wrote: > Hi,**** > > ** ** > > in preparing a statistics lecture using Python ( > http://work.thaslwanter.at/Stats/html/ ), I realized that two important > statistical tests are currently not available in scipy:**** > > ** ** > > **? **McNemar Test (http://en.wikipedia.org/wiki/McNemar%27s_test) > **** > > **? **Cochran?s Q-Test ( > http://en.wikipedia.org/wiki/Cochran%27s_Q_test)**** > > ** ** > > I have found the equations for the McNemar test, and implemented a version > in Python. Would there be an interest in incorporating this module in > scipy.stats?**** > > ** ** > > If I?d get the formulas for Cochran?s Q-test, I would also be keen on > implementing that one.**** > > ** ** > > What do you think?**** > > ** > Sounds good--please submit a pull request! Warren > ** > > thomas**** > > ** ** > > --- > Prof. (FH) PD Dr. Thomas Haslwanter > School of Applied Health and Social Sciences **** > > *University of Applied Sciences* *Upper Austria* > *FH O? Studienbetriebs GmbH* > Garnisonstra?e 21 > 4020 Linz/Austria > Tel.: +43 (0)5 0804 -52170 > Fax: +43 (0)5 0804 -52171 > E-Mail: Thomas.Haslwanter at fh-linz.at > Web: me-research.fh-linz.at > or work.thaslwanter.at **** > > ** ** > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bruno at ft.unicamp.br Fri Jul 5 15:25:02 2013 From: bruno at ft.unicamp.br (Bruno Luciano Amadio Caires) Date: Fri, 05 Jul 2013 16:25:02 -0300 Subject: [SciPy-Dev] Compiling scipy on AIX 6.1 In-Reply-To: <90151d9066835d705fff73fa83dfbc68@ft.unicamp.br> References: <134a6cef654b04dd1739bda90a76f554@ft.unicamp.br> <90151d9066835d705fff73fa83dfbc68@ft.unicamp.br> Message-ID: <1b626cc3614bd71125f0d4857a86b093@ft.unicamp.br> Hi I have trying install following https://github.com/scipy/scipy/issues/1825?source=c. I got this error "" /opt/freeware/lib/python2.6/config/ld_so_aix /usr/bin/xlf95 -bI:/opt/freeware/lib/python2.6/config/python.exp -L/usr/lib -lessl -L/home/bruno/local/lib -lblas -llapack build/temp.aix-6.1-2.6/build/src.aix-6.1-2.6/build/src.aix-6.1-2.6/scipy/lib/lapack/flapackmodule.o build/temp.aix-6.1-2.6/build/src.aix-6.1-2.6/fortranobject.o -L/home/bruno/local/lib -Lbuild/temp.aix-6.1-2.6 -llapack -lblas -o build/lib.aix-6.1-2.6/scipy/lib/lapack/flapack.so ld: 0711-317 ERROR: Undefined symbol: sgbsv_ ld: 0711-317 ERROR: Undefined symbol: dgbsv_ ... "" I saw here http://projects.scipy.org/scipy/attachment/ticket/1299/README.txt how to solve, and I did something like this "" ld -b64 -r liblapack.a -o liblapack-renamed.a -brename:sgbsv,sgbsv_ -brename:dgbsv,dgbsv_ -brename:cgbsv,cgbsv_ -brename:zgbsv,zgbsv_ ...... "" Ok, I create a new lib "liblapack-renamed.a". What can I do now? Re-run the command? How? tks On Mon, 01 Jul 2013 09:55:01 -0300, Bruno Luciano Amadio Caires wrote: > Ok, I'll try > > Tks Ralf > > On Sun, 30 Jun 2013 21:48:57 +0200, Ralf Gommers > wrote: >> On Thu, Jun 27, 2013 at 9:58 PM, Bruno Luciano Amadio Caires wrote: >> Hi, >> I am trying install scipy on AIX 6.1 power7, but I'm getting errors. >> I >> am using XL compiler, and scipy-0.12.0/scipy-0.8.0 >> I have numpy-1.7.0 working that was compiling here with XL. >> Has someone a HOWTO? >> >> Did you see this: https://github.com/scipy/scipy/issues/1825?source=c >> [2] ? >> >> Ralf >> >> >> >> Links: >> ------ >> [1] mailto:bruno at ft.unicamp.br >> [2] https://github.com/scipy/scipy/issues/1825?source=c -- Bruno L. Amadio Caires Inform?tica - FT Universidade Estadual de Campinas Fone:(19)2113-3470 From anubhab91 at gmail.com Sun Jul 7 11:53:11 2013 From: anubhab91 at gmail.com (Anubhab Baksi) Date: Sun, 7 Jul 2013 21:23:11 +0530 Subject: [SciPy-Dev] AttributeError when editing source Message-ID: Hi, I was trying to write an additional function in misc/common. But, when I tried to run that, I got an AttributeError, stating module object has no such attribute. Can anybody, please, tell me why this is happening and how can I fix it? Thank you all. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Jul 7 12:00:54 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 7 Jul 2013 18:00:54 +0200 Subject: [SciPy-Dev] AttributeError when editing source In-Reply-To: References: Message-ID: On Sun, Jul 7, 2013 at 5:53 PM, Anubhab Baksi wrote: > > Hi, > I was trying to write an additional function in misc/common. But, when I > tried to run that, I got an AttributeError, stating module object has no > such attribute. > > Can anybody, please, tell me why this is happening and how can I fix it? > You probably need to add the function name to "__all__". If that's not it, please show us your actual code. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From warren.weckesser at gmail.com Sun Jul 7 14:36:33 2013 From: warren.weckesser at gmail.com (Warren Weckesser) Date: Sun, 7 Jul 2013 14:36:33 -0400 Subject: [SciPy-Dev] `a` and `b` attributes of the stats distributions Message-ID: I've been working on a pull request for one of the distributions in the stats module, and as often happens, this has lead to the urge to do some more clean up. The distributions defined in stats/distributions.py have attributes `a` and `b` that give the lower and upper ends of the support of the distribution (i.e. outside of [a, b], the PDF is 0). A problem with this API is that for many distributions, the support depends on the parameters. Currently, this is handled in many of the distributions by modifying self.a and self.b in the _argcheck() method (I guess under the assumption that any method that needs `a` and `b` will have called `_argcheck()` at some point). This leads to stateful behavior such as: In [1]: from scipy.stats import genpareto In [2]: genpareto.b # Initially b is inf. Out[2]: inf In [3]: genpareto.cdf(0, -2) Out[3]: 0.0 In [4]: genpareto.b # Now b is 0.5 (as a scalar array) Out[4]: array(0.5) In [5]: genpareto.cdf(0, 1) Out[5]: 0.0 In [6]: genpareto.b Out[6]: array(inf) # b is back to inf, but as a scalar array. This API is ugly, and I'd like to fix it. To start, I think there should be a method `_support`: def _support(self, *args): """Returns a tuple (a,b) that gives the support of the distribution.""" ... All the distributions would be modified to call `self._support(...)` to get `a` and `b` instead of using the attributes `self.a` and `self.b`. (The method could be public--I just defaulted to private while working out the appropriate API.) It would be nice to get rid of `a` and `b` entirely, but they are currently public attributes, so I don't think we can simply remove them. Comments? Suggestions? Warren -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sun Jul 7 17:18:20 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 7 Jul 2013 23:18:20 +0200 Subject: [SciPy-Dev] `a` and `b` attributes of the stats distributions In-Reply-To: References: Message-ID: On Sun, Jul 7, 2013 at 8:36 PM, Warren Weckesser wrote: > I've been working on a pull request for one of the distributions in the > stats module, and as often happens, this has lead to the urge to do some > more clean up. > > The distributions defined in stats/distributions.py have attributes `a` > and `b` that give the lower and upper ends of the support of the > distribution (i.e. outside of [a, b], the PDF is 0). A problem with this > API is that for many distributions, the support depends on the parameters. > Currently, this is handled in many of the distributions by modifying self.a > and self.b in the _argcheck() method (I guess under the assumption that any > method that needs `a` and `b` will have called `_argcheck()` at some point). > > This leads to stateful behavior such as: > > In [1]: from scipy.stats import genpareto > > In [2]: genpareto.b # Initially b is inf. > Out[2]: inf > > In [3]: genpareto.cdf(0, -2) > Out[3]: 0.0 > > In [4]: genpareto.b # Now b is 0.5 (as a scalar array) > Out[4]: array(0.5) > > In [5]: genpareto.cdf(0, 1) > Out[5]: 0.0 > > In [6]: genpareto.b > Out[6]: array(inf) # b is back to inf, but as a scalar array. > > This API is ugly, and I'd like to fix it. > > To start, I think there should be a method `_support`: > > def _support(self, *args): > """Returns a tuple (a,b) that gives the support of the > distribution.""" > ... > The private methods shouldn't use ``*args``. Better would be to follow the pattern of pdf & co: a public method support() which defaults to -inf/inf, and an optional private method _support() which takes the correct fixed shape parameters. > All the distributions would be modified to call `self._support(...)` to > get `a` and `b` instead of using the attributes `self.a` and `self.b`. > (The method could be public--I just defaulted to private while working out > the appropriate API.) > > It would be nice to get rid of `a` and `b` entirely, but they are > currently public attributes, so I don't think we can simply remove them. > Backwards compatibility does seem to be a problem here. Not sure what the best solution is. Another issue is that self.a and self.b are used inside private methods, which themselves may not have the correct info to call support() again. Probably all fixable, but could be some work. The code will also become a little more verbose. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Sun Jul 7 18:08:15 2013 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Sun, 7 Jul 2013 18:08:15 -0400 Subject: [SciPy-Dev] `a` and `b` attributes of the stats distributions In-Reply-To: References: Message-ID: On Sun, Jul 7, 2013 at 5:18 PM, Ralf Gommers wrote: > > > > On Sun, Jul 7, 2013 at 8:36 PM, Warren Weckesser > wrote: >> >> I've been working on a pull request for one of the distributions in the >> stats module, and as often happens, this has lead to the urge to do some >> more clean up. >> >> The distributions defined in stats/distributions.py have attributes `a` >> and `b` that give the lower and upper ends of the support of the >> distribution (i.e. outside of [a, b], the PDF is 0). A problem with this >> API is that for many distributions, the support depends on the parameters. >> Currently, this is handled in many of the distributions by modifying self.a >> and self.b in the _argcheck() method (I guess under the assumption that any >> method that needs `a` and `b` will have called `_argcheck()` at some point). That's the assumption, and for distribution that change a or b, we cannot call the private ._xxx methods directly, while for t and normal distribution it's safe and I do it pretty often. >> >> This leads to stateful behavior such as: >> >> In [1]: from scipy.stats import genpareto >> >> In [2]: genpareto.b # Initially b is inf. >> Out[2]: inf >> >> In [3]: genpareto.cdf(0, -2) >> Out[3]: 0.0 >> >> In [4]: genpareto.b # Now b is 0.5 (as a scalar array) >> Out[4]: array(0.5) ugly but it's correct, the code needs to call _argcheck to set the correct bounds. Carrying over state from previous calls is pretty bad for debugging, but all the public methods do it correctly. >> >> In [5]: genpareto.cdf(0, 1) >> Out[5]: 0.0 >> >> In [6]: genpareto.b >> Out[6]: array(inf) # b is back to inf, but as a scalar array. >> >> This API is ugly, and I'd like to fix it. >> >> To start, I think there should be a method `_support`: >> >> def _support(self, *args): >> """Returns a tuple (a,b) that gives the support of the >> distribution.""" >> ... > > > The private methods shouldn't use ``*args``. Better would be to follow the > pattern of pdf & co: a public method support() which defaults to -inf/inf, > and an optional private method _support() which takes the correct fixed > shape parameters. > > >> >> All the distributions would be modified to call `self._support(...)` to >> get `a` and `b` instead of using the attributes `self.a` and `self.b`. (The >> method could be public--I just defaulted to private while working out the >> appropriate API.) >> >> It would be nice to get rid of `a` and `b` entirely, but they are >> currently public attributes, so I don't think we can simply remove them. > > > Backwards compatibility does seem to be a problem here. Not sure what the > best solution is. > > Another issue is that self.a and self.b are used inside private methods, > which themselves may not have the correct info to call support() again. > Probably all fixable, but could be some work. The code will also become a > little more verbose. My preferred solution to the state problem would be to create new instances each time, then we don't have to worry about carrying over and cleaning up state all the time My first impression is that `_support` would not help much, isn't it similar to what _argcheck currently does? We still have to store the information as attributes somewhere, or you have to recalculate the support each time. And then every piece of code that needs to know a and b needs to find it, either you hand it around as additional arguments, or you need every function or method to call `_support` which is similar to what the public methods currently do with _argcheck. (and the private methods do their work after the public methods have called _argcheck and set a and b if necessary). What would be useful is a public (top level generic) public function `support_bounds` (or something like that) that the user can use to get the support of the distribution. (but it won't change the internal headaches. it could just call _argcheck and return a and b, however translated by loc and scale, since a and b are the bounds of the standardized distribution (loc=0, scale=1) Josef > > Ralf > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From ralf.gommers at gmail.com Mon Jul 8 16:05:37 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 8 Jul 2013 22:05:37 +0200 Subject: [SciPy-Dev] `a` and `b` attributes of the stats distributions In-Reply-To: References: Message-ID: On Mon, Jul 8, 2013 at 12:08 AM, wrote: > On Sun, Jul 7, 2013 at 5:18 PM, Ralf Gommers > wrote: > > > > On Sun, Jul 7, 2013 at 8:36 PM, Warren Weckesser > > wrote: > >> > >> All the distributions would be modified to call `self._support(...)` to > >> get `a` and `b` instead of using the attributes `self.a` and `self.b`. > (The > >> method could be public--I just defaulted to private while working out > the > >> appropriate API.) > >> > >> It would be nice to get rid of `a` and `b` entirely, but they are > >> currently public attributes, so I don't think we can simply remove them. > > > > > > Backwards compatibility does seem to be a problem here. Not sure what the > > best solution is. > > > > Another issue is that self.a and self.b are used inside private methods, > > which themselves may not have the correct info to call support() again. > > Probably all fixable, but could be some work. The code will also become a > > little more verbose. > > My preferred solution to the state problem would be to create new > instances each time, then we don't have to worry about carrying over > and cleaning up state all the time > If we could start from scratch that would indeed be the way to go. I don't think it's that much of a problem that we have to go and break all code using the instances now though. So that leaves us with incremental improvements. > > My first impression is that `_support` would not help much, isn't it > similar to what _argcheck currently does? We still have to store the > information as attributes somewhere, or you have to recalculate the > support each time. > I think the latter was the idea - recalculating is cheap. > And then every piece of code that needs to know a and b needs to find > it, either you hand it around as additional arguments, or you need > every function or method to call `_support` which is similar to what > the public methods currently do with _argcheck. > (and the private methods do their work after the public methods have > called _argcheck and set a and b if necessary). > > > What would be useful is a public (top level generic) public function > `support_bounds` (or something like that) that the user can use to get > the support of the distribution. (but it won't change the internal > headaches. > > it could just call _argcheck and return a and b, however translated by > loc and scale, since a and b are the bounds of the standardized > distribution (loc=0, scale=1) > That could indeed be useful, it's simple to add. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Tue Jul 9 10:10:54 2013 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Tue, 9 Jul 2013 10:10:54 -0400 Subject: [SciPy-Dev] `a` and `b` attributes of the stats distributions In-Reply-To: References: Message-ID: On Mon, Jul 8, 2013 at 4:05 PM, Ralf Gommers wrote: > > > > On Mon, Jul 8, 2013 at 12:08 AM, wrote: >> >> On Sun, Jul 7, 2013 at 5:18 PM, Ralf Gommers >> wrote: >> > >> > On Sun, Jul 7, 2013 at 8:36 PM, Warren Weckesser >> > wrote: >> >> >> >> All the distributions would be modified to call `self._support(...)` to >> >> get `a` and `b` instead of using the attributes `self.a` and `self.b`. >> >> (The >> >> method could be public--I just defaulted to private while working out >> >> the >> >> appropriate API.) >> >> >> >> It would be nice to get rid of `a` and `b` entirely, but they are >> >> currently public attributes, so I don't think we can simply remove >> >> them. >> > >> > >> > Backwards compatibility does seem to be a problem here. Not sure what >> > the >> > best solution is. >> > >> > Another issue is that self.a and self.b are used inside private methods, >> > which themselves may not have the correct info to call support() again. >> > Probably all fixable, but could be some work. The code will also become >> > a >> > little more verbose. >> >> My preferred solution to the state problem would be to create new >> instances each time, then we don't have to worry about carrying over >> and cleaning up state all the time > > > If we could start from scratch that would indeed be the way to go. I don't > think it's that much of a problem that we have to go and break all code > using the instances now though. So that leaves us with incremental > improvements. > >> >> >> My first impression is that `_support` would not help much, isn't it >> similar to what _argcheck currently does? We still have to store the >> information as attributes somewhere, or you have to recalculate the >> support each time. > > > I think the latter was the idea - recalculating is cheap. Nothing is cheap if you need to do it a few million times http://mail.scipy.org/pipermail/scipy-dev/2008-November/010331.html Currently, we check and set a, b once in the main methods .a and .b are used in the individual calculations for example .a and .b are used inside a brentq loop https://github.com/scipy/scipy/blob/master/scipy/stats/distributions.py#L1079 But doing a search, I didn't find many cases where .a .b are used in second layer generic functions, I had thought there are more. (If only pdf is given, then cdf uses integrate.quad with .a and .b, ppf uses brentq which need to repeatedly call cdf - double loop with .a and .b in both loops, IIRC I assume a call to numpy.vectorize will also evaluate it again and again for each element.) possibly a bug https://github.com/scipy/scipy/issues/2622 Josef > >> >> And then every piece of code that needs to know a and b needs to find >> it, either you hand it around as additional arguments, or you need >> every function or method to call `_support` which is similar to what >> the public methods currently do with _argcheck. >> (and the private methods do their work after the public methods have >> called _argcheck and set a and b if necessary). >> >> >> What would be useful is a public (top level generic) public function >> `support_bounds` (or something like that) that the user can use to get >> the support of the distribution. (but it won't change the internal >> headaches. >> >> it could just call _argcheck and return a and b, however translated by >> loc and scale, since a and b are the bounds of the standardized >> distribution (loc=0, scale=1) > > > That could indeed be useful, it's simple to add. > > Ralf > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From ralf.gommers at gmail.com Wed Jul 10 17:13:43 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Wed, 10 Jul 2013 23:13:43 +0200 Subject: [SciPy-Dev] subclassing scipy.stats distributions - request for feedback Message-ID: Hi all, We're very close to merging https://github.com/scipy/scipy/pull/2588, which is a nice enhancement by Evgeni Burovski that allows specifying shape parameters as keyword arguments for all distribution methods. With this kind of change it's hard to verify that we're not breaking any non-standard subclasses that people have written. Therefore I have the following requests: 1. please test your code with that PR 2. please provide code samples or descriptions of the way you subclass distributions (if different from what's done in scipy itself), to be used to augment the test suite and documentation. Thanks, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From suryak at ieee.org Mon Jul 15 11:39:11 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 15 Jul 2013 17:39:11 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions Message-ID: Hi all, [idea] Right now we are using Hg for storing submissions inside static root..Is not totally good on a cloud platform with limited resources..its not fast too (takes lot of server load) I rather would like to use SQLite separately for storing submissions (I agree that using Postgre SQL for it is expensive on cloud).. What do you guys say? we dont have to provide all revisions as a Hg repo when downloading.. people can employ some tools to compare if they want... looking for your comments cheers surya -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at hilboll.de Mon Jul 15 11:46:42 2013 From: lists at hilboll.de (Andreas Hilboll) Date: Mon, 15 Jul 2013 17:46:42 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: References: Message-ID: <51E41962.3070405@hilboll.de> On 15.07.2013 17:39, Surya Kasturi wrote: > Hi all, > > [idea] > > Right now we are using Hg for storing submissions inside static root..Is > not totally good on a cloud platform with limited resources..its not > fast too (takes lot of server load) > > I rather would like to use SQLite separately for storing submissions (I > agree that using Postgre SQL for it is expensive on cloud).. > > What do you guys say? > > > we dont have to provide all revisions as a Hg repo when downloading.. > people can employ some tools to compare if they want... I disagree. Using SQLite in a production server environment causes nothing but pain (at least to my experience). I would assume using some DVCS for storing revisions should be the most natural way to do so, and I would hope that there's some webapp libraries availble to show diff's etc, though I don't have any direct experience with this. Oh, and I personally don't see the need to have spyce running on a "cloud platform" (depending on what you mean by that, of course). If you have one single server, then you don't need all the distributed-whatsoever and can concentrate on having everything on one machine. Just my 2ct. -- Andreas. From suryak at ieee.org Mon Jul 15 12:06:00 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 15 Jul 2013 18:06:00 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: <51E41962.3070405@hilboll.de> References: <51E41962.3070405@hilboll.de> Message-ID: On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll wrote: > On 15.07.2013 17:39, Surya Kasturi wrote: > > Hi all, > > > > [idea] > > > > Right now we are using Hg for storing submissions inside static root..Is > > not totally good on a cloud platform with limited resources..its not > > fast too (takes lot of server load) > > > > I rather would like to use SQLite separately for storing submissions (I > > agree that using Postgre SQL for it is expensive on cloud).. > > > > What do you guys say? > > > > > > we dont have to provide all revisions as a Hg repo when downloading.. > > people can employ some tools to compare if they want... > > I disagree. Using SQLite in a production server environment causes > nothing but pain (at least to my experience). I would assume using some > DVCS for storing revisions should be the most natural way to do so, and > I would hope that there's some webapp libraries availble to show diff's > etc, though I don't have any direct experience with this. > > Showing diff is one idea I had too but we can figure out something within database too (I guess... dont know for sure). If we are not a busy site.. with very huge traffic and only using SQLite partly, it should work nicely http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site > Oh, and I personally don't see the need to have spyce running on a > "cloud platform" (depending on what you mean by that, of course). If you > have one single server, then you don't need all the > distributed-whatsoever and can concentrate on having everything on one > machine. > > I didnt get you > Just my 2ct. > > -- Andreas. > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From suryak at ieee.org Mon Jul 15 12:13:54 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 15 Jul 2013 18:13:54 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: References: <51E41962.3070405@hilboll.de> Message-ID: On Mon, Jul 15, 2013 at 6:06 PM, Surya Kasturi wrote: > > > > On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll wrote: > >> On 15.07.2013 17:39, Surya Kasturi wrote: >> > Hi all, >> > >> > [idea] >> > >> > Right now we are using Hg for storing submissions inside static root..Is >> > not totally good on a cloud platform with limited resources..its not >> > fast too (takes lot of server load) >> > >> > I rather would like to use SQLite separately for storing submissions (I >> > agree that using Postgre SQL for it is expensive on cloud).. >> > >> > What do you guys say? >> > >> > >> > we dont have to provide all revisions as a Hg repo when downloading.. >> > people can employ some tools to compare if they want... >> >> I disagree. Using SQLite in a production server environment causes >> nothing but pain (at least to my experience). I would assume using some >> DVCS for storing revisions should be the most natural way to do so, and >> I would hope that there's some webapp libraries availble to show diff's >> etc, though I don't have any direct experience with this. >> >> > Showing diff is one idea I had too but we can figure out something within > database too (I guess... dont know for sure). > If we are not a busy site.. with very huge traffic and only using SQLite > partly, it should work nicely > > > http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site > > >> Oh, and I personally don't see the need to have spyce running on a >> "cloud platform" (depending on what you mean by that, of course). If you >> have one single server, then you don't need all the >> distributed-whatsoever and can concentrate on having everything on one >> machine. >> >> I didnt get you > > >> Just my 2ct. >> >> -- Andreas. >> >> _______________________________________________ >> SciPy-Dev mailing list >> SciPy-Dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/scipy-dev >> > > Truly, I barely have experience with databases, queries etc.. but I guess, its not bad experimenting things.. couchDB does some fantastic stuff.. May be there are other technologies we can use.. (searching!!) I personally don't feel right using repos in static data.. just feel that its very complex that way.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at hilboll.de Mon Jul 15 12:15:10 2013 From: lists at hilboll.de (Andreas Hilboll) Date: Mon, 15 Jul 2013 18:15:10 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: References: <51E41962.3070405@hilboll.de> Message-ID: <51E4200E.5020803@hilboll.de> On 15.07.2013 18:06, Surya Kasturi wrote: > > > > On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll > wrote: > > On 15.07.2013 17:39, Surya Kasturi wrote: > > Hi all, > > > > [idea] > > > > Right now we are using Hg for storing submissions inside static > root..Is > > not totally good on a cloud platform with limited resources..its not > > fast too (takes lot of server load) > > > > I rather would like to use SQLite separately for storing > submissions (I > > agree that using Postgre SQL for it is expensive on cloud).. > > > > What do you guys say? > > > > > > we dont have to provide all revisions as a Hg repo when downloading.. > > people can employ some tools to compare if they want... > > I disagree. Using SQLite in a production server environment causes > nothing but pain (at least to my experience). I would assume using some > DVCS for storing revisions should be the most natural way to do so, and > I would hope that there's some webapp libraries availble to show diff's > etc, though I don't have any direct experience with this. > > > Showing diff is one idea I had too but we can figure out something > within database too (I guess... dont know for sure). > If we are not a busy site.. with very huge traffic and only using SQLite > partly, it should work nicely > > http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site >From my experience, it doesn't, even on low-traffic sites. SQLite doesn't handle concurrency, so every so often you get some "database locked" errors. SQLite is good for development stage, but for production sites, don't use it. BTW, I don't see a problem using PostgreSQL as a database server. Especially for low traffic sites, the performance should be fine. > > > Oh, and I personally don't see the need to have spyce running on a > "cloud platform" (depending on what you mean by that, of course). If you > have one single server, then you don't need all the > distributed-whatsoever and can concentrate on having everything on one > machine. > > I didnt get you Never mind. Just a pun about this "cloud hype" terminology. Cheers, Andreas. > > > Just my 2ct. > > -- Andreas. > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > From suryak at ieee.org Mon Jul 15 12:16:57 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 15 Jul 2013 18:16:57 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: <51E4200E.5020803@hilboll.de> References: <51E41962.3070405@hilboll.de> <51E4200E.5020803@hilboll.de> Message-ID: On Mon, Jul 15, 2013 at 6:15 PM, Andreas Hilboll wrote: > On 15.07.2013 18:06, Surya Kasturi wrote: > > > > > > > > On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll > > wrote: > > > > On 15.07.2013 17:39, Surya Kasturi wrote: > > > Hi all, > > > > > > [idea] > > > > > > Right now we are using Hg for storing submissions inside static > > root..Is > > > not totally good on a cloud platform with limited resources..its > not > > > fast too (takes lot of server load) > > > > > > I rather would like to use SQLite separately for storing > > submissions (I > > > agree that using Postgre SQL for it is expensive on cloud).. > > > > > > What do you guys say? > > > > > > > > > we dont have to provide all revisions as a Hg repo when > downloading.. > > > people can employ some tools to compare if they want... > > > > I disagree. Using SQLite in a production server environment causes > > nothing but pain (at least to my experience). I would assume using > some > > DVCS for storing revisions should be the most natural way to do so, > and > > I would hope that there's some webapp libraries availble to show > diff's > > etc, though I don't have any direct experience with this. > > > > > > Showing diff is one idea I had too but we can figure out something > > within database too (I guess... dont know for sure). > > If we are not a busy site.. with very huge traffic and only using SQLite > > partly, it should work nicely > > > > > http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site > > From my experience, it doesn't, even on low-traffic sites. SQLite > doesn't handle concurrency, so every so often you get some "database > locked" errors. SQLite is good for development stage, but for production > sites, don't use it. > > BTW, I don't see a problem using PostgreSQL as a database server. > Especially for low traffic sites, the performance should be fine. > > > > > > > Oh, and I personally don't see the need to have spyce running on a > > "cloud platform" (depending on what you mean by that, of course). If > you > > have one single server, then you don't need all the > > distributed-whatsoever and can concentrate on having everything on > one > > machine. > > > > I didnt get you > > Never mind. Just a pun about this "cloud hype" terminology. > > Cheers, Andreas. > > > > > > > > Just my 2ct. > > > > -- Andreas. > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > > > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > I am not worried about current PostgreSQL.. I am thinking to do something with static submissions data (as Hg repos) on filesystem.. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at hilboll.de Mon Jul 15 12:21:13 2013 From: lists at hilboll.de (Andreas Hilboll) Date: Mon, 15 Jul 2013 18:21:13 +0200 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: References: <51E41962.3070405@hilboll.de> <51E4200E.5020803@hilboll.de> Message-ID: <51E42179.10906@hilboll.de> On 15.07.2013 18:16, Surya Kasturi wrote: > > > > On Mon, Jul 15, 2013 at 6:15 PM, Andreas Hilboll > wrote: > > On 15.07.2013 18:06, Surya Kasturi wrote: > > > > > > > > On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll > > >> wrote: > > > > On 15.07.2013 17:39, Surya Kasturi wrote: > > > Hi all, > > > > > > [idea] > > > > > > Right now we are using Hg for storing submissions inside static > > root..Is > > > not totally good on a cloud platform with limited > resources..its not > > > fast too (takes lot of server load) > > > > > > I rather would like to use SQLite separately for storing > > submissions (I > > > agree that using Postgre SQL for it is expensive on cloud).. > > > > > > What do you guys say? > > > > > > > > > we dont have to provide all revisions as a Hg repo when > downloading.. > > > people can employ some tools to compare if they want... > > > > I disagree. Using SQLite in a production server environment causes > > nothing but pain (at least to my experience). I would assume > using some > > DVCS for storing revisions should be the most natural way to > do so, and > > I would hope that there's some webapp libraries availble to > show diff's > > etc, though I don't have any direct experience with this. > > > > > > Showing diff is one idea I had too but we can figure out something > > within database too (I guess... dont know for sure). > > If we are not a busy site.. with very huge traffic and only using > SQLite > > partly, it should work nicely > > > > > http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site > > From my experience, it doesn't, even on low-traffic sites. SQLite > doesn't handle concurrency, so every so often you get some "database > locked" errors. SQLite is good for development stage, but for production > sites, don't use it. > > BTW, I don't see a problem using PostgreSQL as a database server. > Especially for low traffic sites, the performance should be fine. > > > > > > > Oh, and I personally don't see the need to have spyce running on a > > "cloud platform" (depending on what you mean by that, of > course). If you > > have one single server, then you don't need all the > > distributed-whatsoever and can concentrate on having > everything on one > > machine. > > > > I didnt get you > > Never mind. Just a pun about this "cloud hype" terminology. > > Cheers, Andreas. > > > > > > > > Just my 2ct. > > > > -- Andreas. > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > > > > > > _______________________________________________ > > SciPy-Dev mailing list > > SciPy-Dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/scipy-dev > > > > > I am not worried about current PostgreSQL.. I am thinking to do > something with static submissions data (as Hg repos) on filesystem.. Well, in your original mail, you said > I rather would like to use SQLite separately for storing submissions (I agree that using Postgre SQL for it is expensive on cloud).. But also for file-system interactions with hg or git I wouldn't be worried with a low-traffic site. After all, gitorious, bitbucket, github, and the like are all based on DVCS which store their data in the filesystem. I doubt that github have their repositories in some SQL database (but might be mistaken here) ... -- Andreas. From pav at iki.fi Mon Jul 15 13:34:36 2013 From: pav at iki.fi (Pauli Virtanen) Date: Mon, 15 Jul 2013 20:34:36 +0300 Subject: [SciPy-Dev] using sqlite for storing submissions In-Reply-To: <51E42179.10906@hilboll.de> References: <51E41962.3070405@hilboll.de> <51E4200E.5020803@hilboll.de> <51E42179.10906@hilboll.de> Message-ID: 15.07.2013 19:21, Andreas Hilboll kirjoitti: [clip] > Well, in your original mail, you said > > > I rather would like to use SQLite separately for storing submissions > > (I agree that using Postgre SQL for it is expensive on cloud).. Well, here "Sqlite" should be interpreted as "SQL database". It'll go through an ORM, so the backend can be Postgres or whatever. > But also for file-system interactions with hg or git I wouldn't be > worried with a low-traffic site. After all, gitorious, bitbucket, > github, and the like are all based on DVCS which store their data in the > filesystem. I doubt that github have their repositories in some SQL > database (but might be mistaken here) ... A problem may be the spawning of hg processes. It needs to load a whole Python interpreter to memory. However, if hg is only invoked on upload time, it probably doesn't matter. I don't remember how it works currently. I'd expect that direct file access is not a bottleneck. So before trying to change this, I'd recommend measuring that this is actually a problem. -- Pauli Virtanen From juanlu001 at gmail.com Tue Jul 16 10:49:57 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Tue, 16 Jul 2013 16:49:57 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value Message-ID: <51E55D95.5020707@gmail.com> Hello all, I've been trying the scipy.signal module lately for some easy things and I had an idea, but I thought it would be better to bring it up in the mailing list before filing a PR on GitHub. My proposal is to implement the __call__ method on signal.lti objects so it returns the value on the transfer function. I took the idea from the `bode` function. This line: y = numpy.polyval(sys.num, jw) / numpy.polyval(sys.den, jw) (https://github.com/scipy/scipy/blob/v0.12.0/scipy/signal/ltisys.py#L940) could be moved to signal.lti.__call__, and then in `bode` write y = sys(jw) The detail is trivial (so is the implementation) but I think it would be a useful thing. What do others think? Shall I open a pull request? From juanlu001 at gmail.com Tue Jul 16 11:28:05 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Tue, 16 Jul 2013 17:28:05 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: <51E55D95.5020707@gmail.com> References: <51E55D95.5020707@gmail.com> Message-ID: <51E56685.6090707@gmail.com> BTW (and this is a side question), is there a reason the code of signal.lti is full of `self.__dict__['X'] = Y` instead of `self.X = Y`? https://github.com/scipy/scipy/blob/v0.12.0/scipy/signal/ltisys.py#L262 From smortaz at exchange.microsoft.com Thu Jul 18 17:57:50 2013 From: smortaz at exchange.microsoft.com (Shahrokh Mortazavi) Date: Thu, 18 Jul 2013 21:57:50 +0000 Subject: [SciPy-Dev] Mixed Python + C debugging support in Visual Studio Message-ID: Hi folks, 1st time poster - apologies if I'm breaking any protocols... We were told that this would be a good alias to announce this on: a few Python & OSS enthusiasts and Microsoft have created a plug-in for Visual Studio that enables Python <-> C/C++ debugging. You may find this useful for debugging your extension modules. A quick video overview of the mixed mode debugging feature: http://www.youtube.com/watch?v=wvJaKQ94lBY&hd=1 (HD) Documentation: https://pytools.codeplex.com/wikipage?title=Mixed-mode%20debugging Python Tools for Visual Studio is free (and OSS): http://pytools.codeplex.com Cheers, s -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Jul 20 11:39:36 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 20 Jul 2013 17:39:36 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: <51E55D95.5020707@gmail.com> References: <51E55D95.5020707@gmail.com> Message-ID: On Tue, Jul 16, 2013 at 4:49 PM, Juan Luis Cano wrote: > Hello all, > > I've been trying the scipy.signal module lately for some easy things and > I had an idea, but I thought it would be better to bring it up in the > mailing list before filing a PR on GitHub. > > My proposal is to implement the __call__ method on signal.lti objects so > it returns the value on the transfer function. > > I took the idea from the `bode` function. This line: > > y = numpy.polyval(sys.num, jw) / numpy.polyval(sys.den, jw) > > (https://github.com/scipy/scipy/blob/v0.12.0/scipy/signal/ltisys.py#L940) > > could be moved to signal.lti.__call__, and then in `bode` write > > y = sys(jw) > > The detail is trivial (so is the implementation) but I think it would be > a useful thing. What do others think? Shall I open a pull request? > Not sure, is writing y = lti_sys(jw) instead of jw, y = lti_sys.freqresp(jw) clearer and umambiguous? It may make sense, but would be good to get the opinion of some users on this. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Sat Jul 20 11:54:46 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sat, 20 Jul 2013 17:54:46 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: <51E56685.6090707@gmail.com> References: <51E55D95.5020707@gmail.com> <51E56685.6090707@gmail.com> Message-ID: On Tue, Jul 16, 2013 at 5:28 PM, Juan Luis Cano wrote: > BTW (and this is a side question), is there a reason the code of > signal.lti is full of `self.__dict__['X'] = Y` instead of `self.X = Y`? > It's very ugly but does have a function. Because of how ``__setattr__`` is implemented you cannot just set self.num & related attributes in ``__init__``. The alternative is to transform num/den/zeros/poles/.... into properties with setter methods that keep all properties in sync. A PR making that change would be welcome. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanlu001 at gmail.com Tue Jul 23 06:29:38 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Tue, 23 Jul 2013 12:29:38 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: References: <51E55D95.5020707@gmail.com> Message-ID: <51EE5B12.3090703@gmail.com> On 07/20/2013 05:39 PM, Ralf Gommers wrote: > > > > On Tue, Jul 16, 2013 at 4:49 PM, Juan Luis Cano > wrote: > > Hello all, > > I've been trying the scipy.signal module lately for some easy > things and > I had an idea, but I thought it would be better to bring it up in the > mailing list before filing a PR on GitHub. > > My proposal is to implement the __call__ method on signal.lti > objects so > it returns the value on the transfer function. > > I took the idea from the `bode` function. This line: > > y = numpy.polyval(sys.num, jw) / numpy.polyval(sys.den, jw) > > (https://github.com/scipy/scipy/blob/v0.12.0/scipy/signal/ltisys.py#L940) > > could be moved to signal.lti.__call__, and then in `bode` write > > y = sys(jw) > > The detail is trivial (so is the implementation) but I think it > would be > a useful thing. What do others think? Shall I open a pull request? > > > Not sure, is writing > y = lti_sys(jw) > instead of > jw, y = lti_sys.freqresp(jw) > clearer and umambiguous? It may make sense, but would be good to get > the opinion of some users on this. > > Ralf I am sorry but I was unaware of the freqresp function, probably it's the clearest way. Maybe it should be used in the `bode` function too. Juan Luis Cano -------------- next part -------------- An HTML attachment was scrubbed... URL: From warren.weckesser at gmail.com Tue Jul 23 08:09:43 2013 From: warren.weckesser at gmail.com (Warren Weckesser) Date: Tue, 23 Jul 2013 08:09:43 -0400 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: <51EE5B12.3090703@gmail.com> References: <51E55D95.5020707@gmail.com> <51EE5B12.3090703@gmail.com> Message-ID: On 7/23/13, Juan Luis Cano wrote: > On 07/20/2013 05:39 PM, Ralf Gommers wrote: >> >> >> >> On Tue, Jul 16, 2013 at 4:49 PM, Juan Luis Cano > > wrote: >> >> Hello all, >> >> I've been trying the scipy.signal module lately for some easy >> things and >> I had an idea, but I thought it would be better to bring it up in the >> mailing list before filing a PR on GitHub. >> >> My proposal is to implement the __call__ method on signal.lti >> objects so >> it returns the value on the transfer function. >> >> I took the idea from the `bode` function. This line: >> >> y = numpy.polyval(sys.num, jw) / numpy.polyval(sys.den, jw) >> >> >> (https://github.com/scipy/scipy/blob/v0.12.0/scipy/signal/ltisys.py#L940) >> >> could be moved to signal.lti.__call__, and then in `bode` write >> >> y = sys(jw) >> >> The detail is trivial (so is the implementation) but I think it >> would be >> a useful thing. What do others think? Shall I open a pull request? >> >> >> Not sure, is writing >> y = lti_sys(jw) >> instead of >> jw, y = lti_sys.freqresp(jw) >> clearer and umambiguous? It may make sense, but would be good to get >> the opinion of some users on this. >> >> Ralf > > I am sorry but I was unaware of the freqresp function, probably it's the > clearest way. Maybe it should be used in the `bode` function too. There was some work on bode and freqresp earlier this year. The latest version of `bode` uses `freqresp`: https://github.com/scipy/scipy/blob/master/scipy/signal/ltisys.py#L856 Warren > > Juan Luis Cano > From juanlu001 at gmail.com Tue Jul 23 08:38:22 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Tue, 23 Jul 2013 14:38:22 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: References: <51E55D95.5020707@gmail.com> <51EE5B12.3090703@gmail.com> Message-ID: <51EE793E.9070001@gmail.com> On 07/23/2013 02:09 PM, Warren Weckesser wrote: > There was some work on bode and freqresp earlier this year. The > latest version of `bode` uses `freqresp`: > https://github.com/scipy/scipy/blob/master/scipy/signal/ltisys.py#L856 Thank you for pointing it out, I was reading the source from my locally installed last tagged version. From juanlu001 at gmail.com Tue Jul 23 12:05:12 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Tue, 23 Jul 2013 18:05:12 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: References: <51E55D95.5020707@gmail.com> <51E56685.6090707@gmail.com> Message-ID: <51EEA9B8.9000902@gmail.com> On 07/20/2013 05:54 PM, Ralf Gommers wrote: > > > > On Tue, Jul 16, 2013 at 5:28 PM, Juan Luis Cano > wrote: > > BTW (and this is a side question), is there a reason the code of > signal.lti is full of `self.__dict__['X'] = Y` instead of `self.X > = Y`? > > > It's very ugly but does have a function. Because of how > ``__setattr__`` is implemented you cannot just set self.num & related > attributes in ``__init__``. The alternative is to transform > num/den/zeros/poles/.... into properties with setter methods that keep > all properties in sync. A PR making that change would be welcome. If I understood correctly, it would be something like this: @property num(self): """Numerator""" return self._num @num.setter num(self, value): self._num = value self.__dict__['zeros'], self.__dict__['poles'], self.dict__['gain'] = tf2zpk(self.num, self.den) self.__dict__['A'], self.__dict__['B'], self.dict__['C'], self.dict__['D'] = tf2ss(self.num, self.den) then you really cannot get rid of `self.__dict__['X']` because if you use the `self.X` then the same circular problem would happen with the setters of the other properties. And on the other hand, when instantiating the object not all the properties are available (e.g.: you assign `num`, but cannot update zeros, poles and gain because `den` is not yet assigned) so a special case has to be included in the setter. What about, instead of storing redundant information on the object, a "canonical" representation is used, for instance (A, B, C, D), and getters are created that compute (num, den) and (zeros, poles, gain)? Maybe one could even cache these properties so they would be computed only once anyway. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Tue Jul 23 17:47:13 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 23 Jul 2013 23:47:13 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: <51EEA9B8.9000902@gmail.com> References: <51E55D95.5020707@gmail.com> <51E56685.6090707@gmail.com> <51EEA9B8.9000902@gmail.com> Message-ID: On Tue, Jul 23, 2013 at 6:05 PM, Juan Luis Cano wrote: > On 07/20/2013 05:54 PM, Ralf Gommers wrote: > > > > > On Tue, Jul 16, 2013 at 5:28 PM, Juan Luis Cano wrote: > >> BTW (and this is a side question), is there a reason the code of >> signal.lti is full of `self.__dict__['X'] = Y` instead of `self.X = Y`? >> > > It's very ugly but does have a function. Because of how ``__setattr__`` > is implemented you cannot just set self.num & related attributes in > ``__init__``. The alternative is to transform num/den/zeros/poles/.... into > properties with setter methods that keep all properties in sync. A PR > making that change would be welcome. > > > If I understood correctly, it would be something like this: > > @property num(self): > """Numerator""" > return self._num > > @num.setter num(self, value): > self._num = value > self.__dict__['zeros'], self.__dict__['poles'], self.dict__['gain'] = > tf2zpk(self.num, self.den) > self.__dict__['A'], self.__dict__['B'], self.dict__['C'], > self.dict__['D'] = tf2ss(self.num, self.den) > > then you really cannot get rid of `self.__dict__['X']` because if you use > the `self.X` then the same circular problem would happen with the setters > of the other properties. > The setter methods can use the private attributes to get around that, for example num.setter updates self._den and not self.den. And on the other hand, when instantiating the object not all the properties > are available (e.g.: you assign `num`, but cannot update zeros, poles and > gain because `den` is not yet assigned) so a special case has to be > included in the setter. > Not really, for the same reason. Assign to self._num and self._den. > What about, instead of storing redundant information on the object, a > "canonical" representation is used, for instance (A, B, C, D), and getters > are created that compute (num, den) and (zeros, poles, gain)? Maybe one > could even cache these properties so they would be computed only once > anyway. > Should also work, but will likely be more complicated. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanlu001 at gmail.com Wed Jul 24 04:06:36 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Wed, 24 Jul 2013 10:06:36 +0200 Subject: [SciPy-Dev] Proposal: calling signal.lti object returns transfer function value In-Reply-To: References: <51E55D95.5020707@gmail.com> <51E56685.6090707@gmail.com> <51EEA9B8.9000902@gmail.com> Message-ID: <51EF8B0C.2070702@gmail.com> On 07/23/2013 11:47 PM, Ralf Gommers wrote: > > > > On Tue, Jul 23, 2013 at 6:05 PM, Juan Luis Cano > wrote: > > On 07/20/2013 05:54 PM, Ralf Gommers wrote: >> >> >> >> On Tue, Jul 16, 2013 at 5:28 PM, Juan Luis Cano >> > wrote: >> >> BTW (and this is a side question), is there a reason the code of >> signal.lti is full of `self.__dict__['X'] = Y` instead of >> `self.X = Y`? >> >> >> It's very ugly but does have a function. Because of how >> ``__setattr__`` is implemented you cannot just set self.num & >> related attributes in ``__init__``. The alternative is to >> transform num/den/zeros/poles/.... into properties with setter >> methods that keep all properties in sync. A PR making that change >> would be welcome. > > If I understood correctly, it would be something like this: > > @property num(self): > """Numerator""" > return self._num > > @num.setter num(self, value): > self._num = value > self.__dict__['zeros'], self.__dict__['poles'], > self.dict__['gain'] = tf2zpk(self.num, self.den) > self.__dict__['A'], self.__dict__['B'], self.dict__['C'], > self.dict__['D'] = tf2ss(self.num, self.den) > > then you really cannot get rid of `self.__dict__['X']` because if > you use the `self.X` then the same circular problem would happen > with the setters of the other properties. > > > The setter methods can use the private attributes to get around that, > for example num.setter updates self._den and not self.den. > > And on the other hand, when instantiating the object not all the > properties are available (e.g.: you assign `num`, but cannot > update zeros, poles and gain because `den` is not yet assigned) so > a special case has to be included in the setter. > > > Not really, for the same reason. Assign to self._num and self._den. Thank you very much for your advice! I implemented these ideas in this pull request: https://github.com/scipy/scipy/pull/2655 I hope we can follow the discussion there and improve the code. Regards Juan Luis Cano -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanlu001 at gmail.com Wed Jul 24 05:05:55 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Wed, 24 Jul 2013 11:05:55 +0200 Subject: [SciPy-Dev] Version 0.12.1? And some newcomer questions Message-ID: <51EF98F3.7070002@gmail.com> I would like to know if there are any plans to release a bugfix version of SciPy 0.12. I've been working with scipy.signal lately and I've stumbled upon some of the bugs that have been fixed on master in April and May since 0.12.0. But in a more general sense, I don't know if there is a roadmap to backport some bugfixes to maintenance/0.12.x (or if there is help needed?) because there is no hint in the issues milestones. I've just submitted my first code contribution to SciPy and as a newcomer (also willing to make some more) I was looking for this kind of information: workflow, guidelines, roadmap, future plans... And I have to say that it is scarce. For example, I was looking for the best way to work with SciPy source code and I ended up doing this: 1. Created a virtualenv 2. Cloned scipy git repo 3. Added `import setuptools` in setup.py 4. Sourced the virtualenv and run `python setup.py develop` this was the easiest way I found to run the tests while coding and not having to reinstall the whole thing every time I changed a file or try some tricky imports from the source directory. But I am sure there is a much better way (and I wonder what other SciPy devs do?). Thanks in advance, best regards, Juan Luis Cano From pav at iki.fi Wed Jul 24 08:47:48 2013 From: pav at iki.fi (Pauli Virtanen) Date: Wed, 24 Jul 2013 15:47:48 +0300 Subject: [SciPy-Dev] Version 0.12.1? And some newcomer questions In-Reply-To: <51EF98F3.7070002@gmail.com> References: <51EF98F3.7070002@gmail.com> Message-ID: 24.07.2013 12:05, Juan Luis Cano kirjoitti: > I would like to know if there are any plans to release a bugfix version > of SciPy 0.12. I've been working with scipy.signal lately and I've > stumbled upon some of the bugs that have been fixed on master in April > and May since 0.12.0. But in a more general sense, I don't know if there > is a roadmap to backport some bugfixes to maintenance/0.12.x (or if > there is help needed?) because there is no hint in the issues milestones. I think the current plan is to not do a point release before 0.13, which is on schedule in the next few months. > I've just submitted my first code contribution to SciPy and as a > newcomer (also willing to make some more) I was looking for this kind of > information: workflow, guidelines, roadmap, future plans... And I have > to say that it is scarce. For example, I was looking for the best way to > work with SciPy source code and I ended up doing this: Did you find these: https://github.com/scipy/scipy/blob/master/HACKING.rst.txt http://docs.scipy.org/doc/scipy/reference/hacking.html If yes, specific suggestions on what to improve would be welcome. -- Pauli Virtanen From juanlu001 at gmail.com Wed Jul 24 09:55:23 2013 From: juanlu001 at gmail.com (Juan Luis Cano) Date: Wed, 24 Jul 2013 15:55:23 +0200 Subject: [SciPy-Dev] Version 0.12.1? And some newcomer questions In-Reply-To: References: <51EF98F3.7070002@gmail.com> Message-ID: <51EFDCCB.1030901@gmail.com> On 07/24/2013 02:47 PM, Pauli Virtanen wrote: > 24.07.2013 12:05, Juan Luis Cano kirjoitti: >> I would like to know if there are any plans to release a bugfix version >> of SciPy 0.12. I've been working with scipy.signal lately and I've >> stumbled upon some of the bugs that have been fixed on master in April >> and May since 0.12.0. But in a more general sense, I don't know if there >> is a roadmap to backport some bugfixes to maintenance/0.12.x (or if >> there is help needed?) because there is no hint in the issues milestones. > I think the current plan is to not do a point release before 0.13, which > is on schedule in the next few months. Fair enough, good to know! >> I've just submitted my first code contribution to SciPy and as a >> newcomer (also willing to make some more) I was looking for this kind of >> information: workflow, guidelines, roadmap, future plans... And I have >> to say that it is scarce. For example, I was looking for the best way to >> work with SciPy source code and I ended up doing this: > Did you find these: > > https://github.com/scipy/scipy/blob/master/HACKING.rst.txt > http://docs.scipy.org/doc/scipy/reference/hacking.html > > If yes, specific suggestions on what to improve would be welcome. Well, indeed I have read those but somehow I missed the part I was interested in (the third item on the FAQ). I have read this: http://docs.scipy.org/doc/numpy/dev/index.html which gives extensive information on how to clone the repo but not how to create a dev installation. Speaking of which, it seems there have been some changes in the SciPy hacking guide. For example, in v0.12.0 "python setupegg.py develop" appears (which apparently is exactly what I did) but it is removed in master. It is not recommended anymore? Anyway, I just learned about "build_ext --inplace" and seems like the way to go. Manually creating a symlink in site-packages feels a bit hackish but I can live with it. Maybe this information should have more important and be also present on the NumPy dev guide (or merge / unify both?), other than that and apart from clarifying why "setupegg.py develop" does not appear anymore I cannot think of any further improvements. Thank you very much for your response. I am aware that NumPy and SciPy are currently maintained by a very small group of people whose time is limited too, and since I've been using it extensively the last years I would love to give my two cents in return. Juan Luis Cano From pav at iki.fi Wed Jul 24 11:14:41 2013 From: pav at iki.fi (Pauli Virtanen) Date: Wed, 24 Jul 2013 18:14:41 +0300 Subject: [SciPy-Dev] Version 0.12.1? And some newcomer questions In-Reply-To: <51EFDCCB.1030901@gmail.com> References: <51EF98F3.7070002@gmail.com> <51EFDCCB.1030901@gmail.com> Message-ID: 24.07.2013 16:55, Juan Luis Cano kirjoitti: [clip] > Speaking of which, it seems there have been some changes in the SciPy > hacking guide. For example, in v0.12.0 "python setupegg.py develop" > appears (which apparently is exactly what I did) but it is removed in > master. It is not recommended anymore? The issue with `develop` is that it modifies the default Python environment, and moreover uses the setuptools install mechanism that overrides PYTHONPATH. These can cause annoyance, so I felt it was best to not explicitly recommend it. The in-place build and setting PYTHONPATH provides the same convenience, but does not alter the default Python environment. > Maybe this information should have more important and be also present on > the NumPy dev guide (or merge / unify both?), other than that and apart > from clarifying why "setupegg.py develop" does not appear anymore I > cannot think of any further improvements. Ok, I moved the recommended development setup instructions to a more prominent place in HACKING. Some reorganization with the Numpy dev documentation could be useful. Some streamlining of the latter could also be necessary --- the instructions for how to contribute via patches is probably more hassle for everyone than just directly using Github. -- Pauli Virtanen From suryak at ieee.org Sun Jul 28 15:17:32 2013 From: suryak at ieee.org (Surya Kasturi) Date: Sun, 28 Jul 2013 21:17:32 +0200 Subject: [SciPy-Dev] ipython notebook integration Message-ID: Hi All, Regarding ipython notebook integration in scipy central 1. should we upload ipynb files or provide an editor (or Text Area)? which one would be nice? Thanks Surya -------------- next part -------------- An HTML attachment was scrubbed... URL: From suryak at ieee.org Sun Jul 28 15:37:45 2013 From: suryak at ieee.org (Surya Kasturi) Date: Sun, 28 Jul 2013 21:37:45 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: I just had a quick look into it and made few conclusions.. 1. the "Revision" Model contains "description" field which can be happily used as a place for ipython notebook data!! But my question is do we need extra "description" field for ipython notebook files? In that case we need to create another field!! Thanks On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > Hi All, > > Regarding ipython notebook integration in scipy central > > 1. should we upload ipynb files or provide an editor (or Text Area)? which > one would be nice? > > Thanks > Surya > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Jul 29 15:13:11 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 29 Jul 2013 21:13:11 +0200 Subject: [SciPy-Dev] labeling issues as easy-fix Message-ID: Hi, At https://github.com/scipy/scipy/issues I've just added an easy-fix label, to guide newcomers to issues that don't require too much scipy background or hard debugging. I had planned to do that for a while, and the upcoming EuroSciPy sprint is a good motivation to get it done. Any help would be much appreciated. Maybe module maintainers could go through their module? Adding labels requires commit rights, but anyone without those who wants to help out can either just leave a comment on the ticket or send me the issue ID. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Jul 29 16:57:05 2013 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 29 Jul 2013 22:57:05 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > Hi All, > > Regarding ipython notebook integration in scipy central > > 1. should we upload ipynb files or provide an editor (or Text Area)? which > one would be nice? > Uploading ipynb files and providing a link to view them via http://nbviewer.ipython.org/ would be a good solution. An editor would be a lot of work that's outside the scope of the site I'd think. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jul 29 17:02:34 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 29 Jul 2013 23:02:34 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: Le 29 juil. 2013 ? 22:57, Ralf Gommers a ?crit : > > > > On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > Hi All, > > Regarding ipython notebook integration in scipy central > > 1. should we upload ipynb files or provide an editor (or Text Area)? which one would be nice? > > Uploading ipynb files and providing a link to view them via http://nbviewer.ipython.org/ would be a good solution. An editor would be a lot of work that's outside the scope of the site I'd think. We will support custom css on nbviewer if it interests you. We can also discuss at something like api.nbviewer.ipython.org if you like to embed the html, or have more parametrized requests / custom things. Agree that text editor is too much. As for uploading, would there be a record of different versions ? -- M From suryak at ieee.org Mon Jul 29 17:25:14 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 29 Jul 2013 23:25:14 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: On Mon, Jul 29, 2013 at 10:57 PM, Ralf Gommers wrote: > > > > On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > >> Hi All, >> >> Regarding ipython notebook integration in scipy central >> >> 1. should we upload ipynb files or provide an editor (or Text Area)? >> which one would be nice? >> > > Uploading ipynb files and providing a link to view them via > http://nbviewer.ipython.org/ would be a good solution. An editor would be > a lot of work that's outside the scope of the site I'd think. > > Ralf > > > > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > > In that case, that would be nothing more than a "link" type with some special rendering of URL.. Probably, we also need to think what should be the scope of the site. I didn't pay much attention to this point so far. We want to catalog[1] all scipy- related work or we build a repository[2] of scipy related work.. considering "snippets" submissions, [2] is what we are considering "links" submissions [1] is what we are! Probably, my words are misleading here.. but hope you all got my point! PS: this work is postponed and django upgrade will be continued. However, I am free to hear your suggestions on mailing list. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From suryak at ieee.org Mon Jul 29 17:27:48 2013 From: suryak at ieee.org (Surya Kasturi) Date: Mon, 29 Jul 2013 23:27:48 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: On Mon, Jul 29, 2013 at 11:02 PM, Matthias BUSSONNIER < bussonniermatthias at gmail.com> wrote: > > Le 29 juil. 2013 ? 22:57, Ralf Gommers a ?crit : > > > > > > > > > On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > > Hi All, > > > > Regarding ipython notebook integration in scipy central > > > > 1. should we upload ipynb files or provide an editor (or Text Area)? > which one would be nice? > > > > Uploading ipynb files and providing a link to view them via > http://nbviewer.ipython.org/ would be a good solution. An editor would be > a lot of work that's outside the scope of the site I'd think. > > We will support custom css on nbviewer if it interests you. We can also > discuss at something like api.nbviewer.ipython.org if you like to embed the html, or have more parametrized requests / custom > things. > This sounds quite interesting! but have to see if we require it considering that we are not doing anything more than storing URLs of nbviewer pages > > Agree that text editor is too much. > > As for uploading, would there be a record of different versions ? > Well, we are using Hg for storing other data! hope we can use it for ipynb files too! > -- > M > > _______________________________________________ > SciPy-Dev mailing list > SciPy-Dev at scipy.org > http://mail.scipy.org/mailman/listinfo/scipy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bussonniermatthias at gmail.com Mon Jul 29 17:57:32 2013 From: bussonniermatthias at gmail.com (Matthias BUSSONNIER) Date: Mon, 29 Jul 2013 23:57:32 +0200 Subject: [SciPy-Dev] ipython notebook integration In-Reply-To: References: Message-ID: <59921293-3D43-4BF3-80BF-202A949FBB6A@gmail.com> Le 29 juil. 2013 ? 23:27, Surya Kasturi a ?crit : > > > > On Mon, Jul 29, 2013 at 11:02 PM, Matthias BUSSONNIER wrote: > > Le 29 juil. 2013 ? 22:57, Ralf Gommers a ?crit : > > > > > > > > > On Sun, Jul 28, 2013 at 9:17 PM, Surya Kasturi wrote: > > Hi All, > > > > Regarding ipython notebook integration in scipy central > > > > 1. should we upload ipynb files or provide an editor (or Text Area)? which one would be nice? > > > > Uploading ipynb files and providing a link to view them via http://nbviewer.ipython.org/ would be a good solution. An editor would be a lot of work that's outside the scope of the site I'd think. > > We will support custom css on nbviewer if it interests you. We can also discuss at something like api.nbviewer.ipython.org if > you like to embed the html, or have more parametrized requests / custom things. > > This sounds quite interesting! but have to see if we require it considering that we are not doing anything more than storing URLs of nbviewer pages You might not want all our cell toolbar and a "back" button. I would suggest against storing only nbviewer url only as whats happened most of the time is that origin url move. So lots of nbviewer urls might be broken. Unless you make sure the url point back at a resource hosted by scipy. The advantage of Nbviewer is you are on another subdomain and so have no concern about JS injection. What I think we should do if you want to use nbviewer is in 2 step. Through the (future) api : - you send " i've got this ipynb on scipy at this url. " - i'll send you back : "I can render it at this url" So no concern about url changing etc... Will mot probably not have time to work on it soon, but I'll be happy to polish the detail with you if need later. > Agree that text editor is too much. > > As for uploading, would there be a record of different versions ? > > Well, we are using Hg for storing other data! hope we can use it for ipynb files too! Great ! Just to be sure. -- M