From gaitan at gmail.com  Fri Apr  4 19:55:32 2014
From: gaitan at gmail.com (=?UTF-8?B?TWFydMOtbiBHYWl0w6Fu?=)
Date: Fri, 4 Apr 2014 14:55:32 -0300
Subject: [Cython] Move IPython's cython extension to cython
Message-ID: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>

As you probably know, IPython has a "cython magic" [1], an extension to
interface ipython with cython, allowing to use cython code in the
interactive session.

Originally, few of these "language specific magics" (R, cython, octave,
etc) were included builtin in IPython, but then the IPython core developers
decided break out them (with fair arguments IMHO [2]), either to be
included in the "language package" they depend on or as a third party
extensions.

At the moment,  cython-magic is the only one language specific extension
left to move.
The rest were moved to the packages they depend on [3]

So, this message is to ask if the Cython project would like to import this
code, task that  Brian Grangrer (the author of the cython-magic extension
and IPython core dev)  entrusted me [3].

Of course, I could help you preparing a PR that migrate the module, its
tests and documentation, preserving the git history as far as possible.

let me know your opinion/decision

 cheers
Martin.


[1] http://ipython.org/ipython-doc/dev/config/extensions/cythonmagic.html
[2] https://github.com/ipython/ipython/issues/3150
[3] https://github.com/ipython/ipython/issues/3803
[4] https://github.com/ipython/ipython/issues/3150#issuecomment-37203217


-- 
mgaitan.github.io
textosyprextextos.com.ar



-- 
mgaitan.github.io
textosyprextextos.com.ar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20140404/a1a06a63/attachment.html>

From robertwb at gmail.com  Sun Apr  6 05:37:41 2014
From: robertwb at gmail.com (Robert Bradshaw)
Date: Sat, 5 Apr 2014 20:37:41 -0700
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
Message-ID: <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>

On Fri, Apr 4, 2014 at 10:55 AM, Mart?n Gait?n <gaitan at gmail.com> wrote:
> As you probably know, IPython has a "cython magic" [1], an extension to
> interface ipython with cython, allowing to use cython code in the
> interactive session.
>
> Originally, few of these "language specific magics" (R, cython, octave, etc)
> were included builtin in IPython, but then the IPython core developers
> decided break out them (with fair arguments IMHO [2]), either to be included
> in the "language package" they depend on or as a third party extensions.
>
> At the moment,  cython-magic is the only one language specific extension
> left to move.
> The rest were moved to the packages they depend on [3]
>
> So, this message is to ask if the Cython project would like to import this
> code, task that  Brian Grangrer (the author of the cython-magic extension
> and IPython core dev)  entrusted me [3].
>
> Of course, I could help you preparing a PR that migrate the module, its
> tests and documentation, preserving the git history as far as possible.
>
> let me know your opinion/decision

I'd say this could generally make sense. Would it be the case that iff
cython is installed then the %cython magic would be available and just
work? Also, how testable is this (e.g. via automated tests vs. having
to fire up a notebook and test it manually)?

- Robert

From gaitan at gmail.com  Sun Apr  6 06:24:10 2014
From: gaitan at gmail.com (=?UTF-8?B?TWFydMOtbiBHYWl0w6Fu?=)
Date: Sun, 6 Apr 2014 01:24:10 -0300
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
 <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
Message-ID: <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>

On Sun, Apr 6, 2014 at 12:37 AM, Robert Bradshaw <robertwb at gmail.com> wrote:

> I'd say this could generally make sense. Would it be the case that iff
> cython is installed then the %cython magic would be available and just
> work?
>

Yes, but before to use it, you'll need to load the extension somewhere.
Could be with ipython's %load_ext or setting up it in your config script to
be loaded automatically on the startup.

For example, see how octave magic (included in the package oct2py) works

http://nbviewer.ipython.org/github/blink1073/oct2py/blob/master/example/octavemagic_extension.ipynb?create=1

and here is the code
https://github.com/blink1073/oct2py/tree/master/oct2py/ipython

Also, how testable is this (e.g. via automated tests vs. having
> to fire up a notebook and test it manually)?
>

There is some unittests

https://github.com/ipython/ipython/blob/master/IPython/extensions/tests/test_cythonmagic.py

Of course, they could/should be included in the moving. I could refactor
them to remove Nose as a requirement, but obviously IPython (probably few
versions) would be mandatory to run them.

let me know any other consideration/question.

cheers
Martin

-- 
mgaitan.github.io
textosyprextextos.com.ar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20140406/1ddc104e/attachment.html>

From stefan_ml at behnel.de  Sun Apr  6 09:18:05 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Apr 2014 09:18:05 +0200
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
 <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
 <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>
Message-ID: <5340FFAD.7080407@behnel.de>

Mart?n Gait?n, 06.04.2014 06:24:
> On Sun, Apr 6, 2014 at 12:37 AM, Robert Bradshaw wrote:
>> I'd say this could generally make sense. Would it be the case that iff
>> cython is installed then the %cython magic would be available and just
>> work?
> 
> Yes, but before to use it, you'll need to load the extension somewhere.
> Could be with ipython's %load_ext or setting up it in your config script to
> be loaded automatically on the startup.
> 
> For example, see how octave magic (included in the package oct2py) works
> 
> http://nbviewer.ipython.org/github/blink1073/oct2py/blob/master/example/octavemagic_extension.ipynb?create=1
> 
> and here is the code
> https://github.com/blink1073/oct2py/tree/master/oct2py/ipython

Would it be ok to expose the "load_ipython_extension()" function directly
on the Cython package? Then you'd just say "%load_ext Cython". I suggest to
drop the implementation into Cython/Build/IPythonMagic.py and import it
from there inside of "load_ipython_extension()". There is a "Tests" package
in there as well.

It's also likely that the implementation could benefit from a refactoring
within that package to avoid code duplication.


>> Also, how testable is this (e.g. via automated tests vs. having
>> to fire up a notebook and test it manually)?
> 
> There is some unittests
> 
> https://github.com/ipython/ipython/blob/master/IPython/extensions/tests/test_cythonmagic.py
> 
> Of course, they could/should be included in the moving. I could refactor
> them to remove Nose as a requirement, but obviously IPython (probably few
> versions) would be mandatory to run them.

Yes, getting rid of nosetest looks simple enough. We have our own test
runner in runtests.py that also runs unit tests.

We could install IPython on our CI server to run the tests there, so I
don't see a problem with it (as long as the tests do *not* run if IPython
is not available, e.g. on Travis).

Stefan


From stefan_ml at behnel.de  Sun Apr  6 12:12:19 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Apr 2014 12:12:19 +0200
Subject: [Cython] memoryview slice transpose
In-Reply-To: <53348A20.7020305@duke.edu>
References: <53348A20.7020305@duke.edu>
Message-ID: <53412883.1060806@behnel.de>

Syam Gadde, 27.03.2014 21:29:
> I wanted to re-offer for discussion a fix for a bug I reported a while 
> back.  The code that breaks is this:
> 
> import numpy
> cimport numpy
> 
> def testfunc():
>     a = numpy.arange(12).reshape([3,4])
>     cdef numpy.int_t[:,:] a_view = a
>     cdef numpy.int_t[:,:] b = a_view[:,:].T
> 
> One problem is that when a memoryview slice is decremented, the .memview 
> and .data fields are not set to NULL, and any error that causes a jump 
> to cleanup code will attempt to decref it again.
> 
> The second problem (I think) is that assignment of a transpose of 
> memoryview slices to a variable is not resulting in an incref on the 
> underlying slice, even though the variable is not a temporary and 
> continues to persist.
> 
> Here is a patch that solves the problem for me.
> 
> https://github.com/SyamGadde/cython/commit/5739d8b908f18c4fc9103ef04e39964d61af3495

Hmm, I can't see why the additional NULL setting would change anything.
That's what the call to put_xdecref_memoryviewslice() already did that
immediately precedes them (it actually calls "__Pyx_XDEC_MEMVIEW()" in
MemoryView_C.c).


> The part I am least sure about is removing the "if" qualifications on 
> the incref and decref:
> 
>    if self.obj.is_name  or self.obj.is_attribute and self.obj.is_memslice_transpose:
> 
> that was obviously there for a reason, but it causes problems in the above case.  If there is a less extreme solution I'd be happy to take it.

It's more likely that this was the change that made the difference. I can't
say why the original code behaved that way, but it may well be that it
misbehaves in an error case. Essentially, it doesn't incref the slice in
certain cases (as an optimisation?), and if an error occurs, it might still
end up trying to decref it. I think it's worth investigating that a bit
further.

Stefan


From gaitan at gmail.com  Sun Apr  6 18:26:51 2014
From: gaitan at gmail.com (=?UTF-8?B?TWFydMOtbiBHYWl0w6Fu?=)
Date: Sun, 6 Apr 2014 13:26:51 -0300
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <5340FFAD.7080407@behnel.de>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
 <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
 <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>
 <5340FFAD.7080407@behnel.de>
Message-ID: <CAOCZAuw_ku3RXEzSgn4RFqWvi_4Vna-15VuZHbkdYuuXX-Kv_w@mail.gmail.com>

On Sun, Apr 6, 2014 at 4:18 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Would it be ok to expose the "load_ipython_extension()" function directly
> on the Cython package? Then you'd just say "%load_ext Cython". I suggest
> to  drop the implementation into Cython/Build/IPythonMagic.py and import it
> from there inside of "load_ipython_extension()". There is a "Tests" package
> in there as well.
>

Ok, it sounds very good. Simpler for users with low "pollution" for cython.


> Yes, getting rid of nosetest looks simple enough. We have our own test
> runner in runtests.py that also runs unit tests.
>
> We could install IPython on our CI server to run the tests there, so I
> don't see a problem with it (as long as the tests do *not* run if IPython
> is not available, e.g. on Travis).
>

Great. So, should I start the pull request and continue the discussion in
the context of the code? There is something I must to read before
"contribute to cython" ?

Thanks a lot for your answers
Martin

-- 
mgaitan.github.io
textosyprextextos.com.ar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20140406/7c351e7a/attachment.html>

From syam.gadde at duke.edu  Mon Apr  7 21:38:49 2014
From: syam.gadde at duke.edu (Syam Gadde)
Date: Mon, 7 Apr 2014 19:38:49 +0000
Subject: [Cython] memoryview slice transpose
In-Reply-To: <53412883.1060806@behnel.de>
References: <53348A20.7020305@duke.edu> <53412883.1060806@behnel.de>
Message-ID: <5342FEC9.4010101@duke.edu>

My patch actually addressed two slightly different issues, both of which 
have to do with refcounts of transposed slices.  I'll address the first 
one below since that's the one that's not so obvious.

Here's the code that causes an "Acquisition count is 0" error:

import numpy
cimport numpy

def testfunc():
    a = numpy.arange(12).reshape([3,4])
    cdef numpy.int_t[:,:] a_view = a
    cdef numpy.int_t[:,:] b = a_view[:,:].T
    raise Exception()  # comment this out and you still get a refcount 
error at a different line

testfunc()


On 04/06/2014 06:12 AM, Stefan Behnel wrote:
> Syam Gadde, 27.03.2014 21:29:
>> One problem is that when a memoryview slice is decremented, the .memview
>> and .data fields are not set to NULL, and any error that causes a jump
>> to cleanup code will attempt to decref it again.
> Hmm, I can't see why the additional NULL setting would change anything.
> That's what the call to put_xdecref_memoryviewslice() already did that
> immediately precedes them (it actually calls "__Pyx_XDEC_MEMVIEW()" in
> MemoryView_C.c).

But only if the refcount of the memview goes to zero.Maybe that's what 
needs to change?

See the case below (look for the *****'s).  memviewslice objects are 
defined at the top of the function and on the stack, so they persist 
throughout the function.  Several memviewslice objects may share the 
same base memview object.  So if a memviewslice object finished doing 
what it needs to do, it needs to decrement the refcount of the memview 
object it's pointing to using __PYX_XDEC_MEMVIEW, *and* detach itself 
from the memview (that's the setting .memview = NULL part).  The reason 
the explicit detaching is important is that there are extra 
__PYX_XDEC_MEMVIEW() calls in the error cleanup code, presumably to 
decrement the reference counts for still attached memviews in the case 
there is an error/exception.  If the memview field is set to NULL 
whenever a slice is done with it, the extra calls to __PYX_XDEC_MEMVIEW 
are OK because it checks to see whether .memview is NULL before going 
forward.  But currently, if an error/exception occurs while the slice is 
still attached to the memview, you can end up with a refcount < 0.

If __PYX_XDEC_MEMVIEW did indeed set .memview to NULL on every call, 
then this wouldn't be necessary.  But I have not tried that to see if it 
breaks something else.


   /* "testtranspose.pyx":7
  *    a = numpy.arange(12).reshape([3,4])
  *    cdef numpy.int_t[:,:] a_view = a
  *    cdef numpy.int_t[:,:] b = a_view[:,:].T             # <<<<<<<<<<<<<<
  *    raise Exception()
  *
  */
   __pyx_t_6 = -1;
   __pyx_t_5.data = __pyx_v_a_view.data;
   __pyx_t_5.memview = __pyx_v_a_view.memview;
   __PYX_INC_MEMVIEW(&__pyx_t_5, 0);
   __pyx_t_5.shape[0] = __pyx_v_a_view.shape[0];
__pyx_t_5.strides[0] = __pyx_v_a_view.strides[0];
     __pyx_t_5.suboffsets[0] = -1;

__pyx_t_5.shape[1] = __pyx_v_a_view.shape[1];
__pyx_t_5.strides[1] = __pyx_v_a_view.strides[1];
     __pyx_t_5.suboffsets[1] = -1;

__pyx_t_7 = __pyx_t_5;
   if (unlikely(__pyx_memslice_transpose(&__pyx_t_7) == 0)) 
{__pyx_filename = __p
yx_f[0]; __pyx_lineno = 7; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
   __PYX_XDEC_MEMVIEW(&__pyx_t_5, 1);//**************************
   __pyx_v_b = __pyx_t_7;
   __pyx_t_7.memview = NULL;
   __pyx_t_7.data = NULL;

   /* "testtranspose.pyx":8
  *    cdef numpy.int_t[:,:] a_view = a
  *    cdef numpy.int_t[:,:] b = a_view[:,:].T
  *    raise Exception()             # <<<<<<<<<<<<<<
  *
  * testfunc()
  */
   __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_Exception, 
__pyx_empty_tuple, NULL); if (unlikely(!__pyx_t_1)) {__pyx_filename = 
__pyx_f[0]; __pyx_lineno = 8; __pyx_clineno = __LINE__; goto 
__pyx_L1_error;}
   __Pyx_GOTREF(__pyx_t_1);
   __Pyx_Raise(__pyx_t_1, 0, 0, 0);
   __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
   {__pyx_filename = __pyx_f[0]; __pyx_lineno = 8; __pyx_clineno = 
__LINE__; goto __pyx_L1_error;}

   /* "testtranspose.pyx":4
  * cimport numpy
  *
  * def testfunc():             # <<<<<<<<<<<<<<
  *    a = numpy.arange(12).reshape([3,4])
  *    cdef numpy.int_t[:,:] a_view = a
  */

   /* function exit code */
   __pyx_L1_error:;  // ****** exception causes a jump to here *****
   __Pyx_XDECREF(__pyx_t_1);
   __Pyx_XDECREF(__pyx_t_2);
   __Pyx_XDECREF(__pyx_t_3);
   __PYX_XDEC_MEMVIEW(&__pyx_t_4, 1);
   __PYX_XDEC_MEMVIEW(&__pyx_t_5, 1); //****************************
   __PYX_XDEC_MEMVIEW(&__pyx_t_7, 1);
   __Pyx_AddTraceback("testtranspose.testfunc", __pyx_clineno, 
__pyx_lineno, __pyx_filename);
   __pyx_r = NULL;
   __Pyx_XDECREF(__pyx_v_a);
   __PYX_XDEC_MEMVIEW(&__pyx_v_a_view, 1);
   __PYX_XDEC_MEMVIEW(&__pyx_v_b, 1);
   __Pyx_XGIVEREF(__pyx_r);
   __Pyx_RefNannyFinishContext();
   return __pyx_r;

From gaitan at gmail.com  Mon Apr 14 00:06:29 2014
From: gaitan at gmail.com (=?UTF-8?B?TWFydMOtbiBHYWl0w6Fu?=)
Date: Sun, 13 Apr 2014 19:06:29 -0300
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <CAOCZAuw_ku3RXEzSgn4RFqWvi_4Vna-15VuZHbkdYuuXX-Kv_w@mail.gmail.com>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
 <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
 <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>
 <5340FFAD.7080407@behnel.de>
 <CAOCZAuw_ku3RXEzSgn4RFqWvi_4Vna-15VuZHbkdYuuXX-Kv_w@mail.gmail.com>
Message-ID: <CAOCZAuz3XQDfiK6H2JMQagb1gq+JmCYoZ2HdT=16Hy-vN2sUVw@mail.gmail.com>

On Sun, Apr 6, 2014 at 1:26 PM, Mart?n Gait?n <gaitan at gmail.com> wrote:
> So, should I start the pull request and continue the discussion in the
> context of the code?

Hi: ping.

should I?

-- 
mgaitan.github.io
textosyprextextos.com.ar

From stefan_ml at behnel.de  Mon Apr 14 08:09:03 2014
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 14 Apr 2014 08:09:03 +0200
Subject: [Cython] Move IPython's cython extension to cython
In-Reply-To: <CAOCZAuz3XQDfiK6H2JMQagb1gq+JmCYoZ2HdT=16Hy-vN2sUVw@mail.gmail.com>
References: <CAOCZAuzaoXOWO3yyRTov8ERpVuXykrw4+fHF8D-HV7-0+v2BWw@mail.gmail.com>
 <CADiQ+QAZ8xfVqnZ9X2NhuQ2MvPjMFk6+PkV-isVnD3DAQt6yGQ@mail.gmail.com>
 <CAOCZAuzXqcsL5CTSjmDFVqyX_g8AYyKftwhiT71PWO-Pj=jznA@mail.gmail.com>
 <5340FFAD.7080407@behnel.de>
 <CAOCZAuw_ku3RXEzSgn4RFqWvi_4Vna-15VuZHbkdYuuXX-Kv_w@mail.gmail.com>
 <CAOCZAuz3XQDfiK6H2JMQagb1gq+JmCYoZ2HdT=16Hy-vN2sUVw@mail.gmail.com>
Message-ID: <534B7B7F.5080704@behnel.de>

Mart?n Gait?n, 14.04.2014 00:06:
> On Sun, Apr 6, 2014 at 1:26 PM, Mart?n Gait?n wrote:
>> So, should I start the pull request and continue the discussion in the
>> context of the code?
> 
> Hi: ping.
> 
> should I?

Sorry, please do!

Current master is Py2.6+, so feel free to depend on that.

Stefan


From dalcinl at gmail.com  Mon Apr 21 13:53:27 2014
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 21 Apr 2014 14:53:27 +0300
Subject: [Cython] MPI_Type_create_hindexed_block() segfaults
Message-ID: <CAEcYPwCoTq41Tv9Peeb2vzYYnfCMYf=bNr5tk9TrfYpJZqFkZw@mail.gmail.com>

I believe the problem is in the following source code line
(file:ompi_datatype_args.c, line:221):

https://bitbucket.org/ompiteam/ompi-svn-mirror/src/v1.8/ompi/datatype/ompi_datatype_args.c?at=v1.8#cl-221

I think you should just remove that bogus line, and that's all.


[dalcinl at kw2060 openmpi]$ cat type_hindexed_block.c
#include <mpi.h>
int main(int argc, char *argv[])
{
  MPI_Aint disps[] = {0};
  MPI_Datatype datatype;
  MPI_Init(&argc, &argv);
  MPI_Type_create_hindexed_block(1, 1, disps, MPI_BYTE, &datatype);
  MPI_Finalize();
  return 0;
}
[dalcinl at kw2060 openmpi]$ mpicc type_hindexed_block.c
[dalcinl at kw2060 openmpi]$ ./a.out
[kw2060:20304] *** Process received signal ***
[kw2060:20304] Signal: Segmentation fault (11)
[kw2060:20304] Signal code: Address not mapped (1)
[kw2060:20304] Failing at address: 0x6a
[kw2060:20304] [ 0] /lib64/libpthread.so.0[0x327c40f750]
[kw2060:20304] [ 1] /lib64/libc.so.6[0x327bc94126]
[kw2060:20304] [ 2]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(ompi_datatype_set_args+0x7f1)[0x7f8f0158b62a]
[kw2060:20304] [ 3]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(MPI_Type_create_hindexed_block+0x24d)[0x7f8f015cedc8]
[kw2060:20304] [ 4] ./a.out[0x40080c]
[kw2060:20304] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x327bc21d65]
[kw2060:20304] [ 6] ./a.out[0x4006f9]
[kw2060:20304] *** End of error message ***
Segmentation fault (core dumped)


-- 
Lisandro Dalcin
---------------
CIMEC (UNL/CONICET)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1016)
Tel/Fax: +54-342-4511169

From dalcinl at gmail.com  Mon Apr 21 13:55:19 2014
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 21 Apr 2014 14:55:19 +0300
Subject: [Cython] MPI_Type_create_hindexed_block() segfaults
In-Reply-To: <CAEcYPwCoTq41Tv9Peeb2vzYYnfCMYf=bNr5tk9TrfYpJZqFkZw@mail.gmail.com>
References: <CAEcYPwCoTq41Tv9Peeb2vzYYnfCMYf=bNr5tk9TrfYpJZqFkZw@mail.gmail.com>
Message-ID: <CAEcYPwAoVQMSEtaXUKaROZ7v7no+VhcVNkArmvofw+W1qh1snQ@mail.gmail.com>

I'm very sorry for this bogus email, please ignore.

On 21 April 2014 14:53, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> I believe the problem is in the following source code line
> (file:ompi_datatype_args.c, line:221):
>
> https://bitbucket.org/ompiteam/ompi-svn-mirror/src/v1.8/ompi/datatype/ompi_datatype_args.c?at=v1.8#cl-221
>
> I think you should just remove that bogus line, and that's all.
>
>
> [dalcinl at kw2060 openmpi]$ cat type_hindexed_block.c
> #include <mpi.h>
> int main(int argc, char *argv[])
> {
>   MPI_Aint disps[] = {0};
>   MPI_Datatype datatype;
>   MPI_Init(&argc, &argv);
>   MPI_Type_create_hindexed_block(1, 1, disps, MPI_BYTE, &datatype);
>   MPI_Finalize();
>   return 0;
> }
> [dalcinl at kw2060 openmpi]$ mpicc type_hindexed_block.c
> [dalcinl at kw2060 openmpi]$ ./a.out
> [kw2060:20304] *** Process received signal ***
> [kw2060:20304] Signal: Segmentation fault (11)
> [kw2060:20304] Signal code: Address not mapped (1)
> [kw2060:20304] Failing at address: 0x6a
> [kw2060:20304] [ 0] /lib64/libpthread.so.0[0x327c40f750]
> [kw2060:20304] [ 1] /lib64/libc.so.6[0x327bc94126]
> [kw2060:20304] [ 2]
> /home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(ompi_datatype_set_args+0x7f1)[0x7f8f0158b62a]
> [kw2060:20304] [ 3]
> /home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(MPI_Type_create_hindexed_block+0x24d)[0x7f8f015cedc8]
> [kw2060:20304] [ 4] ./a.out[0x40080c]
> [kw2060:20304] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x327bc21d65]
> [kw2060:20304] [ 6] ./a.out[0x4006f9]
> [kw2060:20304] *** End of error message ***
> Segmentation fault (core dumped)
>
>
> --
> Lisandro Dalcin
> ---------------
> CIMEC (UNL/CONICET)
> Predio CONICET-Santa Fe
> Colectora RN 168 Km 472, Paraje El Pozo
> 3000 Santa Fe, Argentina
> Tel: +54-342-4511594 (ext 1016)
> Tel/Fax: +54-342-4511169



-- 
Lisandro Dalcin
---------------
CIMEC (UNL/CONICET)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1016)
Tel/Fax: +54-342-4511169

From redbrain at gcc.gnu.org  Wed Apr 30 17:54:46 2014
From: redbrain at gcc.gnu.org (Philip Herron)
Date: Wed, 30 Apr 2014 16:54:46 +0100
Subject: [Cython] GCC Pxd
Message-ID: <CAEvRbermtDgYKcUYMsjwuU91i6GCiMEHPDR_X=MPnYuEYxhpgg@mail.gmail.com>

Hey all,

Some of you may remember a project i worked on for GSoC 2012, i wasn't
really happy with the project over all for a long time it basicaly
required at the time people to be running GCC from latest SVN/GIT for
it to work correctly.

Now gcc >= 4.7 is becoming more normal, so i went back into it and
rewrote it from scratch now and its shaping up to look fairly decent
and usable now.

https://github.com/redbrain/cython/tree/master/Tools/gccpxd

So the idea for this project is that you write a small configuration
file and it will integrate with your C/C++ build system for example
you could do:

./configure CC=cygcc ...
make
...

And you will get a project.pxd file after the build of your software.
the cygcc is simply a wrapper over GCC that shoves in the fplugin
arguments so you can run:

cygcc -g -O2 -Wall -c test.c -o test.o

So build systems wont really notice anything different it should just
think its gcc. The configuration file example in the folder .cygcc:

$ cat .cygcc
header = 'test.h'             # the header to get declaratiosn for
inputs = ['test.h', 'test.c'] # any locations of declarations that
would be involved
output = 'test.pxd'          # the output

The cygcc compiler looks for this in the current director or looks up
at the parent directory untill it finds it or it will fail with an
error message.

It will only ouput once if the test.pxd file exists in the current
working directory it wont do anything.

$ cat test.pxd
cdef extern from "test.h":
  int test (int, int)
  int global_variable
  int static_variable # this is a bug but hey...
  struct bla:
    int a
    float b

So far for the example code in there its working pretty well the gcc
python plugin has stabalized alot and so has some of gcc's bits a
bobs. I will try and keep you all updated i think the plugin is doing
alot more work now so this doesnt have to do so much.

Thanks to Robert Bradshaw for baring with me in the whole thing
because i probably wasn't the easiest to work with ;) lol.

Thanks everyone cython is awesome.

--Phil

From ulf.worsoe at mosek.com  Wed Apr 30 12:45:50 2014
From: ulf.worsoe at mosek.com (Ulf Worsoe)
Date: Wed, 30 Apr 2014 12:45:50 +0200
Subject: [Cython] Bug in C code generation
Message-ID: <CA+Qkgy-hru4GAAdrVB9hXMQFgiag5pKwYViBo5wWy3Y-oPmg5w@mail.gmail.com>

Hi,

I have encounterered a bug in the following situation with Cython 0.20.1:

cdef funname():
  cdef numpy.ndarray[numpy.int32_t,ndim=1] myvar
  if True:
    ... block that never initializes myvar
  else
    ... block that initializes myvar

The resulting C code does not declare the variable for the buffer object of
myvar (my guess is that cython detects that it is never used and drops it),
but the cleanup part of the C function contains code that frees the buffer,
and thus refers to a variable that was never declared.

I have not posted the complete code since the .pyx is around 50k lines
long, but the offending method looks like this, and the problematic
variable is _3_perm in line 4:

---------------
  cdef _toStringArray__3JJ_3S(self,numpy.ndarray[numpy.int64_t,ndim=1]
_0_subi,numpy.int64_t _1_dstidx,numpy.ndarray[object,ndim=1] _2_result):
    assert _0_subi is None or isinstance(_0_subi,numpy.ndarray)
    assert _2_result is None or isinstance(_2_result,numpy.ndarray)
    cdef numpy.ndarray[numpy.int64_t,ndim=1] _3_perm
    cdef numpy.float64_t _4_c
    cdef numpy.int32_t _5_b
    cdef numpy.int32_t _6_ni
    cdef numpy.int64_t _7_i
    cdef numpy.int32_t _8_p
    cdef mosek_fusion_Utils_StringBuffer _9_sb
    for _10_i in xrange(0,(_0_subi).shape[0]):
      if ((_0_subi[_10_i] < 0) or (_0_subi[_10_i] >= (self.shape.size))):
        raise mosek_fusion_IndexError_ctor_S("Constraint index out of
bounds")
    if (((_0_subi).shape[0] + _1_dstidx) > (_2_result).shape[0]):
      raise mosek_fusion_LengthError_ctor_S("Result array is too small to
hold the result")
    _9_sb=mosek.fusion.Utils.StringBuffer()
    _8_p=0
    if True:
      for _11_k in xrange(0,(_0_subi).shape[0]):
        _7_i=_0_subi[_11_k]
        _6_ni=self.nativeindexes[_7_i]
        _5_b=_8_p
        while ((_8_p < ((self.cache.subi)).shape[0]) and
((self.cache.subi)[_8_p] == _7_i)):
          _8_p += 1
        _9_sb._clear_()._a_S(self.name)._a_S("[")._a_S(self.shape.indexToString(_7_i))._a_S("]
: ")
        for _12_j in xrange(_5_b,_8_p):
          _4_c=(self.cache.cof)[_12_j]
          if (_4_c > 0):
            _9_sb._a_S(" + ")._a_D(_4_c)._a_S(" ")
          elif (_4_c < 0):
            _9_sb._a_S(" - ")._a_D((- _4_c))._a_S(" ")

self.model._nativeVarToStr_ILmosek_4fusion_4Utils_4StringBuffer_2((self.cache.subj)[_12_j],_9_sb)
        if (self.cache_bfix != None):
          if (self.cache_bfix[_7_i] > 0):
            _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i])
          elif (self.cache_bfix[_7_i] < 0):
            _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i])
        _9_sb._a_S(" ")

self._domainToString_JLmosek_4fusion_4Utils_4StringBuffer_2(_7_i,_9_sb)
        _2_result[(numpy.int64(_11_k) + _1_dstidx)] = _9_sb._toString_()
    else:
      _3_perm=numpy.zeros(((_0_subi).shape[0],),
dtype=numpy.dtype(numpy.int64))

mosek_fusion_CommonTools_argQsort__3J_3J_3JJJ(_3_perm,_0_subi,None,0l,(_0_subi).shape[0])
---------------


-- 
Ulf Wors?e
Mosek ApS

ulf.worsoe at mosek.com
www.mosek.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cython-devel/attachments/20140430/fcdb79dd/attachment.html>