From eliswilson at hushmail.com Wed May 1 00:06:53 2013 From: eliswilson at hushmail.com (eliswilson at hushmail.com) Date: Tue, 30 Apr 2013 18:06:53 -0400 Subject: [Cython] Biggest Fake Conference in Computer Science Message-ID: <20130430220653.D791114DBE2@smtp.hushmail.com> Biggest Fake Conference in Computer Science We are researchers from different parts of the world and conducted a study on the world?s biggest bogus computer science conference WORLDCOMP http://sites.google.com/site/worlddump1 organized by Prof. Hamid Arabnia from University of Georgia, USA. We submitted a fake paper to WORLDCOMP 2011 and again (the same paper with a modified title) to WORLDCOMP 2012. This paper had numerous fundamental mistakes. Sample statements from that paper include: (1). Binary logic is fuzzy logic and vice versa (2). Pascal developed fuzzy logic (3). Object oriented languages do not exhibit any polymorphism or inheritance (4). TCP and IP are synonyms and are part of OSI model (5). Distributed systems deal with only one computer (6). Laptop is an example for a super computer (7). Operating system is an example for computer hardware Also, our paper did not express any conceptual meaning. However, it was accepted both the times without any modifications (and without any reviews) and we were invited to submit the final paper and a payment of $500+ fee to present the paper. We decided to use the fee for better purposes than making Prof. Hamid Arabnia richer. After that, we received few reminders from WORLDCOMP to pay the fee but we never responded. This fake paper is different from the two fake papers already published (see https://sites.google.com/site/worlddump4 for details) in WORLDCOMP. We MUST say that you should look at the above website if you have any thoughts of participating in WORLDCOMP. DBLP and other indexing agencies have stopped indexing WORLDCOMP?s proceedings since 2011 due to its fakeness. See http://www.informatik.uni-trier.de/~ley/db/conf/icai/index.html for of one of the conferences of WORLDCOMP and notice that there is no listing after 2010. See Section 2 of http://sites.google.com/site/dumpconf for comments from well-known researchers about WORLDCOMP. The status of your WORLDCOMP papers can be changed from scientific to other (i.e., junk or non-technical) at any time. Better not to have a paper than having it in WORLDCOMP and spoil the resume and peace of mind forever! Our study revealed that WORLDCOMP is money making business, using University of Georgia mask, for Prof. Hamid Arabnia. He is throwing out a small chunk of that money (around 20 dollars per paper published in WORLDCOMP?s proceedings) to his puppet (Mr. Ashu Solo or A.M.G. Solo) who publicizes WORLDCOMP and also defends it at various forums, using fake/anonymous names. The puppet uses fake names and defames other conferences to divert traffic to WORLDCOMP. He also makes anonymous phone calls and threatens the critiques of WORLDCOMP (See Item 7 of Section 5 of above website). That is, the puppet does all his best to get a maximum number of papers published at WORLDCOMP to get more money into his (and Prof. Hamid Arabnia?s) pockets. Prof. Hamid Arabnia makes a lot of tricks. For example, he appeared in a newspaper to fool the public, claiming him a victim of cyber-attack (see Item 8 in Section 5 of above website). Monte Carlo Resort (the venue of WORLDCOMP for more than 10 years, until 2012) has refused to provide the venue for WORLDCOMP?13 because of the fears of their image being tarnished due to WORLDCOMP?s fraudulent activities. That is why WORLDCOMP?13 is taking place at a different resort. WORLDCOMP will not be held after 2013. The draft paper submission deadline is over but still there are no committee members, no reviewers, and there is no conference Chairman. The only contact details available on WORLDCOMP?s website is just an email address! We ask Prof. Hamid Arabnia to publish all reviews for all the papers (after blocking identifiable details) since 2000 conference. Reveal the names and affiliations of all the reviewers (for each year) and how many papers each reviewer had reviewed on average. We also ask him to look at the Open Challenge (Section 6) at https://sites.google.com/site/moneycomp1 and respond if he has any professional values. Sorry for posting to multiple lists. Spreading the word is the only way to stop this bogus conference. Please forward this message to other mailing lists and people. We are shocked with Prof. Hamid Arabnia and his puppet?s activities at http://worldcomp-fake-bogus.blogspot.com Search Google using the keyword worldcomp fake for additional links. From dalcinl at gmail.com Wed May 1 09:14:53 2013 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Wed, 1 May 2013 10:14:53 +0300 Subject: [Cython] Regresion in 0.19 Message-ID: See the example below, the last hunk fails with Cython 0.19. It seems Cython is confusing attribute access with constness. CYTHON=cython echo 'cdef import from *:' > defs.pxd echo ' enum:SIZE' >> defs.pxd echo 'from defs cimport *' > code1.pyx echo 'cdef char buf[SIZE]' >> code1.pyx $CYTHON code1.pyx echo 'cimport defs' > code2.pyx echo 'cdef char buf[defs.SIZE]' >> code2.pyx $CYTHON code2.pyx -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From qvcolonel at gmail.com Wed May 1 09:37:19 2013 From: qvcolonel at gmail.com (=?UTF-8?B?5bGI6bmP?=) Date: Wed, 1 May 2013 15:37:19 +0800 Subject: [Cython] Regresion in 0.19 In-Reply-To: References: Message-ID: Why is cython so slow? 2013/5/1 Lisandro Dalcin > See the example below, the last hunk fails with Cython 0.19. It seems > Cython is confusing attribute access with constness. > > > CYTHON=cython > > echo 'cdef import from *:' > defs.pxd > echo ' enum:SIZE' >> defs.pxd > > echo 'from defs cimport *' > code1.pyx > echo 'cdef char buf[SIZE]' >> code1.pyx > $CYTHON code1.pyx > > echo 'cimport defs' > code2.pyx > echo 'cdef char buf[defs.SIZE]' >> code2.pyx > $CYTHON code2.pyx > > > -- > Lisandro Dalcin > --------------- > CIMEC (INTEC/CONICET-UNL) > Predio CONICET-Santa Fe > Colectora RN 168 Km 472, Paraje El Pozo > 3000 Santa Fe, Argentina > Tel: +54-342-4511594 (ext 1011) > Tel/Fax: +54-342-4511169 > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > -- while(sleeping){ ++money; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Wed May 1 09:39:42 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 09:39:42 +0200 Subject: [Cython] Regresion in 0.19 In-Reply-To: References: Message-ID: <5180C6BE.5080707@behnel.de> ??, 01.05.2013 09:37: > Why is cython so slow? This looks like thread-hijacking to me. Maybe you want to elaborate on this in a new post to the cython-users mailing list. Stefan From stefan_ml at behnel.de Wed May 1 09:45:23 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 09:45:23 +0200 Subject: [Cython] Regresion in 0.19 In-Reply-To: References: Message-ID: <5180C813.2050408@behnel.de> Lisandro Dalcin, 01.05.2013 09:14: > See the example below, the last hunk fails with Cython 0.19. It seems > Cython is confusing attribute access with constness. > > > CYTHON=cython > > echo 'cdef import from *:' > defs.pxd > echo ' enum:SIZE' >> defs.pxd > > echo 'from defs cimport *' > code1.pyx > echo 'cdef char buf[SIZE]' >> code1.pyx > $CYTHON code1.pyx > > echo 'cimport defs' > code2.pyx > echo 'cdef char buf[defs.SIZE]' >> code2.pyx > $CYTHON code2.pyx With the above setup, I get Cython compile errors in 0.19: Error compiling Cython file: ------------------------------------------------------------ ... cimport defs cdef char buf[defs.SIZE] ^ ------------------------------------------------------------ code2.pyx:2:18: Not allowed in a constant expression Interesting enough, this happens during declaration analysis, not during type analysis. Looks like that's needed because the array size is part of the declaration. ISTM that the implementation of analyse_const_expression() is wrong. It doesn't care about the result of analyse_types(). Stefan From stefan_ml at behnel.de Wed May 1 10:28:09 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 10:28:09 +0200 Subject: [Cython] Regresion in 0.19 In-Reply-To: <5180C813.2050408@behnel.de> References: <5180C813.2050408@behnel.de> Message-ID: <5180D219.50808@behnel.de> Stefan Behnel, 01.05.2013 09:45: > Lisandro Dalcin, 01.05.2013 09:14: >> See the example below, the last hunk fails with Cython 0.19. It seems >> Cython is confusing attribute access with constness. >> >> >> CYTHON=cython >> >> echo 'cdef import from *:' > defs.pxd >> echo ' enum:SIZE' >> defs.pxd >> >> echo 'from defs cimport *' > code1.pyx >> echo 'cdef char buf[SIZE]' >> code1.pyx >> $CYTHON code1.pyx >> >> echo 'cimport defs' > code2.pyx >> echo 'cdef char buf[defs.SIZE]' >> code2.pyx >> $CYTHON code2.pyx > > With the above setup, I get Cython compile errors in 0.19: > > Error compiling Cython file: > ------------------------------------------------------------ > ... > cimport defs > cdef char buf[defs.SIZE] > ^ > ------------------------------------------------------------ > code2.pyx:2:18: Not allowed in a constant expression > > Interesting enough, this happens during declaration analysis, not during > type analysis. Looks like that's needed because the array size is part of > the declaration. > > ISTM that the implementation of analyse_const_expression() is wrong. It > doesn't care about the result of analyse_types(). Yep, an oversight of my analyse_types() refactoring and cleanup. https://github.com/scoder/cython/commit/db1f0ad5a5ac45d2a1da45211f9d344b4d83cab8 I'll give it a run through Jenkins before pushing it upstream. Stefan From stefan_ml at behnel.de Wed May 1 11:50:15 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 11:50:15 +0200 Subject: [Cython] Autodoc improvements (#216) In-Reply-To: References: <5171A4E9.10700@behnel.de> <51725160.6020008@behnel.de> <51727F04.9080307@behnel.de> Message-ID: <5180E557.4050205@behnel.de> Nikita Nemkin, 26.04.2013 13:33: > On Tue, 23 Apr 2013 10:19:15 +0600, Robert Bradshaw wrote: > >> Jumping into this thread late, improvements (1) and (3) certainly seem >> beneficial. As far as documenting "attributes," I can't see much of a >> use, but the downsides seem are low enough (accidentally assigning a >> stray string to the previously declared attribute) and the >> implementation non-invasive enough that I'm +0.5 for this change. It >> doesn't introduce new syntax or behavior, just additional semantic >> meaning for those (otherwise unlikely to occur) strings and sure beats >> writing them out as properties (though the fact that cdef attributes >> are properties should be considered more of an implementation detail >> IMHO). > > I've just found a 4 year old ticket requesting the same feature (2): > http://trac.cython.org/cython_trac/ticket/206 Just because there's a ticket doesn't mean that it should be fixed, but I'll give in on this one. Nikita, if you add the test that proves that newlines prevent a subsequent string from becoming a docstring, i.e. that cdef attr "this is not a docstring" behaves as expected and is different from cdef attr "this is a docstring" I'll merge your pull request. Stefan From stefan_ml at behnel.de Wed May 1 11:53:37 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 11:53:37 +0200 Subject: [Cython] Autodoc improvements (#216) In-Reply-To: <5180E557.4050205@behnel.de> References: <5171A4E9.10700@behnel.de> <51725160.6020008@behnel.de> <51727F04.9080307@behnel.de> <5180E557.4050205@behnel.de> Message-ID: <5180E621.1010505@behnel.de> Stefan Behnel, 01.05.2013 11:50: > Nikita Nemkin, 26.04.2013 13:33: >> On Tue, 23 Apr 2013 10:19:15 +0600, Robert Bradshaw wrote: >> >>> Jumping into this thread late, improvements (1) and (3) certainly seem >>> beneficial. As far as documenting "attributes," I can't see much of a >>> use, but the downsides seem are low enough (accidentally assigning a >>> stray string to the previously declared attribute) and the >>> implementation non-invasive enough that I'm +0.5 for this change. It >>> doesn't introduce new syntax or behavior, just additional semantic >>> meaning for those (otherwise unlikely to occur) strings and sure beats >>> writing them out as properties (though the fact that cdef attributes >>> are properties should be considered more of an implementation detail >>> IMHO). >> >> I've just found a 4 year old ticket requesting the same feature (2): >> http://trac.cython.org/cython_trac/ticket/206 > > Just because there's a ticket doesn't mean that it should be fixed, but > I'll give in on this one. Nikita, if you add the test that proves that > newlines prevent a subsequent string from becoming a docstring, i.e. that > > cdef attr > > "this is not a docstring" > > behaves as expected and is different from > > cdef attr > "this is a docstring" > > I'll merge your pull request. ... and I just noticed that you already did that, and also removed the warning about stray docstrings. Sorry for that. I'll merge it in. Stefan From stefan_ml at behnel.de Wed May 1 19:57:58 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 01 May 2013 19:57:58 +0200 Subject: [Cython] [cython-users] Bug in 0.19 (with example) In-Reply-To: References: Message-ID: <518157A6.9090007@behnel.de> Eric Frederich, 01.05.2013 18:59: > I have discovered a bug in Cython 0.19 > > This code works in 0.15.1 but not in 0.19. > I haven't tested it anywhere in between. > > The following 3 files are listed at the end of this email. > blah.h > blah.pxd > test.pyx > > The errors I get are the following: > test.pyx:3:18: Not allowed in a constant expression > test.pyx:3:18: Array dimension not integer > test.pyx:3:20: Variable type 'char []' is incomplete > > The line is: > cdef char[blah.FOO] x > > If I do: > cdef char[int(blah.FOO)] x > ... I get the following error: Thanks for the report. Looks like a known bug, however, already reported and fixed: http://mail.python.org/pipermail/cython-devel/2013-May/003631.html Stefan From eric.frederich at gmail.com Wed May 1 21:05:50 2013 From: eric.frederich at gmail.com (Eric Frederich) Date: Wed, 1 May 2013 15:05:50 -0400 Subject: [Cython] [cython-users] Bug in 0.19 (with example) In-Reply-To: <518157A6.9090007@behnel.de> References: <518157A6.9090007@behnel.de> Message-ID: Thanks, Glad to see it was caught. ... and glad that there is a workaround for me at the moment with the +0 ~Eric On Wed, May 1, 2013 at 1:57 PM, Stefan Behnel wrote: > Eric Frederich, 01.05.2013 18:59: >> I have discovered a bug in Cython 0.19 >> >> This code works in 0.15.1 but not in 0.19. >> I haven't tested it anywhere in between. >> >> The following 3 files are listed at the end of this email. >> blah.h >> blah.pxd >> test.pyx >> >> The errors I get are the following: >> test.pyx:3:18: Not allowed in a constant expression >> test.pyx:3:18: Array dimension not integer >> test.pyx:3:20: Variable type 'char []' is incomplete >> >> The line is: >> cdef char[blah.FOO] x >> >> If I do: >> cdef char[int(blah.FOO)] x >> ... I get the following error: > > Thanks for the report. Looks like a known bug, however, already reported > and fixed: > > http://mail.python.org/pipermail/cython-devel/2013-May/003631.html > > Stefan > > -- > > --- > You received this message because you are subscribed to the Google Groups "cython-users" group. > To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > From stefan_ml at behnel.de Fri May 3 09:29:18 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 03 May 2013 09:29:18 +0200 Subject: [Cython] any more fixes for 0.19.1 ? Message-ID: <5183674E.1060601@behnel.de> Hi, since there's an unpleasant regression in 0.19, I'd like to release 0.19.1 soon. If you have any more fixes that should go in, you may want to apply them now or at least speak up to delay the release. I'm aware of ticket 810, but it doesn't seem like a regression and I don't currently see an obvious simple way to reproduce it. It looks like a classical "last reference dies while still in use" kind of problem to me. Maybe even a bug in Sage rather than Cython that just accidentally worked in older releases, although I'm not saying that there is nothing Cython could do about it. http://trac.cython.org/cython_trac/ticket/810 Ticket 814 (compiler crash during C++ code analysis) would also be worth closing before a release as it might be an easy fix, even though it's not a regression (crashes for me also in 0.18). http://trac.cython.org/cython_trac/ticket/814 I also think that Ticket 775 ("memoryview" name leaks into module namespace) should eventually be fixed as it's annoying and unexpected, although not necessarily right in this release (it's been like this forever). http://trac.cython.org/cython_trac/ticket/775 Anything else? Stefan From stefan_ml at behnel.de Fri May 3 15:49:19 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 03 May 2013 15:49:19 +0200 Subject: [Cython] any more fixes for 0.19.1 ? In-Reply-To: <5183674E.1060601@behnel.de> References: <5183674E.1060601@behnel.de> Message-ID: <5183C05F.8020108@behnel.de> Stefan Behnel, 03.05.2013 09:29: > since there's an unpleasant regression in 0.19, I'd like to release 0.19.1 > soon. If you have any more fixes that should go in, you may want to apply > them now or at least speak up to delay the release. > > I'm aware of ticket 810, but it doesn't seem like a regression and I don't > currently see an obvious simple way to reproduce it. It looks like a > classical "last reference dies while still in use" kind of problem to me. > Maybe even a bug in Sage rather than Cython that just accidentally worked > in older releases, although I'm not saying that there is nothing Cython > could do about it. > > http://trac.cython.org/cython_trac/ticket/810 > > Ticket 814 (compiler crash during C++ code analysis) would also be worth > closing before a release as it might be an easy fix, even though it's not a > regression (crashes for me also in 0.18). > > http://trac.cython.org/cython_trac/ticket/814 > > I also think that Ticket 775 ("memoryview" name leaks into module > namespace) should eventually be fixed as it's annoying and unexpected, > although not necessarily right in this release (it's been like this forever). > > http://trac.cython.org/cython_trac/ticket/775 > > Anything else? There's also the C++ default arguments bug: http://thread.gmane.org/gmane.comp.python.cython.user/1379 I think we should at least emit a warning for now. Stefan From nikita at nemkin.ru Sat May 4 16:28:13 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Sat, 04 May 2013 20:28:13 +0600 Subject: [Cython] any more fixes for 0.19.1 ? In-Reply-To: <5183C05F.8020108@behnel.de> References: <5183674E.1060601@behnel.de> <5183C05F.8020108@behnel.de> Message-ID: On Fri, 03 May 2013 19:49:19 +0600, Stefan Behnel wrote: > I also think that Ticket 775 ("memoryview" name leaks into module > namespace) should eventually be fixed as it's annoying and unexpected, > although not necessarily right in this release (it's been like this > forever). > > http://trac.cython.org/cython_trac/ticket/775 I'm working on this one and I need an opinion: should utility classes be made internal by default or should they use @cython.internal explicitly? Best regards, Nikita Nemkin From stefan_ml at behnel.de Sat May 4 19:21:26 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 04 May 2013 19:21:26 +0200 Subject: [Cython] any more fixes for 0.19.1 ? In-Reply-To: References: <5183674E.1060601@behnel.de> <5183C05F.8020108@behnel.de> Message-ID: <51854396.3020800@behnel.de> Nikita Nemkin, 04.05.2013 16:28: > On Fri, 03 May 2013 19:49:19 +0600, Stefan Behnel wrote: > >> I also think that Ticket 775 ("memoryview" name leaks into module >> namespace) should eventually be fixed as it's annoying and unexpected, >> although not necessarily right in this release (it's been like this >> forever). >> >> http://trac.cython.org/cython_trac/ticket/775 > > I'm working on this one and I need an opinion: should utility > classes be made internal by default or should they use > @cython.internal explicitly? I'm not sure what the exact problem is here, but in general, I'd say that they should be declared in the CythonScope only, and not leak into the module scope at all. Mark would know more details on this one. Stefan From stefan_ml at behnel.de Sun May 5 13:03:54 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 05 May 2013 13:03:54 +0200 Subject: [Cython] [cython] Hide Cython utility classes (like memoryview) from Python level module scope. (#222) In-Reply-To: References: Message-ID: <51863C9A.5040704@behnel.de> Nikita Nemkin, 04.05.2013 19:52: > Two changes included: > > 1) cdef classes in utility code can have compiler directives attached to them. This is not used anywhere ATM, but memoryviews may benefit from ``@cython.final``. > > 2) All utility classes are excluded from module dictionary by *implicitly* marking them with ``@cython.internal`` . This fixes [#775](http://trac.cython.org/cython_trac/ticket/775), test is included. > > I don't quite understand what CythonScope is and how utility classes are *supposed* to be hidden, but as it is now, utility code scope is merged into main module scope and there is nothing special about its classes. > > BTW if a user declares his own class with the same name as utility class (for example, ``memoryview``), everything breaks down. > You can merge this Pull Request by running: > > git pull https://github.com/nnemkin/cython cy_utility_decorators > > Or you can view, comment on it, or merge it online at: > > https://github.com/cython/cython/pull/222 I wonder why utility classes actually need a Python name. Can't they just live with None as "name"? All they should really need is their cname and a properly analysed entry stored in the right places, so deleting their Python visible name when merging them into the main module should fix this. Stefan From nikita at nemkin.ru Sun May 5 14:26:28 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Sun, 05 May 2013 18:26:28 +0600 Subject: [Cython] [cython] Hide Cython utility classes (like memoryview) from Python level module scope. (#222) In-Reply-To: <51863C9A.5040704@behnel.de> References: <51863C9A.5040704@behnel.de> Message-ID: On Sun, 05 May 2013 17:03:54 +0600, Stefan Behnel wrote: > Nikita Nemkin, 04.05.2013 19:52: >> Two changes included: >> >> 1) cdef classes in utility code can have compiler directives attached >> to them. This is not used anywhere ATM, but memoryviews may benefit >> from ``@cython.final``. >> >> 2) All utility classes are excluded from module dictionary by >> *implicitly* marking them with ``@cython.internal`` . This fixes >> [#775](http://trac.cython.org/cython_trac/ticket/775), test is included. >> >> I don't quite understand what CythonScope is and how utility >> classes are *supposed* to be hidden, but as it is now, utility code >> scope is merged into main module scope and there is nothing special >> about its classes. >> >> BTW if a user declares his own class with the same name as utility >> class (for example, ``memoryview``), everything breaks down. > > I wonder why utility classes actually need a Python name. Can't they just > live with None as "name"? All they should really need is their cname and > a > properly analysed entry stored in the right places, so deleting their > Python visible name when merging them into the main module should fix > this. entry.name serves for general identification and bookkeeping, not just to provide a python level name. Non-null entry name is a very useful invariant, I'd rather not break it for something trivial like name hiding. All codegen algorithms will have to worry about (class) entries with null names afterwards. Even if it works currently, it may break in the future. Anyway, just setting entry.name to None does not work, because it is not the only place to get a python name (and of course it's never checked for None). For example, module init code uses ClassScope.class_name. Some other code may use entry.type.name etc... Best regards, Nikita Nemkin From stefan_ml at behnel.de Sun May 5 14:40:40 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 05 May 2013 14:40:40 +0200 Subject: [Cython] [cython] Hide Cython utility classes (like memoryview) from Python level module scope. (#222) In-Reply-To: References: <51863C9A.5040704@behnel.de> Message-ID: <51865348.9010707@behnel.de> Nikita Nemkin, 05.05.2013 14:26: > On Sun, 05 May 2013 17:03:54 +0600, Stefan Behnel wrote: > >> Nikita Nemkin, 04.05.2013 19:52: >>> Two changes included: >>> >>> 1) cdef classes in utility code can have compiler directives attached to >>> them. This is not used anywhere ATM, but memoryviews may benefit from >>> ``@cython.final``. >>> >>> 2) All utility classes are excluded from module dictionary by >>> *implicitly* marking them with ``@cython.internal`` . This fixes >>> [#775](http://trac.cython.org/cython_trac/ticket/775), test is included. >>> >>> I don't quite understand what CythonScope is and how utility classes >>> are *supposed* to be hidden, but as it is now, utility code scope is >>> merged into main module scope and there is nothing special about its >>> classes. >>> >>> BTW if a user declares his own class with the same name as utility class >>> (for example, ``memoryview``), everything breaks down. >> >> I wonder why utility classes actually need a Python name. Can't they just >> live with None as "name"? All they should really need is their cname and a >> properly analysed entry stored in the right places, so deleting their >> Python visible name when merging them into the main module should fix this. > > entry.name serves for general identification and bookkeeping, not just > to provide a python level name. Non-null entry name is a very useful > invariant, I'd rather not break it for something trivial like name hiding. > > All codegen algorithms will have to worry about (class) entries with null > names afterwards. Even if it works currently, it may break in the future. > > Anyway, just setting entry.name to None does not work, because it is not > the only place to get a python name (and of course it's never checked for > None). > For example, module init code uses ClassScope.class_name. Some other code > may use entry.type.name etc... Ah, yes. It's way more complex for classes than for, say, lambda functions (and even those have a fake name). Anyway, given how easy it is to declare utility code @internal to get this fixed, I think it's not worth looking for a different solution. I merged your pull request. Stefan From stefan_ml at behnel.de Mon May 6 21:26:08 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 06 May 2013 21:26:08 +0200 Subject: [Cython] [cython] Order cdef classes by inheritance before generating the code. (#223) In-Reply-To: References: Message-ID: <518803D0.6070908@behnel.de> Nikita Nemkin, 05.05.2013 16:13: > cdef classes must be (topologically) ordered by inheritance to satisfy the following dependencies: > 1) dealloc function definition: derived classes call base class dealloc; > 2) vtable pointer initialization in module init function. > Wrong order causes C/C++ compilation failure for 1) and segfault on import for 2). > > Normally, source order is the correct order, but forward declarations can change this. See included test case for example. > I hit this problem because I forward declare all my classes in alphabetic order (and then include class definitions from .pxi files, one class per file). > > The simple (although not obvious) workaround is to reorder forward delarations, but I believe it's worth fixing permanently. > You can merge this Pull Request by running: > > git pull https://github.com/nnemkin/cython cdef_class_order > > Or you can view, comment on it, or merge it online at: > > https://github.com/cython/cython/pull/223 Luckily, this broke the Sage build, by running into an infinite loop while sorting. https://sage.math.washington.edu:8091/hudson/job/sage-build/1778/ We clearly need more tests for extension type hierarchies spread across multiple modules. Stefan From hroest_nospam2333 at quantentunnel.de Tue May 7 22:38:38 2013 From: hroest_nospam2333 at quantentunnel.de (=?UTF-8?B?SGFubmVzIFLDtnN0?=) Date: Tue, 7 May 2013 22:38:38 +0200 Subject: [Cython] Generated C++ code incompatible with Clang In-Reply-To: References: Message-ID: Hi Robert After some internal discussion, we have not found an obvious way to not have the extra typedef. Is this a big problem? Maybe there are other people on the list that are more fluent in C++ and could think of a way to do it? Greetings Hannes On 30 April 2013 07:48, Robert Bradshaw wrote: > On Fri, Apr 12, 2013 at 8:04 AM, Hannes R?st > wrote: > > Dear mailing list > > > > In the process of developing a rather large Cython project where we wrap > the > > scientific OpenMS library (http://sourceforge.net/projects/open-ms/) > using > > Cython, we have come across an issue with the C++ code that is generated > by > > Cython. The issue is that Cython generates C++ that is not compatible > with the > > Clang compiler. > > > > I hope this is the correct mailing list to report this issue since it > seems to > > be a bug in Cython and not in Clang (after some discussion we have > concluded > > that Clang is most likely correct in rejecting the generated code). > Since we do > > not have access to the issue tracker, we hoped posting our issue here > would > > clear up matters. > > > > The issue occurs with the following minimal testcase: > > > > from libcpp.vector cimport vector as libcpp_vector > > from cython.operator cimport dereference as deref, preincrement as inc > > > > cdef class TestClass: > > > > cdef libcpp_vector[float] inst > > > > def __iter__(self): > > it = self.inst.begin() > > while it != self.inst.end(): > > yield deref(it) > > inc(it) > > > > When compiled with Cython to C++, it generates C++ that cannot be > compiled with > > Clang (however it seems that gcc and MSVS accept the code). It seems > that the > > same issue with Clang was already discussed here: > > https://bugzilla.mozilla.org/show_bug.cgi?id=623303 and the conclusion > was that > > Clang is correct in reporting the code as erroneous. > > > > In the above case, the invalid C++ code that gets generated is: > > > > p->__pyx_v_it.std::vector::iterator::~iterator(); > > > > The correct code for the above case would probably be (this is just a > > suggestion, it compiles with gcc and clang on our machines and our tests > run > > through with it): > > > > typedef std::vector::iterator _it; > > p->__pyx_v_it.~_it(); > > > > > > We have tested it with the 0.18 release as well as the newest 0.19b2 > build from > > github (58131b68dc033fc7ca269d875a2aab2b4e9646a2) and the results were > the > > same. > > Thanks for the report. Can you think of a solution that doesn't > require a special typedef declaration for each type you might want to > call a destructor on? > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertwb at gmail.com Tue May 7 22:55:28 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Tue, 7 May 2013 13:55:28 -0700 Subject: [Cython] Generated C++ code incompatible with Clang In-Reply-To: References: Message-ID: It's not a big problem, but it'll be an ugly and invasive amount of bookkeeping to manage these top-level typedefs per type deletion. On Tue, May 7, 2013 at 1:38 PM, Hannes R?st wrote: > Hi Robert > > After some internal discussion, we have not found an obvious way to not have > the extra typedef. Is this a big problem? > > Maybe there are other people on the list that are more fluent in C++ and > could think of a way to do it? > > Greetings > > Hannes > > On 30 April 2013 07:48, Robert Bradshaw wrote: >> >> On Fri, Apr 12, 2013 at 8:04 AM, Hannes R?st >> wrote: >> > Dear mailing list >> > >> > In the process of developing a rather large Cython project where we wrap >> > the >> > scientific OpenMS library (http://sourceforge.net/projects/open-ms/) >> > using >> > Cython, we have come across an issue with the C++ code that is generated >> > by >> > Cython. The issue is that Cython generates C++ that is not compatible >> > with the >> > Clang compiler. >> > >> > I hope this is the correct mailing list to report this issue since it >> > seems to >> > be a bug in Cython and not in Clang (after some discussion we have >> > concluded >> > that Clang is most likely correct in rejecting the generated code). >> > Since we do >> > not have access to the issue tracker, we hoped posting our issue here >> > would >> > clear up matters. >> > >> > The issue occurs with the following minimal testcase: >> > >> > from libcpp.vector cimport vector as libcpp_vector >> > from cython.operator cimport dereference as deref, preincrement as inc >> > >> > cdef class TestClass: >> > >> > cdef libcpp_vector[float] inst >> > >> > def __iter__(self): >> > it = self.inst.begin() >> > while it != self.inst.end(): >> > yield deref(it) >> > inc(it) >> > >> > When compiled with Cython to C++, it generates C++ that cannot be >> > compiled with >> > Clang (however it seems that gcc and MSVS accept the code). It seems >> > that the >> > same issue with Clang was already discussed here: >> > https://bugzilla.mozilla.org/show_bug.cgi?id=623303 and the conclusion >> > was that >> > Clang is correct in reporting the code as erroneous. >> > >> > In the above case, the invalid C++ code that gets generated is: >> > >> > p->__pyx_v_it.std::vector::iterator::~iterator(); >> > >> > The correct code for the above case would probably be (this is just a >> > suggestion, it compiles with gcc and clang on our machines and our tests >> > run >> > through with it): >> > >> > typedef std::vector::iterator _it; >> > p->__pyx_v_it.~_it(); >> > >> > >> > We have tested it with the 0.18 release as well as the newest 0.19b2 >> > build from >> > github (58131b68dc033fc7ca269d875a2aab2b4e9646a2) and the results were >> > the >> > same. >> >> Thanks for the report. Can you think of a solution that doesn't >> require a special typedef declaration for each type you might want to >> call a destructor on? >> _______________________________________________ >> cython-devel mailing list >> cython-devel at python.org >> http://mail.python.org/mailman/listinfo/cython-devel > > > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > From stefan_ml at behnel.de Wed May 8 16:25:24 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 08 May 2013 16:25:24 +0200 Subject: [Cython] Cython 0.19.1 release candidate Message-ID: <518A6054.4010606@behnel.de> Hi, here's a release candidate for 0.19.1. I accept feedback for the next two days, then I'll send the final release out. http://cython.org/release/Cython-0.19.1rc1.tar.gz The changelog is on github: https://github.com/cython/cython/blob/1491df80144a13ac4c6d8027283f7619a716fb58/CHANGES.rst Stefan From ryan at essential-elements.net Wed May 8 20:09:14 2013 From: ryan at essential-elements.net (Ryan Pessa) Date: Wed, 8 May 2013 13:09:14 -0500 Subject: [Cython] nogil doesn't seem to work when defined on extern cpp functions Message-ID: Hello, I ran into an interesting problem today. It doesn't seem like Cython respects the `nogil` statement on extern cpp functions. I am trying to use a blocking I/O function, and am running it in secondary thread so I can use another library function to cancel it. I have tried it both on the `extern` line: cdef extern from "digitalpersona/dpfp_api.h" nogil: uint32_t DPFPGetEvent(dp_uid_t* pDevUID, dp_device_event_t** ppEvent, uint32_t uTimeoutMsec) and on the function itself: cdef extern from "digitalpersona/dpfp_api.h" nogil: uint32_t DPFPGetEvent(dp_uid_t* pDevUID, dp_device_event_t** ppEvent, uint32_t uTimeoutMsec) nogil Either way, this statement still blocks other threads: res = dpfp_api.DPFPGetEvent(pIdDev, &pEvent, dpfp_api.DP_TIMEOUT_INFINITE) While this statement does not: with nogil: res = dpfp_api.DPFPGetEvent(pIdDev, &pEvent, dpfp_api.DP_TIMEOUT_INFINITE) All variables used (res, pIdDev, pEvent) are C variables. Obviously this is not a huge issue, as I can bypass the GIL using `with nogil`. But I figured it would be a good idea to report it anyway. Ryan Pessa AerisPOS Project Manager / Developer Essential Elements | AerisPOS -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan_ml at behnel.de Wed May 8 20:43:00 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 08 May 2013 20:43:00 +0200 Subject: [Cython] project proposal: line tracing/profiling, coverage, etc. Message-ID: <518A9CB4.5050403@behnel.de> Hi, in the past, Robert and I have invested some work into supporting profiling and tracing in Cython, and I would like to see this finalised. While Python level tracing and profiling might not seem like the ideal tools for performance critical Cython code, they are actually pretty handy when much of your code is Python code and you want to do, say, coverage analysis over your whole code base, without having to exclude the Cython parts. So I've opened a ticket that highlights the missing bits. http://trac.cython.org/cython_trac/ticket/815 If anyone's interested in pursuing this, I'd be glad to help getting you on the right track. I don't even think there's all that much left to do, but it's still a bit of cleanup and maybe some naughty little details along the way. Would make a good entry project, IMHO. Anyone interested? Stefan From stefan_ml at behnel.de Wed May 8 20:48:29 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 08 May 2013 20:48:29 +0200 Subject: [Cython] nogil doesn't seem to work when defined on extern cpp functions In-Reply-To: References: Message-ID: <518A9DFD.10600@behnel.de> Hi, this question is better suited for the cython-users mailing list. Ryan Pessa, 08.05.2013 20:09: > I ran into an interesting problem today. It doesn't seem like Cython > respects the `nogil` statement on extern cpp functions. I am trying to use > a blocking I/O function, and am running it in secondary thread so I can use > another library function to cancel it. > > I have tried it both on the `extern` line: > cdef extern from "digitalpersona/dpfp_api.h" nogil: > uint32_t DPFPGetEvent(dp_uid_t* pDevUID, dp_device_event_t** > ppEvent, uint32_t uTimeoutMsec) > and on the function itself: > cdef extern from "digitalpersona/dpfp_api.h" nogil: > uint32_t DPFPGetEvent(dp_uid_t* pDevUID, dp_device_event_t** > ppEvent, uint32_t uTimeoutMsec) nogil > > Either way, this statement still blocks other threads: > res = dpfp_api.DPFPGetEvent(pIdDev, &pEvent, > dpfp_api.DP_TIMEOUT_INFINITE) > While this statement does not: > with nogil: > res = dpfp_api.DPFPGetEvent(pIdDev, &pEvent, > dpfp_api.DP_TIMEOUT_INFINITE) > All variables used (res, pIdDev, pEvent) are C variables. > > Obviously this is not a huge issue, as I can bypass the GIL using `with > nogil`. But I figured it would be a good idea to report it anyway. Declarations won't implicitly free the GIL for you. They just state that the function doesn't need the GIL and can thus be called safely from within a "nogil" block. Freeing the GIL is an explicit operation that is left to the user. Otherwise, you'd end up with ambiguous code semantics. Imaging you'd call two functions in a row that are both declared nogil - should Cython lock the GIL in between or not? Stefan From stefan_ml at behnel.de Wed May 8 20:58:12 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 08 May 2013 20:58:12 +0200 Subject: [Cython] project proposal: line tracing/profiling, coverage, etc. In-Reply-To: <518A9CB4.5050403@behnel.de> References: <518A9CB4.5050403@behnel.de> Message-ID: <518AA044.5080509@behnel.de> Stefan Behnel, 08.05.2013 20:43: > in the past, Robert and I have invested some work into supporting profiling > and tracing in Cython, and I would like to see this finalised. While Python > level tracing and profiling might not seem like the ideal tools for > performance critical Cython code, they are actually pretty handy when much > of your code is Python code and you want to do, say, coverage analysis over > your whole code base, without having to exclude the Cython parts. > > So I've opened a ticket that highlights the missing bits. > > http://trac.cython.org/cython_trac/ticket/815 > > If anyone's interested in pursuing this, I'd be glad to help getting you on > the right track. I don't even think there's all that much left to do, but > it's still a bit of cleanup and maybe some naughty little details along the > way. Would make a good entry project, IMHO. And maybe I should mention that there's a second side to this: with another bit of work to improve the configuration of code objects, Cython implemented Python functions could become even more Python compatible by allowing for better introspection, e.g. of function signatures and their parameters. A PEP 362 implementation (the __signature__ API) doesn't seem all that far either. http://www.python.org/dev/peps/pep-0362/ > Anyone interested? Stefan From robertwb at gmail.com Wed May 8 21:00:29 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Wed, 8 May 2013 12:00:29 -0700 Subject: [Cython] Cython 0.19.1 release candidate In-Reply-To: <518A6054.4010606@behnel.de> References: <518A6054.4010606@behnel.de> Message-ID: It'd be nice to pull in https://github.com/cython/cython/commit/5574592e569e0cce5f1277b6f0c441d6d19122b5 as well. On Wed, May 8, 2013 at 7:25 AM, Stefan Behnel wrote: > Hi, > > here's a release candidate for 0.19.1. I accept feedback for the next two > days, then I'll send the final release out. > > http://cython.org/release/Cython-0.19.1rc1.tar.gz > > The changelog is on github: > > https://github.com/cython/cython/blob/1491df80144a13ac4c6d8027283f7619a716fb58/CHANGES.rst > > > Stefan > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel From stefan_ml at behnel.de Wed May 8 21:12:28 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 08 May 2013 21:12:28 +0200 Subject: [Cython] Cython 0.19.1 release candidate In-Reply-To: References: <518A6054.4010606@behnel.de> Message-ID: <518AA39C.8020808@behnel.de> Robert Bradshaw, 08.05.2013 21:00: > It'd be nice to pull in > https://github.com/cython/cython/commit/5574592e569e0cce5f1277b6f0c441d6d19122b5 > as well. Done. Stefan From greg.ewing at canterbury.ac.nz Thu May 9 00:56:08 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 09 May 2013 10:56:08 +1200 Subject: [Cython] nogil doesn't seem to work when defined on extern cpp functions In-Reply-To: References: Message-ID: <518AD808.9070601@canterbury.ac.nz> Ryan Pessa wrote: > It doesn't seem like Cython > respects the `nogil` statement on extern cpp functions. The nogil declaration doesn't work that way. Declaring a function as nogil just says that it's safe to call it without the GIL -- it doesn't actually cause the GIL to be released. You need a 'with nogil' block to do that. -- Greg From stefan_ml at behnel.de Thu May 9 06:05:00 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 09 May 2013 06:05:00 +0200 Subject: [Cython] [cython-users] mimetype and icon for Cython programs? In-Reply-To: References: <518A5DDF.80409@fedoraproject.org> Message-ID: <518B206C.7040005@behnel.de> Shriramana Sharma, 09.05.2013 05:17: > On Thu, May 9, 2013 at 5:30 AM, Robert Bradshaw wrote: >> Feel free to use >> the log at http://en.wikipedia.org/wiki/File:Cython-logo.svg (with our >> without the trailing ython). > > For proper transparency I removed the white rectangle used to make O > look like C: > > http://en.wikipedia.org/wiki/File:Cython-logo.svg Just to keep things official and to avoid the possibility of legal conflicts, I uploaded my original version here: https://de.wikipedia.org/wiki/Datei:Cython-logo.svg Apparently, I cannot overwrite the english version file. If someone else can, please replace it. > But since this apparently isn't under a free licence, it can't be part > of the official KDE distro. Is there a way out? Not sure how KDE handles this, but it also uses the official Python logo for .py files. So I can't imagine it being a problem. > I have submitted https://bugs.kde.org/show_bug.cgi?id=319557 to > include this into KDE. Please use the official version instead then. > This still leaves us with the question as to where we are supposed to > register this mimetype officially if necessary. IANA, I guess: https://www.iana.org/assignments/media-types Not sure it's worth it, though. Stefan From stefan_ml at behnel.de Sat May 11 11:01:02 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 11 May 2013 11:01:02 +0200 Subject: [Cython] Cython 0.19.1 released Message-ID: <518E08CE.7050706@behnel.de> Hi all, Cython 0.19.1 is following the course of the recent 0.19 release. This is mainly a bug fix release that adds only minor new features and fixes some bugs and regressions in the latest release. Upgrading is generally recommended. You can get it from here: http://cython.org/release/Cython-0.19.1.tar.gz http://cython.org/release/Cython-0.19.1.zip Release notes: https://github.com/cython/cython/blob/8f07f8d8265799cada5046fe0b7c0807d7b41fb3/CHANGES.rst Documentation: http://docs.cython.org/ Major changes include: * A warning is emitted when negative literal indices are found inside of a code section that disables ``wraparound`` handling. This helps with fixing invalid code that might fail in the face of future compiler optimisations. * A regression in 0.19 was fixed that rejected valid C expressions from being used in C array size declarations. * Testing large (> int) C integer values for their truth value could fail due to integer wrap-around. * The automatic signature documentation tries to preserve more semantics of default arguments and argument types. Specifically, ``bint`` arguments now appear as type ``bool``. Have fun, Stefan From robertwb at gmail.com Fri May 17 07:03:54 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Thu, 16 May 2013 22:03:54 -0700 Subject: [Cython] [cython-users] Re: Cython Typed Memoryviews versus Cython+Numpy Speed ups In-Reply-To: <517111BB.8060009@behnel.de> References: <8804baca-9772-4805-a6e5-922aa03d4443@googlegroups.com> <517111BB.8060009@behnel.de> Message-ID: On Fri, Apr 19, 2013 at 2:43 AM, Stefan Behnel wrote: > Robert Bradshaw, 17.04.2013 06:02: >> On Tue, Apr 16, 2013 at 6:46 PM, pythonOmetrist >>> Attached HTML seems mostly white, and not clear where to go from here. Nogil >>> seems to be one option. >> >> Just a brief glance at the code, you have a lot of n_k_w[x][y] which >> is vastly more expensive than n_k_w[x, y]. > > Is that always equivalent? If so, we might want to transform it internally. I think so, but I'm not sure. If so, +1 to making this a transformation. From vitja.makarov at gmail.com Tue May 21 12:26:35 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 21 May 2013 14:26:35 +0400 Subject: [Cython] CF based type inference Message-ID: Hi! Recently I've started work on new type inference engine. Now it's almost ready and I want to discuss it. It works like this: first infer type for each assignment then for whole entry. It has some advantages over previous algorithm: - it handles assignment cycles, see test_swap() for example - it can infer type using info about assignments on the whole entry: a = 1 b = a a = "str" a is python object and b is integer. Here are testcases that show some new cases it can solve: https://github.com/vitek/cython /blob/_type_inference_new/tests/run/type_inference_new.pyx Here is branch for it https://github.com/vitek/cython /tree/_type_inference_new -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From markflorisson88 at gmail.com Tue May 21 13:39:06 2013 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 21 May 2013 12:39:06 +0100 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: On 21 May 2013 11:26, Vitja Makarov wrote: > Hi! > > Recently I've started work on new type inference engine. Now it's almost > ready and I want to discuss it. > > It works like this: first infer type for each assignment then for whole > entry. It has some advantages over previous algorithm: > - it handles assignment cycles, see test_swap() for example > - it can infer type using info about assignments on the whole entry: > > a = 1 > b = a > a = "str" > > a is python object and b is integer. > > Here are testcases that show some new cases it can solve: > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx > > Here is branch for it > https://github.com/vitek/cython/tree/_type_inference_new > > -- > vitja. > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > Hey Vitja, Cool! How do you want to handle variable merge at control flow joins? Would you promote (backwards incompatible), use a union type, or object? E.g. what does this result in: x = 0 for i in range(N): x += 0.2 * i print typeof(x) From vitja.makarov at gmail.com Tue May 21 15:14:48 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 21 May 2013 17:14:48 +0400 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: 2013/5/21 mark florisson > On 21 May 2013 11:26, Vitja Makarov wrote: > > Hi! > > > > Recently I've started work on new type inference engine. Now it's almost > > ready and I want to discuss it. > > > > It works like this: first infer type for each assignment then for whole > > entry. It has some advantages over previous algorithm: > > - it handles assignment cycles, see test_swap() for example > > - it can infer type using info about assignments on the whole entry: > > > > a = 1 > > b = a > > a = "str" > > > > a is python object and b is integer. > > > > Here are testcases that show some new cases it can solve: > > > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx > > > > Here is branch for it > > https://github.com/vitek/cython/tree/_type_inference_new > > > > -- > > vitja. > > > > _______________________________________________ > > cython-devel mailing list > > cython-devel at python.org > > http://mail.python.org/mailman/listinfo/cython-devel > > > > Hey Vitja, > > Cool! How do you want to handle variable merge at control flow joins? > Would you promote (backwards incompatible), use a union type, or > object? E.g. what does this result in: > > x = 0 > for i in range(N): > x += 0.2 * i > print typeof(x) Hi Mark, Nothing changed in your example old algorithm was able to handle this "simple" kind of cycles as well as new one: def foo(int N): x = 0 for i in range(N): x += 0.2 * i print typeof(x) So this function (not that N is an integer here) will print 'dobule'. With new algorithm we can go further: def foo(int N): x = 1 y = 0 for i in range(N): x = x * 0.1 + y * 0.2 y = x * 0.3 + y * 0.4 print typeof(x), typeof(y) Here both x and y will be inferred as double -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From markflorisson88 at gmail.com Tue May 21 15:32:16 2013 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 21 May 2013 14:32:16 +0100 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: On 21 May 2013 14:14, Vitja Makarov wrote: > > > > 2013/5/21 mark florisson >> >> On 21 May 2013 11:26, Vitja Makarov wrote: >> > Hi! >> > >> > Recently I've started work on new type inference engine. Now it's almost >> > ready and I want to discuss it. >> > >> > It works like this: first infer type for each assignment then for whole >> > entry. It has some advantages over previous algorithm: >> > - it handles assignment cycles, see test_swap() for example >> > - it can infer type using info about assignments on the whole entry: >> > >> > a = 1 >> > b = a >> > a = "str" >> > >> > a is python object and b is integer. >> > >> > Here are testcases that show some new cases it can solve: >> > >> > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx >> > >> > Here is branch for it >> > https://github.com/vitek/cython/tree/_type_inference_new >> > >> > -- >> > vitja. >> > >> > _______________________________________________ >> > cython-devel mailing list >> > cython-devel at python.org >> > http://mail.python.org/mailman/listinfo/cython-devel >> > >> >> Hey Vitja, >> >> Cool! How do you want to handle variable merge at control flow joins? >> Would you promote (backwards incompatible), use a union type, or >> object? E.g. what does this result in: >> >> x = 0 >> for i in range(N): >> x += 0.2 * i >> print typeof(x) > > > Hi Mark, > > Nothing changed in your example old algorithm was able to handle this > "simple" kind of cycles as well as new one: > > def foo(int N): > x = 0 > for i in range(N): > x += 0.2 * i > print typeof(x) > > So this function (not that N is an integer here) will print 'dobule'. > > With new algorithm we can go further: > > def foo(int N): > x = 1 > y = 0 > for i in range(N): > x = x * 0.1 + y * 0.2 > y = x * 0.3 + y * 0.4 > print typeof(x), typeof(y) > > Here both x and y will be inferred as double > Ok, so I assume it promotes the incoming types (all reaching definitions)? If N == 0, then when using objects you get an int, otherwise a double. What happens when the reaching types cannot be promoted together? Do you fall back to object? > -- > vitja. > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > From vitja.makarov at gmail.com Tue May 21 15:41:46 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 21 May 2013 17:41:46 +0400 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: 2013/5/21 mark florisson > On 21 May 2013 14:14, Vitja Makarov wrote: > > > > > > > > 2013/5/21 mark florisson > >> > >> On 21 May 2013 11:26, Vitja Makarov wrote: > >> > Hi! > >> > > >> > Recently I've started work on new type inference engine. Now it's > almost > >> > ready and I want to discuss it. > >> > > >> > It works like this: first infer type for each assignment then for > whole > >> > entry. It has some advantages over previous algorithm: > >> > - it handles assignment cycles, see test_swap() for example > >> > - it can infer type using info about assignments on the whole entry: > >> > > >> > a = 1 > >> > b = a > >> > a = "str" > >> > > >> > a is python object and b is integer. > >> > > >> > Here are testcases that show some new cases it can solve: > >> > > >> > > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx > >> > > >> > Here is branch for it > >> > https://github.com/vitek/cython/tree/_type_inference_new > >> > > >> > -- > >> > vitja. > >> > > >> > _______________________________________________ > >> > cython-devel mailing list > >> > cython-devel at python.org > >> > http://mail.python.org/mailman/listinfo/cython-devel > >> > > >> > >> Hey Vitja, > >> > >> Cool! How do you want to handle variable merge at control flow joins? > >> Would you promote (backwards incompatible), use a union type, or > >> object? E.g. what does this result in: > >> > >> x = 0 > >> for i in range(N): > >> x += 0.2 * i > >> print typeof(x) > > > > > > Hi Mark, > > > > Nothing changed in your example old algorithm was able to handle this > > "simple" kind of cycles as well as new one: > > > > def foo(int N): > > x = 0 > > for i in range(N): > > x += 0.2 * i > > print typeof(x) > > > > So this function (not that N is an integer here) will print 'dobule'. > > > > With new algorithm we can go further: > > > > def foo(int N): > > x = 1 > > y = 0 > > for i in range(N): > > x = x * 0.1 + y * 0.2 > > y = x * 0.3 + y * 0.4 > > print typeof(x), typeof(y) > > > > Here both x and y will be inferred as double > > > > Ok, so I assume it promotes the incoming types (all reaching > definitions)? If N == 0, then when using objects you get an int, > otherwise a double. What happens when the reaching types cannot be > promoted together? Do you fall back to object? Can you provide example please? And yes if type cannot be inferred it fallback to object. -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From markflorisson88 at gmail.com Tue May 21 15:45:29 2013 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 21 May 2013 14:45:29 +0100 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: On 21 May 2013 14:41, Vitja Makarov wrote: > > > > 2013/5/21 mark florisson >> >> On 21 May 2013 14:14, Vitja Makarov wrote: >> > >> > >> > >> > 2013/5/21 mark florisson >> >> >> >> On 21 May 2013 11:26, Vitja Makarov wrote: >> >> > Hi! >> >> > >> >> > Recently I've started work on new type inference engine. Now it's >> >> > almost >> >> > ready and I want to discuss it. >> >> > >> >> > It works like this: first infer type for each assignment then for >> >> > whole >> >> > entry. It has some advantages over previous algorithm: >> >> > - it handles assignment cycles, see test_swap() for example >> >> > - it can infer type using info about assignments on the whole entry: >> >> > >> >> > a = 1 >> >> > b = a >> >> > a = "str" >> >> > >> >> > a is python object and b is integer. >> >> > >> >> > Here are testcases that show some new cases it can solve: >> >> > >> >> > >> >> > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx >> >> > >> >> > Here is branch for it >> >> > https://github.com/vitek/cython/tree/_type_inference_new >> >> > >> >> > -- >> >> > vitja. >> >> > >> >> > _______________________________________________ >> >> > cython-devel mailing list >> >> > cython-devel at python.org >> >> > http://mail.python.org/mailman/listinfo/cython-devel >> >> > >> >> >> >> Hey Vitja, >> >> >> >> Cool! How do you want to handle variable merge at control flow joins? >> >> Would you promote (backwards incompatible), use a union type, or >> >> object? E.g. what does this result in: >> >> >> >> x = 0 >> >> for i in range(N): >> >> x += 0.2 * i >> >> print typeof(x) >> > >> > >> > Hi Mark, >> > >> > Nothing changed in your example old algorithm was able to handle this >> > "simple" kind of cycles as well as new one: >> > >> > def foo(int N): >> > x = 0 >> > for i in range(N): >> > x += 0.2 * i >> > print typeof(x) >> > >> > So this function (not that N is an integer here) will print 'dobule'. >> > >> > With new algorithm we can go further: >> > >> > def foo(int N): >> > x = 1 >> > y = 0 >> > for i in range(N): >> > x = x * 0.1 + y * 0.2 >> > y = x * 0.3 + y * 0.4 >> > print typeof(x), typeof(y) >> > >> > Here both x and y will be inferred as double >> > >> >> Ok, so I assume it promotes the incoming types (all reaching >> definitions)? If N == 0, then when using objects you get an int, >> otherwise a double. What happens when the reaching types cannot be >> promoted together? Do you fall back to object? > > > Can you provide example please? And yes if type cannot be inferred it > fallback to object. > Sure, e.g. if cond: x = 2 else: x = "hello" use(x) A tricky one is this: try: x = f() # let's say 'double' x = g() # let's say 'string' finally: print typeof(x) # what does this print if 'f' can raise an exception? > > -- > vitja. > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > From vitja.makarov at gmail.com Tue May 21 15:49:45 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 21 May 2013 17:49:45 +0400 Subject: [Cython] CF based type inference In-Reply-To: References: Message-ID: 2013/5/21 mark florisson > On 21 May 2013 14:41, Vitja Makarov wrote: > > > > > > > > 2013/5/21 mark florisson > >> > >> On 21 May 2013 14:14, Vitja Makarov wrote: > >> > > >> > > >> > > >> > 2013/5/21 mark florisson > >> >> > >> >> On 21 May 2013 11:26, Vitja Makarov wrote: > >> >> > Hi! > >> >> > > >> >> > Recently I've started work on new type inference engine. Now it's > >> >> > almost > >> >> > ready and I want to discuss it. > >> >> > > >> >> > It works like this: first infer type for each assignment then for > >> >> > whole > >> >> > entry. It has some advantages over previous algorithm: > >> >> > - it handles assignment cycles, see test_swap() for example > >> >> > - it can infer type using info about assignments on the whole > entry: > >> >> > > >> >> > a = 1 > >> >> > b = a > >> >> > a = "str" > >> >> > > >> >> > a is python object and b is integer. > >> >> > > >> >> > Here are testcases that show some new cases it can solve: > >> >> > > >> >> > > >> >> > > https://github.com/vitek/cython/blob/_type_inference_new/tests/run/type_inference_new.pyx > >> >> > > >> >> > Here is branch for it > >> >> > https://github.com/vitek/cython/tree/_type_inference_new > >> >> > > >> >> > -- > >> >> > vitja. > >> >> > > >> >> > _______________________________________________ > >> >> > cython-devel mailing list > >> >> > cython-devel at python.org > >> >> > http://mail.python.org/mailman/listinfo/cython-devel > >> >> > > >> >> > >> >> Hey Vitja, > >> >> > >> >> Cool! How do you want to handle variable merge at control flow joins? > >> >> Would you promote (backwards incompatible), use a union type, or > >> >> object? E.g. what does this result in: > >> >> > >> >> x = 0 > >> >> for i in range(N): > >> >> x += 0.2 * i > >> >> print typeof(x) > >> > > >> > > >> > Hi Mark, > >> > > >> > Nothing changed in your example old algorithm was able to handle this > >> > "simple" kind of cycles as well as new one: > >> > > >> > def foo(int N): > >> > x = 0 > >> > for i in range(N): > >> > x += 0.2 * i > >> > print typeof(x) > >> > > >> > So this function (not that N is an integer here) will print 'dobule'. > >> > > >> > With new algorithm we can go further: > >> > > >> > def foo(int N): > >> > x = 1 > >> > y = 0 > >> > for i in range(N): > >> > x = x * 0.1 + y * 0.2 > >> > y = x * 0.3 + y * 0.4 > >> > print typeof(x), typeof(y) > >> > > >> > Here both x and y will be inferred as double > >> > > >> > >> Ok, so I assume it promotes the incoming types (all reaching > >> definitions)? If N == 0, then when using objects you get an int, > >> otherwise a double. What happens when the reaching types cannot be > >> promoted together? Do you fall back to object? > > > > > > Can you provide example please? And yes if type cannot be inferred it > > fallback to object. > > > > Sure, e.g. > > if cond: > x = 2 > else: > x = "hello" > use(x) > > x will be of object type here. > A tricky one is this: > > try: > x = f() # let's say 'double' > x = g() # let's say 'string' > finally: > print typeof(x) # what does this print if 'f' can raise an > exception? > > As well as here. Can it raise exception or not doesn't matter here since typeof() returns type of entry and it's calculated at compile time. -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikita at nemkin.ru Wed May 22 11:13:02 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Wed, 22 May 2013 15:13:02 +0600 Subject: [Cython] Compiler crash fix Message-ID: Hi, Someone please apply this patch (too simple for pull request): diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py index 0406fad..640d002 100755 --- a/Cython/Compiler/ExprNodes.py +++ b/Cython/Compiler/ExprNodes.py @@ -4993,7 +4993,7 @@ class AttributeNode(ExprNode): # creates a corresponding NameNode and returns it, otherwise # returns None. type = self.obj.analyse_as_extension_type(env) - if type: + if type and type.scope: entry = type.scope.lookup_here(self.attribute) if entry and entry.is_cmethod: if type.is_builtin_type: It fixes CompilerCrash (None does not have "lookup_here" method) that I have observed on two occasions: 1) cdef class Name; cimport Name; Name.attr 2) from X cimport Name; Name.attr # cimport_from_pyx is active, Name is a class with errors Makes me wonder if ErrorScope should be introduced to avoid None scope checks. Best regards, Nikita Nemkin From markflorisson88 at gmail.com Wed May 22 11:48:47 2013 From: markflorisson88 at gmail.com (mark florisson) Date: Wed, 22 May 2013 10:48:47 +0100 Subject: [Cython] Compiler crash fix In-Reply-To: References: Message-ID: On 22 May 2013 10:13, Nikita Nemkin wrote: > Hi, > > Someone please apply this patch (too simple for pull request): > > diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py > index 0406fad..640d002 100755 > --- a/Cython/Compiler/ExprNodes.py > +++ b/Cython/Compiler/ExprNodes.py > @@ -4993,7 +4993,7 @@ class AttributeNode(ExprNode): > # creates a corresponding NameNode and returns it, otherwise > # returns None. > type = self.obj.analyse_as_extension_type(env) > - if type: > + if type and type.scope: > entry = type.scope.lookup_here(self.attribute) > if entry and entry.is_cmethod: > if type.is_builtin_type: > > It fixes CompilerCrash (None does not have "lookup_here" method) > that I have observed on two occasions: > 1) cdef class Name; cimport Name; Name.attr > 2) from X cimport Name; Name.attr # cimport_from_pyx is active, Name is a > class with errors > > Makes me wonder if ErrorScope should be introduced to avoid None scope > checks. > > > Best regards, > Nikita Nemkin > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel Pull requests are useful not just for large patches, but for small ones since they serve as documentation, assign credit to the right person, and allow the merger to observe the travis-ci results. The ease with which things can be merged contribute largely to it getting merged. From vitja.makarov at gmail.com Fri May 24 07:02:20 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 24 May 2013 09:02:20 +0400 Subject: [Cython] How should be C-clobal deletion handled? Message-ID: Hi! Recently I've found that the following code causes segmentation fault: cdef object f del f print f So the question is: how should that work? global objects are implicitly initialized to None and no CF and no cf analysis is performed for it. So I see three options here: 1. prohibit cglobal deletion 2. set it back to None 3. check for a null value at every reference and assignment -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri May 24 08:00:41 2013 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 24 May 2013 18:00:41 +1200 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: Message-ID: <519F0209.8090000@canterbury.ac.nz> Vitja Makarov wrote: > Recently I've found that the following code causes segmentation fault: > > cdef object f > del f > print f FWIW, Pyrex disallows using del on cdef names. -- Greg From stefan_ml at behnel.de Fri May 24 21:32:03 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 24 May 2013 21:32:03 +0200 Subject: [Cython] Compiler crash fix In-Reply-To: References: Message-ID: <519FC033.8080502@behnel.de> Am 22.05.2013 11:48, schrieb mark florisson: > On 22 May 2013 10:13, Nikita Nemkin wrote: >> Someone please apply this patch (too simple for pull request): >> >> diff --git a/Cython/Compiler/ExprNodes.py b/Cython/Compiler/ExprNodes.py >> index 0406fad..640d002 100755 >> --- a/Cython/Compiler/ExprNodes.py >> +++ b/Cython/Compiler/ExprNodes.py >> @@ -4993,7 +4993,7 @@ class AttributeNode(ExprNode): >> # creates a corresponding NameNode and returns it, otherwise >> # returns None. >> type = self.obj.analyse_as_extension_type(env) >> - if type: >> + if type and type.scope: >> entry = type.scope.lookup_here(self.attribute) >> if entry and entry.is_cmethod: >> if type.is_builtin_type: >> >> It fixes CompilerCrash (None does not have "lookup_here" method) >> that I have observed on two occasions: >> 1) cdef class Name; cimport Name; Name.attr >> 2) from X cimport Name; Name.attr # cimport_from_pyx is active, Name is a >> class with errors >> >> Makes me wonder if ErrorScope should be introduced to avoid None scope >> checks. > > Pull requests are useful not just for large patches, but for small > ones since they serve as documentation, assign credit to the right > person, and allow the merger to observe the travis-ci results. The > ease with which things can be merged contribute largely to it getting > merged. All of the above, plus: once there's a test associated with a bug fix, the patch usually stops looking "too short to merit a pull request". And tests are the only way to make sure that bugs don't come back, i.e. that the patch is not wasted in the long run. Stefan From robertwb at gmail.com Sat May 25 08:34:22 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Fri, 24 May 2013 23:34:22 -0700 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: Message-ID: On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: > Hi! > > Recently I've found that the following code causes segmentation fault: > > cdef object f > del f > print f > > So the question is: how should that work? > > global objects are implicitly initialized to None and no CF and no cf > analysis is performed for it. > > So I see three options here: > > 1. prohibit cglobal deletion > 2. set it back to None > 3. check for a null value at every reference and assignment I'd go for 1, with 2 as a backup option. - Robert From stefan_ml at behnel.de Sat May 25 09:54:17 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 25 May 2013 09:54:17 +0200 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: Message-ID: <51A06E29.4080108@behnel.de> Am 25.05.2013 08:34, schrieb Robert Bradshaw: > On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: >> Recently I've found that the following code causes segmentation fault: >> >> cdef object f >> del f >> print f >> >> So the question is: how should that work? >> >> global objects are implicitly initialized to None and no CF and no cf >> analysis is performed for it. >> >> So I see three options here: >> >> 1. prohibit cglobal deletion >> 2. set it back to None >> 3. check for a null value at every reference and assignment > > I'd go for 1, with 2 as a backup option. +1 for 1. Stefan From vitja.makarov at gmail.com Sat May 25 19:29:29 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 25 May 2013 21:29:29 +0400 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: <51A06E29.4080108@behnel.de> References: <51A06E29.4080108@behnel.de> Message-ID: 2013/5/25 Stefan Behnel > Am 25.05.2013 08:34, schrieb Robert Bradshaw: > > On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: > >> Recently I've found that the following code causes segmentation fault: > >> > >> cdef object f > >> del f > >> print f > >> > >> So the question is: how should that work? > >> > >> global objects are implicitly initialized to None and no CF and no cf > >> analysis is performed for it. > >> > >> So I see three options here: > >> > >> 1. prohibit cglobal deletion > >> 2. set it back to None > >> 3. check for a null value at every reference and assignment > > > > I'd go for 1, with 2 as a backup option. > > +1 for 1. > > Stefan > > > I tried to disable it and found that it's already used for global C++ objects deletion. It would be strange to have global deletion for C++ objects and not for ordinary python objects. -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikita at nemkin.ru Sat May 25 19:37:29 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Sat, 25 May 2013 23:37:29 +0600 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: <51A06E29.4080108@behnel.de> Message-ID: On Sat, 25 May 2013 23:29:29 +0600, Vitja Makarov wrote: > 2013/5/25 Stefan Behnel > >> Am 25.05.2013 08:34, schrieb Robert Bradshaw: >> > On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: >> >> Recently I've found that the following code causes segmentation >> fault: >> >> >> >> cdef object f >> >> del f >> >> print f >> >> >> >> So the question is: how should that work? >> >> >> >> global objects are implicitly initialized to None and no CF and no cf >> >> analysis is performed for it. >> >> >> >> So I see three options here: >> >> >> >> 1. prohibit cglobal deletion >> >> 2. set it back to None >> >> 3. check for a null value at every reference and assignment >> > >> > I'd go for 1, with 2 as a backup option. >> >> +1 for 1. >> >> Stefan >> >> >> > I tried to disable it and found that it's already used for global C++ > objects deletion. It would be strange to have global deletion for C++ > objects and not for ordinary python objects. There is nothing strange about it. C++ pointers behave differently from Python objects and users are well aware of that. Moreover, C++ deletion is a totally different operation that just happens to reuse the same keyword. Best regards, Nikita Nemkin From robertwb at gmail.com Sat May 25 19:50:27 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Sat, 25 May 2013 10:50:27 -0700 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: <51A06E29.4080108@behnel.de> Message-ID: On May 25, 2013 10:37 AM, "Vitja Makarov" wrote: > > > > > 2013/5/25 Stefan Behnel >> >> Am 25.05.2013 08:34, schrieb Robert Bradshaw: >> > On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: >> >> Recently I've found that the following code causes segmentation fault: >> >> >> >> cdef object f >> >> del f >> >> print f >> >> >> >> So the question is: how should that work? >> >> >> >> global objects are implicitly initialized to None and no CF and no cf >> >> analysis is performed for it. >> >> >> >> So I see three options here: >> >> >> >> 1. prohibit cglobal deletion >> >> 2. set it back to None >> >> 3. check for a null value at every reference and assignment >> > >> > I'd go for 1, with 2 as a backup option. >> >> +1 for 1. >> >> Stefan >> >> > > I tried to disable it and found that it's already used for global C++ objects deletion. It would be strange to have global deletion for C++ objects and not for ordinary python objects. > Not that odd; C++ users are used to the concept of null pointers, Python users not so much. We can check this easily. > > -- > vitja. > > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vitja.makarov at gmail.com Sun May 26 09:48:58 2013 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sun, 26 May 2013 11:48:58 +0400 Subject: [Cython] How should be C-clobal deletion handled? In-Reply-To: References: <51A06E29.4080108@behnel.de> Message-ID: 2013/5/25 Robert Bradshaw > > On May 25, 2013 10:37 AM, "Vitja Makarov" wrote: > > > > > > > > > > 2013/5/25 Stefan Behnel > >> > >> Am 25.05.2013 08:34, schrieb Robert Bradshaw: > >> > On Thu, May 23, 2013 at 10:02 PM, Vitja Makarov wrote: > >> >> Recently I've found that the following code causes segmentation > fault: > >> >> > >> >> cdef object f > >> >> del f > >> >> print f > >> >> > >> >> So the question is: how should that work? > >> >> > >> >> global objects are implicitly initialized to None and no CF and no cf > >> >> analysis is performed for it. > >> >> > >> >> So I see three options here: > >> >> > >> >> 1. prohibit cglobal deletion > >> >> 2. set it back to None > >> >> 3. check for a null value at every reference and assignment > >> > > >> > I'd go for 1, with 2 as a backup option. > >> > >> +1 for 1. > >> > >> Stefan > >> > >> > > > > I tried to disable it and found that it's already used for global C++ > objects deletion. It would be strange to have global deletion for C++ > objects and not for ordinary python objects. > > > > Not that odd; C++ users are used to the concept of null pointers, Python > users not so much. We can check this easily. > > > Ok, I've pushed a fix https://github.com/cython/cython/commit/adbe80eaa2e5f84060201562118026c7b69a440e -- vitja. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikita at nemkin.ru Sun May 26 19:04:05 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Sun, 26 May 2013 23:04:05 +0600 Subject: [Cython] Exception check optimization Message-ID: Hi, I wonder why is __pyx_filename (in exception check blocks) tracked dynamically? AFAIK it's impossible to split function body between multiple files (include only works at the top level), which makes filename a compile time constant for any given function. If the above is correct, __pyx_filename variable can be eliminated, saving at least 5 bytes per exception check (more on x64). On a related note, why is c_lineno enabled by default? So far I have only found one pretty rare case when it is useful - debugging autogenerated code in module init. I suspect that most people don't bother turning it off and get suboptimal code. Best regards, Nikita Nemkin From nikita at nemkin.ru Sun May 26 19:55:49 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Sun, 26 May 2013 23:55:49 +0600 Subject: [Cython] Old style dependency tracking Message-ID: Hi, I wonder if it's time to to remove old dependency tracking code? I mean this: http://trac.cython.org/cython_trac/ticket/379, CompilationOptions.recursive, .dep file generation etc. Since it is broken for a long time already, there is no problem with backwards (in)compatibility. And cythonize already provides equivalent functionality. Best regards, Nikita Nemkin From robertwb at gmail.com Tue May 28 03:46:12 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Mon, 27 May 2013 18:46:12 -0700 Subject: [Cython] Old style dependency tracking In-Reply-To: References: Message-ID: Makes sense to me. https://github.com/robertwb/cython/commit/67c183d5f62dcba600ceaa7a240157a3c2e5d5c2 On Sun, May 26, 2013 at 10:55 AM, Nikita Nemkin wrote: > Hi, > > I wonder if it's time to to remove old dependency tracking code? > I mean this: http://trac.cython.org/cython_trac/ticket/379, > CompilationOptions.recursive, .dep file generation etc. > > Since it is broken for a long time already, there is no > problem with backwards (in)compatibility. > And cythonize already provides equivalent functionality. > > > Best regards, > Nikita Nemkin > _______________________________________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/mailman/listinfo/cython-devel From alexander.belchenko at gmail.com Wed May 29 11:01:36 2013 From: alexander.belchenko at gmail.com (Alexander Belchenko) Date: Wed, 29 May 2013 09:01:36 +0000 (UTC) Subject: [Cython] Demos is not build Message-ID: I'm trying to build Demos as suggested in USAGE.txt and got error from linker: cheese.obj : error LNK2019: unresolved external symbol _find_cheeses referenced in function ___pyx_pf_6cheese_find C:\work\Python\Cython\Demos\cheese.pyd : fatal error LNK1120: 1 unresolved externals The same problem with released 0.19.1 and latest version from github. My system is Windows 7 64 bit with Python 2.7.5 32-bit installed on it. From robertwb at gmail.com Thu May 30 04:25:55 2013 From: robertwb at gmail.com (Robert Bradshaw) Date: Wed, 29 May 2013 19:25:55 -0700 Subject: [Cython] Exception check optimization In-Reply-To: References: Message-ID: On Sun, May 26, 2013 at 10:04 AM, Nikita Nemkin wrote: > Hi, > > I wonder why is __pyx_filename (in exception check blocks) > tracked dynamically? AFAIK it's impossible to split function > body between multiple files (include only works at the top level), > which makes filename a compile time constant for any given function. > > If the above is correct, __pyx_filename variable can be eliminated, > saving at least 5 bytes per exception check (more on x64). Probably because exception checks are heavy-weight enough that we haven't bothered optimizing them at this level yet. > On a related note, why is c_lineno enabled by default? So far I > have only found one pretty rare case when it is useful - debugging > autogenerated code in module init. Well, it used to be much more useful back when Cython was less mature. > I suspect that most people don't bother turning it off and get > suboptimal code. Again, raising exceptions is heavy-weight enough that I don't think this is a significant performance issue. These days we certainly could consider turning it off, because it is often irrelevant and a deviation from how exceptions look in Python. - Robert From linkmauve at linkmauve.fr Thu May 30 12:18:45 2013 From: linkmauve at linkmauve.fr (Emmanuel Gil Peyrot) Date: Thu, 30 May 2013 12:18:45 +0200 Subject: [Cython] Compiler crash on unmatched kwarg Message-ID: <20130530101845.GA6097@desktop> Hi, When the name of a kwarg in the definition and the declaration of a function doesn?t match, the compiler crash instead of returning a meaningful error. I have joined a testcase, it is reproductible at least on 0.19.1 and latest git. -- Emmanuel Gil Peyrot XMPP: OpenPGP: 24B1D609 -------------- next part -------------- A non-text attachment was scrubbed... Name: testcase.tar.xz Type: application/octet-stream Size: 392 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From dnaeon at gmail.com Thu May 30 13:23:39 2013 From: dnaeon at gmail.com (Marin Atanasov Nikolov) Date: Thu, 30 May 2013 14:23:39 +0300 Subject: [Cython] Core dump when Python is built without debug symbols Message-ID: Hello, First, apologies if this is not exactly the right place to post this, but I have tried already cython-users@ and my post is still pending somewhere for approval, so I've decided to give it a shot here. I'm working on Cython wrappers for a C library and everything was going smooth on my dev machine, until I've decided to install the wrappers on another machine. The only difference between these machines is that my dev machine has Python built with debug symbols and the other machine does not have debug symbols in Python. The issue I'm having and was trying to solve for the past few days is that if I run my Cython wrappers on a machine where Python does not have debug symbols it core dumps ugly. If I run the same code on another machine with Python + debug symbols then everything is running perfect. I've done some debugging and tracing as well, and noticed that the core dump happens at PyTuple_Size() when I have Python *without* debug symbols. Running Python + debug symbols shows that the interpreter takes a different path and does not core dump (does not even call PyTuple_Size() for some reason) Anyway, I've uploaded the backtrace and some gdb tracing in here: * http://users.unix-heaven.org/~dnaeon/cython-wrappers/wrappers-trace.txt The code that I have you can find in here: * http://users.unix-heaven.org/~dnaeon/cython-wrappers/src/ What really puzzles me is why it core dumps when Python does not have debug symbols and why it takes a different path (and not core dumping) when Python is built with debug symbols. I'd say that if it core dumps because of a NULL pointer or something similar, then having Python with or without debug symbols should not make a difference, but it does for some reason. I've been working on this for the past few days and I'm out of ideas now. Any help/hints/feedback is much appreciated. Thanks and regards, Marin -- Marin Atanasov Nikolov dnaeon AT gmail DOT com http://www.unix-heaven.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikita at nemkin.ru Thu May 30 14:08:34 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Thu, 30 May 2013 18:08:34 +0600 Subject: [Cython] Core dump when Python is built without debug symbols In-Reply-To: References: Message-ID: On Thu, 30 May 2013 17:23:39 +0600, Marin Atanasov Nikolov wrote: > Hello, > > First, apologies if this is not exactly the right place to post this, > but I > have tried already cython-users@ and my post is still pending somewhere > for approval, so I've decided to give it a shot here. > > I'm working on Cython wrappers for a C library and everything was going > smooth on my dev machine, until I've decided to install the wrappers on > another machine. > > The only difference between these machines is that my dev machine has > Python built with debug symbols and the other machine does not have debug > symbols in Python. > > The issue I'm having and was trying to solve for the past few days is > that > if I run my Cython wrappers on a machine where Python does not have debug > symbols it core dumps ugly. If I run the same code on another machine > with > Python + debug symbols then everything is running perfect. > > I've done some debugging and tracing as well, and noticed that the core > dump happens at PyTuple_Size() when I have Python *without* debug > symbols. > Running Python + debug symbols shows that the interpreter takes a > different path and does not core dump (does not even call PyTuple_Size() > for some reason) > > Anyway, I've uploaded the backtrace and some gdb tracing in here: > > * http://users.unix-heaven.org/~dnaeon/cython-wrappers/wrappers-trace.txt > > The code that I have you can find in here: > > * http://users.unix-heaven.org/~dnaeon/cython-wrappers/src/ > > What really puzzles me is why it core dumps when Python does not have > debug > symbols and why it takes a different path (and not core dumping) when > Python is built with debug symbols. > > I'd say that if it core dumps because of a NULL pointer or something > similar, then having Python with or without debug symbols should not > make a > difference, but it does for some reason. > > I've been working on this for the past few days and I'm out of ideas now. > > Any help/hints/feedback is much appreciated. In pkg-db.pxi, this line: return Pkg(pkg) and in pkg-pkg.pxi this line: def __cinit__(self, pkg): self._pkg = pkg are the problem. You are trying to pass a C pointer to __cinit__. What actually happens is that, because of the cast, Cython treats pkg as as Python object _itself_ and tries to do some object operations on it. Two ways to solve this (really common) problem: 1) Pass your pointer as a real python object containing the integer value of the pointer: def __cinit__(self, pkg): self._pkg = pkg # or Py_intptr_t return Pkg(pkg) Note the cast. When used on a C pointer, it makes a C integer that Cython convert to python int object. And when is used on a python int object, Cyton extracts its integer value as a C-level value, so it is castable to a C-level pointer. Using Py_ssize_t (or Py_intptr_t or your platform intptr_t) guarantees that the pointer will fit. 2) Don't use __cinit__. Declare your own cdef init method and call that. Since it's cdef method, it can take any C-level arguments, pointerw, whatever without the need to convert to python objects and back. @cython.final # small optimization cdef _init(self, c_pkg.pkg * pkg): self._pkg = pkg # cdef __cinit__ not needed. Maybe use it to set default # attribute values. cdef Pkg pkg = Pkg.__new__(Pkg) # fast object creation. Pkg() also works. pkg._init(pkg) (When you use the __new__ trick, __cnit__ is called without arguments and __init__ is not called at all. The benefit to this ugly syntax is that __new__ is faster than creating the object with traditional Pyhton syntax.) Also, if all your initialization is pointer assigment, you can just do it directly: cdef Pkg pkg = Pkg.__new__(Pkg) # fast object creation pkg._pkg = pkg There is a problem related to cython-dev. I think the code you wrote should have never compiled in the first place. cast must be required if you want to reinterpret a C pointer as object or vice versa. Best regards, Nikita Nemkin From nikita at nemkin.ru Thu May 30 14:38:35 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Thu, 30 May 2013 18:38:35 +0600 Subject: [Cython] Exception check optimization In-Reply-To: References: Message-ID: On Thu, 30 May 2013 08:25:55 +0600, Robert Bradshaw wrote: > On Sun, May 26, 2013 at 10:04 AM, Nikita Nemkin wrote: >> Hi, >> >> I wonder why is __pyx_filename (in exception check blocks) >> tracked dynamically? AFAIK it's impossible to split function >> body between multiple files (include only works at the top level), >> which makes filename a compile time constant for any given function. >> >> If the above is correct, __pyx_filename variable can be eliminated, >> saving at least 5 bytes per exception check (more on x64). > > Probably because exception checks are heavy-weight enough that we > haven't bothered optimizing them at this level yet. By "exception checks" I mean these things: if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 15; __pyx_clineno = __LINE__; goto __pyx_L1_error;} They are fast, but the problem is the extra code: neither __pyx_filename nor __pyx_clineno assignments are necessary. Removing them reduces binary size by 2-3%. If you're OK with it, I'll make a patch that removes __pyx_filename and disables __pyx_clineno by default. Best regards, Nikita Nemkin From dnaeon at gmail.com Thu May 30 15:51:16 2013 From: dnaeon at gmail.com (Marin Atanasov Nikolov) Date: Thu, 30 May 2013 16:51:16 +0300 Subject: [Cython] Core dump when Python is built without debug symbols In-Reply-To: References: Message-ID: On Thu, May 30, 2013 at 3:08 PM, Nikita Nemkin wrote: > > > In pkg-db.pxi, this line: > > return Pkg(pkg) > > and in pkg-pkg.pxi this line: > > def __cinit__(self, pkg): > self._pkg = pkg > > are the problem. > > Thanks, Nikita! That was the problem indeed :) The funny part is that I was looking for a way to actually pass a C pointer to __cinit__ and doing it that way the compiler didn't complain at all. In fact it used to work, until I've removed Python debug symbols and then I started seeing surprises :) > You are trying to pass a C pointer to __cinit__. What actually > happens is that, because of the cast, Cython treats pkg as > as Python object _itself_ and tries to do some object operations on it. > > Two ways to solve this (really common) problem: > > 1) Pass your pointer as a real python object containing the integer value > of the pointer: > > def __cinit__(self, pkg): > self._pkg = pkg # or Py_intptr_t > > return Pkg(pkg) > > Note the cast. When used on a C pointer, it makes a C > integer that Cython convert to python int object. And when > is used on a python int object, Cyton extracts its integer value > as a C-level value, so it is castable to a C-level pointer. > > Using Py_ssize_t (or Py_intptr_t or your platform intptr_t) guarantees > that the pointer will fit. > > 2) Don't use __cinit__. Declare your own cdef init method and call that. > Since it's cdef method, it can take any C-level arguments, pointerw, > whatever without the need to convert to python objects and back. > > @cython.final # small optimization > cdef _init(self, c_pkg.pkg * pkg): > self._pkg = pkg > > # cdef __cinit__ not needed. Maybe use it to set default > # attribute values. > > cdef Pkg pkg = Pkg.__new__(Pkg) # fast object creation. Pkg() also > works. > pkg._init(pkg) > > (When you use the __new__ trick, __cnit__ is called without > arguments and __init__ is not called at all. The benefit to > this ugly syntax is that __new__ is faster than creating the object > with traditional Pyhton syntax.) > > Also, if all your initialization is pointer assigment, you can > just do it directly: > > cdef Pkg pkg = Pkg.__new__(Pkg) # fast object creation > pkg._pkg = pkg > > > I've chosen to stick with 2) -- I guess I like it better and having an additional _init() is fine with me, makes things clearer. Thanks again! > There is a problem related to cython-dev. I think the code you > wrote should have never compiled in the first place. > cast must be required if you want to reinterpret a C pointer > as object or vice versa. > > Actually the code compiles just fine and also works as expected :) The only condition is to have Python with debug symbols installed. Any idea why having Python with debug symbols actually overcomes this issue? Thanks again Nikita! Best regards, Marin > > Best regards, > Nikita Nemkin > ______________________________**_________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/**mailman/listinfo/cython-devel > -- Marin Atanasov Nikolov dnaeon AT gmail DOT com http://www.unix-heaven.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikita at nemkin.ru Thu May 30 16:21:55 2013 From: nikita at nemkin.ru (Nikita Nemkin) Date: Thu, 30 May 2013 20:21:55 +0600 Subject: [Cython] Core dump when Python is built without debug symbols In-Reply-To: References: Message-ID: On Thu, 30 May 2013 19:51:16 +0600, Marin Atanasov Nikolov wrote: > Any idea why having Python with debug symbols actually overcomes this > issue? In debug builds, Python object structure is prepended with a couple of extra debug fields, which means a different piece of memory was wrongly interpreted and manipulated as a Python object. It is pure luck that whatever values were there didn't cause a crash. Best regards, Nikita Nemkin From stefan_ml at behnel.de Thu May 30 17:12:35 2013 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 30 May 2013 17:12:35 +0200 Subject: [Cython] Exception check optimization In-Reply-To: References: Message-ID: <51A76C63.9030105@behnel.de> Am 30.05.2013 14:38, schrieb Nikita Nemkin: > On Thu, 30 May 2013 08:25:55 +0600, Robert Bradshaw wrote: >> On Sun, May 26, 2013 at 10:04 AM, Nikita Nemkin wrote: >>> I wonder why is __pyx_filename (in exception check blocks) >>> tracked dynamically? AFAIK it's impossible to split function >>> body between multiple files (include only works at the top level), >>> which makes filename a compile time constant for any given function. What about includes? Are they allowed inside of functions? If not, then this sounds like a safe assumption. If they are allowed, then it should still be possible to special case the file name setting, i.e. set it to the default at function entry and only set it to a different value if it happens to change inside of a function. The C compiler should often (normally?) be smart enough to eliminate the constant assignment. >>> If the above is correct, __pyx_filename variable can be eliminated, >>> saving at least 5 bytes per exception check (more on x64). >> >> Probably because exception checks are heavy-weight enough that we >> haven't bothered optimizing them at this level yet. > > By "exception checks" I mean these things: > > if (unlikely(!__pyx_t_2)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = > 15; __pyx_clineno = __LINE__; goto __pyx_L1_error;} > > They are fast, but the problem is the extra code: neither > __pyx_filename nor __pyx_clineno assignments are necessary. > Removing them reduces binary size by 2-3%. > > If you're OK with it, I'll make a patch that removes __pyx_filename > and disables __pyx_clineno by default. I'm fine with both. I consider seeing the C line a debug feature that people shouldn't need normally. Stefan From dnaeon at gmail.com Thu May 30 17:52:41 2013 From: dnaeon at gmail.com (Marin Atanasov Nikolov) Date: Thu, 30 May 2013 18:52:41 +0300 Subject: [Cython] Core dump when Python is built without debug symbols In-Reply-To: References: Message-ID: On Thu, May 30, 2013 at 5:21 PM, Nikita Nemkin wrote: > On Thu, 30 May 2013 19:51:16 +0600, Marin Atanasov Nikolov < > dnaeon at gmail.com> wrote: > > Any idea why having Python with debug symbols actually overcomes this >> issue? >> > > In debug builds, Python object structure is prepended with a couple of > extra debug fields, which means a different piece of memory was wrongly > interpreted and manipulated as a Python object. > > It is pure luck that whatever values were there didn't cause a crash. > > > Thanks for the insight, Nikita! Your assistance is much appreciated! Best regards, Marin > > Best regards, > Nikita Nemkin > ______________________________**_________________ > cython-devel mailing list > cython-devel at python.org > http://mail.python.org/**mailman/listinfo/cython-devel > -- Marin Atanasov Nikolov dnaeon AT gmail DOT com http://www.unix-heaven.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: