From matusvalo at gmail.com  Tue May 16 15:09:52 2023
From: matusvalo at gmail.com (matus valo)
Date: Tue, 16 May 2023 21:09:52 +0200
Subject: [Cython] cython 3 migration update and next releases
Message-ID: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>

Hi All,

I would like to inform you about recent porting of projects to Cython 3.
Recently, I participated in migration of 3 bigger projects to Cython 3:

* Scipy - scipy main branch is currently compilable with Cython 3 with test
suite passing. Cython 3 master branch is used in CI to ensure that all
changes are compatible with Cython 3.
* Pandas - pandas library is compilable with Cython 3, the test suite is
passing except one test case (not caused by Cython in my opinion) -
https://github.com/pandas-dev/pandas/issues/53125
* scikit-image - library is compilable with Cython 3 (test suite was not
run though)

When migrating to Cython 3, I was able to find out several issues in the
Cython, all of them are merged in master now. Hence, I would like to ask
about next steps. It would help greatly to release Cython 3 beta3. This
will allow me to pin scipy CI to real pre-release instead of master branch.

Moreover, I would like to ask whether we can do the final Cython 3 release
after beta 3. The rationale is that the projects won't start really using
Cython 3 until we do the final release. Now, we have 3 big users of Cython
migrated, hence I think we have some confidence that Cython 3 is ready.
What do you think?


Thanks,


Matus
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cython-devel/attachments/20230516/9bf8d898/attachment.html>

From dalcinl at gmail.com  Wed May 17 08:44:36 2023
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 17 May 2023 15:44:36 +0300
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
Message-ID: <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>

On Tue, 16 May 2023 at 22:10, matus valo <matusvalo at gmail.com> wrote:

>  The rationale is that the projects won't start really using Cython 3
> until we do the final release. Now, we have 3 big users of Cython migrated,
> hence I think we have some confidence that Cython 3 is ready. What do you
> think?
>

I would not say that mpi4py is a big project, but it is around 15K lines of
Cython code, and it is also ready for Cython 3. I'll update mpi4py
requirements to `cython>=3` as soon as Cython 3 is released.

-- 
Lisandro Dalcin
============
Senior Research Scientist
Extreme Computing Research Center (ECRC)
King Abdullah University of Science and Technology (KAUST)
http://ecrc.kaust.edu.sa/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cython-devel/attachments/20230517/e64d09a1/attachment.html>

From stefan_ml at behnel.de  Sat May 20 05:59:51 2023
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 20 May 2023 11:59:51 +0200
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
Message-ID: <9927105c-458d-9253-6641-88546e2b25b5@behnel.de>

matus valo schrieb am 16.05.23 um 21:09:
> I would like to inform you about recent porting of projects to Cython 3.
> Recently, I participated in migration of 3 bigger projects to Cython 3:

Thanks a lot for doing this, Mat??. It helps Cython as much as it helps 
these projects.


> When migrating to Cython 3, I was able to find out several issues in the
> Cython, all of them are merged in master now. Hence, I would like to ask
> about next steps. It would help greatly to release Cython 3 beta3. This
> will allow me to pin scipy CI to real pre-release instead of master branch.

I'll try to get beta 3 released soon, but need to find a bit of consecutive 
time to get it out. There are still a couple of PRs that I'd like to look 
through.


> Moreover, I would like to ask whether we can do the final Cython 3 release
> after beta 3. The rationale is that the projects won't start really using
> Cython 3 until we do the final release. Now, we have 3 big users of Cython
> migrated, hence I think we have some confidence that Cython 3 is ready.
> What do you think?

It's probably a good time to have a final call for merges. Promoting and 
voting for PRs is welcome.

Stefan


From dw-git at d-woods.co.uk  Sun May 21 04:44:04 2023
From: dw-git at d-woods.co.uk (da-woods)
Date: Sun, 21 May 2023 09:44:04 +0100
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <9927105c-458d-9253-6641-88546e2b25b5@behnel.de>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
 <9927105c-458d-9253-6641-88546e2b25b5@behnel.de>
Message-ID: <a0350381-42be-d588-c377-8d0daaf2abe2@d-woods.co.uk>


On 20/05/2023 10:59, Stefan Behnel wrote:
> It's probably a good time to have a final call for merges. Promoting 
> and voting for PRs is welcome.

I've flagged a couple of PRs for possible merging. They're both fairly 
minor Python-compatibility fixes so it probably isn't a disaster if 
they're missed though.

Unfortunately, I doubt I can get the module state refactor into a usable 
state in time.

David


From sf at fermigier.com  Wed May 17 08:55:43 2023
From: sf at fermigier.com (Stefane Fermigier)
Date: Wed, 17 May 2023 14:55:43 +0200
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
 <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>
Message-ID: <48f917a7-e45a-4015-0bb8-5316d8671ec7@fermigier.com>


On 17/05/2023 14:44, Lisandro Dalcin wrote:
>
>
> On Tue, 16 May 2023 at 22:10, matus valo <matusvalo at gmail.com> wrote:
>
>     ?The rationale is that the projects won't start really using
>     Cython 3 until we do the final release. Now, we have 3 big users
>     of Cython migrated, hence I think we have some confidence that
>     Cython 3 is ready. What do you think?
>
>
> I would not say that mpi4py is a big project, but it is around 15K 
> lines of Cython code, and it is also ready for Cython 3. I'll update 
> mpi4py requirements to `cython>=3` as soon as Cython 3 is released.


IFAIK, 15k lines of Cython makes it among one of the largest Cython 
projects I'm aware of (I did some research a couple of years ago):

https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10-000-lines-of-cython-code

 ? S.

-- 

Stefane Fermigier - http://fermigier.com/ - 
http://twitter.com/sfermigier - http://linkedin.com/in/sfermigier

Founder & CEO, Abilian - Enterprise Social Software - 
http://www.abilian.com/

Founder & Software Architect - Nua open source PaaS project - 
https://nua.rocks/

Co-Founder & Co-Chairman, National Council for Free & Open Source 
Software (CNLL) - http://cnll.fr/

Co-Founder & Board Member, Association Professionnelle Europ?enne du 
Logiciel Libre (APELL) - https://www.apell.info/

Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) 
- https://www.euclidia.eu/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cython-devel/attachments/20230517/31b7f630/attachment.html>

From dimpase+github at gmail.com  Sun May 21 05:38:36 2023
From: dimpase+github at gmail.com (Dima Pasechnik)
Date: Sun, 21 May 2023 10:38:36 +0100
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <48f917a7-e45a-4015-0bb8-5316d8671ec7@fermigier.com>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
 <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>
 <48f917a7-e45a-4015-0bb8-5316d8671ec7@fermigier.com>
Message-ID: <CAAWYfq16PeY430k6BzKJCg1uP1Fc89fC7n_w+z2yLH-2ogbH5A@mail.gmail.com>

SageMath has 700K Cython lines, yet not mentioned.

On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf at fermigier.com> wrote:

>
> On 17/05/2023 14:44, Lisandro Dalcin wrote:
>
>
>
> On Tue, 16 May 2023 at 22:10, matus valo <matusvalo at gmail.com> wrote:
>
>>  The rationale is that the projects won't start really using Cython 3
>> until we do the final release. Now, we have 3 big users of Cython migrated,
>> hence I think we have some confidence that Cython 3 is ready. What do you
>> think?
>>
>
> I would not say that mpi4py is a big project, but it is around 15K lines
> of Cython code, and it is also ready for Cython 3. I'll update mpi4py
> requirements to `cython>=3` as soon as Cython 3 is released.
>
>
> IFAIK, 15k lines of Cython makes it among one of the largest Cython
> projects I'm aware of (I did some research a couple of years ago):
>
>
> https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10-000-lines-of-cython-code
>
>   S.
>
> --
>
> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier
> - http://linkedin.com/in/sfermigier
>
> Founder & CEO, Abilian - Enterprise Social Software -
> http://www.abilian.com/
>
> Founder & Software Architect - Nua open source PaaS project -
> https://nua.rocks/
>
> Co-Founder & Co-Chairman, National Council for Free & Open Source Software
> (CNLL) - http://cnll.fr/
>
> Co-Founder & Board Member, Association Professionnelle Europ?enne du
> Logiciel Libre (APELL) - https://www.apell.info/
>
> Co-Founder & Spokesperson, European Cloud Industrial Alliance (EUCLIDIA) -
> https://www.euclidia.eu/
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/cython-devel/attachments/20230521/2d9cc50f/attachment-0001.html>

From stefan_ml at behnel.de  Sun May 21 06:17:59 2023
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 21 May 2023 12:17:59 +0200
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <CAAWYfq16PeY430k6BzKJCg1uP1Fc89fC7n_w+z2yLH-2ogbH5A@mail.gmail.com>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
 <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>
 <48f917a7-e45a-4015-0bb8-5316d8671ec7@fermigier.com>
 <CAAWYfq16PeY430k6BzKJCg1uP1Fc89fC7n_w+z2yLH-2ogbH5A@mail.gmail.com>
Message-ID: <abc28a02-957f-f4ff-69fd-aa72eea2dc27@behnel.de>

Dima Pasechnik schrieb am 21.05.23 um 11:38:
> On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf at fermigier.com> wrote:
>> IFAIK, 15k lines of Cython makes it among one of the largest Cython
>> projects I'm aware of (I did some research a couple of years ago):
>>
>> https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10-000-lines-of-cython-code
>
 > SageMath has 700K Cython lines, yet not mentioned.

Certainly worth mentioning, yes.

Looking at the numbers, I also noticed that lxml is listed in the 5-10k 
lines range. It actually has about 18k lines of Cython code (.pyx/.pxi 
files) and another 1.5k lines in compiled Python (.py) files, according to 
pygount [1]. I tried sloccount first, but that doesn't seem to have Cython 
support.

Might be worth redoing that count for the other projects as well.

Stefan


[1] https://pypi.org/project/pygount/


From dimpase+github at gmail.com  Sun May 21 08:59:55 2023
From: dimpase+github at gmail.com (Dima Pasechnik)
Date: Sun, 21 May 2023 13:59:55 +0100
Subject: [Cython] cython 3 migration update and next releases
In-Reply-To: <abc28a02-957f-f4ff-69fd-aa72eea2dc27@behnel.de>
References: <CAHm8GShcJ60-u48C77RqpjX5Ggw=3fc2JNmr2W3n_Aa+OJW46w@mail.gmail.com>
 <CAEcYPwDQJu8X9pO4p-gM_H13VT6+fj+HbnvSRZ1dwj_KzOnXNg@mail.gmail.com>
 <48f917a7-e45a-4015-0bb8-5316d8671ec7@fermigier.com>
 <CAAWYfq16PeY430k6BzKJCg1uP1Fc89fC7n_w+z2yLH-2ogbH5A@mail.gmail.com>
 <abc28a02-957f-f4ff-69fd-aa72eea2dc27@behnel.de>
Message-ID: <CAAWYfq1wATiG63KJAObMb85Z015X7jJRe7vS4zQ46rpUYeWnAw@mail.gmail.com>

On thing to take care of before the release is to get rid of the hugely outdated
advice to ship generated C/C++ files!

On Sun, May 21, 2023 at 11:24?AM Stefan Behnel <stefan_ml at behnel.de> wrote:
>
> Dima Pasechnik schrieb am 21.05.23 um 11:38:
> > On Sun, 21 May 2023, 10:21 Stefane Fermigier, <sf at fermigier.com> wrote:
> >> IFAIK, 15k lines of Cython makes it among one of the largest Cython
> >> projects I'm aware of (I did some research a couple of years ago):
> >>
> >> https://github.com/sfermigier/awesome-cython#some-projects-with-more-that-10-000-lines-of-cython-code
> >
>  > SageMath has 700K Cython lines, yet not mentioned.
>
> Certainly worth mentioning, yes.
>
> Looking at the numbers, I also noticed that lxml is listed in the 5-10k
> lines range. It actually has about 18k lines of Cython code (.pyx/.pxi
> files) and another 1.5k lines in compiled Python (.py) files, according to
> pygount [1]. I tried sloccount first, but that doesn't seem to have Cython
> support.
>
> Might be worth redoing that count for the other projects as well.
>
> Stefan
>
>
> [1] https://pypi.org/project/pygount/
>
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> https://mail.python.org/mailman/listinfo/cython-devel