setup.py + cython == chicken and the egg problem

[Christian Gollwitzer]

Am 16.08.22 um 23:03 schrieb Dan Stromberg:
> I'm attempting to package up a python package that uses Cython.
> 
> Rather than build binaries for everything under the sun, I've been focusing
> on including the .pyx file and running cython on it at install time.  This
> requires a C compiler, but I'm OK with that.
>
> However, when I try to install the package from test.pypi.org, I get:
> $ python3 -m pip install -i https://test.pypi.org/simple/ pyx-treap
> below cmd output started 2022 Tue Aug 16 01:55:16 PM PDT
> Looking in indexes: https://test.pypi.org/simple/
> Collecting pyx-treap
>    Downloading
> https://test-files.pythonhosted.org/packages/3a/41/af5360934adccfc086a39e1f720323895144b53454ff6dacc0f06267db55/pyx_treap-2.0.15.tar.gz
> (125 kB)
> 
>   ????????????????????????????????????????????????????????????????????????????????
> 125.9/125.9 kB 1.9 MB/s eta 0:00:00
>    Installing build dependencies ... error
>    error: subprocess-exited-with-error
> 
>    ×? pip subprocess to install build dependencies did not run successfully.
>    ?? exit code: 1
>    ????> [3 lines of output]
>        Looking in indexes: https://test.pypi.org/simple/
>        ERROR: Could not find a version that satisfies the requirement
> setuptools (from versions: none)
>        ERROR: No matching distribution found for setuptools
>        [end of output]
> 
>    note: This error originates from a subprocess, and is likely not a
> problem with pip.
> error: subprocess-exited-with-error


I looked at your code and I think you are trying too hard. As far as I 
understand, you need Cython to be installed before the build process 
begins. Your entry in pyproject.toml should take care of that.
But you also have these lines in your setup.py

subprocess.check_call('%s -m pip install cython' % (sys.executable, ), 
shell=True)
subprocess.check_call('%s -m cython pyx_treap.pyx' % (sys.executable, ), 
shell=True)

The first one calls out to pip while pip is already running, I'm not 
sure that this will work, but judging from the error message it is 
looking for the requirements also from test.pypi. Maybe this is the 
reason that it fails (the error message says that it can't find 
setuptools). So jut delete this line and it might already work

The second line, which compiles the Cython code, also runs *at every 
invocation of setup.py*, even if you'd do just

	python3 setup.py --help

It may still work, but the correct way to do it is to create a build 
extension for setuptools. In my project you can see this here:

	https://github.com/j-from-b/CDEF/blob/main/setup.py#L88

OTOH, I would be surprised if Cython did not have this already, indeed 
you imported cythonize from Cython.Build. So maybe just deleting these 
two lines and it might work?

	Christian