How to install Python package from source on Windows

[Deborah Swanson]

Gregory Ewing wrote, on Monday, May 15, 2017 4:55 PM
> 
> Deborah Swanson wrote:
> > It continues to amaze me that Anaconda and Python.org, probably the 
> > two biggest distributors of official Python builds, are now relying
on 
> > Visual C++. Why can't Python developers write the entire setup and 
> > installation code in Python?
> 
> The setup and installation code *is* written in Python. It's 
> just that some of the source of the particular package you're 
> trying to install is written in C, so installing it *from
> source* necessarily requires a C compiler.
> 
> Most people's Windows machines aren't set up for compiling C 
> code, and for that reason, prebuilt versions of such packages 
> are usually supplied for Windows.
> 
> So you have two reasonable options:
> 
> 1) Find a prebuilt wheel that's compatible with your system.
> 
> 2) Find another package written in pure Python that does what 
> you want.
> 
> It's also possible that you don't need the parts of 
> recordclass that are written in C. If that's the case you 
> might be able to install the pure Python parts manually. 
> That's usually just a matter of dropping the .py files in 
> appropriate places.

That might end up being the best choice for me, although I'm not
entirely sure what the appropriate places to drop the .py files are.
Probably the Scripts directory is one of them, but I don't know which
files go there and which files go other places, and I definitely don't
know what those other places are.

However, I think MRAB's success in pip installing recordclass after
upgrading pip might make workarounds unnecessary for me. I also haven't
upgraded pip since I last installed Python, and I did see a nag notice
that I should upgrade it. So that seems like the next step.

> > Relatedly, why is Python lacking in the basic build tools for users
to 
> > easily build and install packages and full releases from source? 
> > Something for Python (loosely) akin to the earliest versions of
Visual 
> > Studio would do the job,
> 
> Having the Python ecosystem stuck on some ancient version of 
> Visual Studio would be a bad situation for many reasons. 

Oh, and I was never thinking that should be done, that's why I said
"loosely akin to". Just something along those lines, updated for Python
and current realities, not the genuine original in the flesh!

My point then was to develop purely Python tools to do the job we're
leaning on Visual Studio to do, but I've learned in the last few
messages why that might be more complicated than I originally thought it
would be.

> One 
> of the main uses of C extensions is to interface with 
> external libraries, which will most likely have been compiled 
> with a recent toolset and rely on recent versions of 
> Microsoft's runtime libraries.

Another point I was unaware of, so thanks.

> There's also the fact that Microsoft doesn't make the free 
> versions of Visual Studio available forever and forbids 
> anyone else distributing them, so sufficiently old versions 
> are simply not available to most people.

Another reason why it's sad that Python hasn't been making its own build
tools, or incorporating and developing open software tools all along.
This is a tail that could easily end up wagging the dog, and soon. But
likely there are stll many things I don't know or haven't fully
understood from what I've learned today.
 
> The alternative would be to use some other free compiler such 
> as MinGW as the standard compiler for Python and its 
> extensions. But this has been decided against on the grounds 
> that it is better to use the standard Windows tools to 
> compile code for Windows, for the same compatibility reasons 
> as above, and that means Visual Studio.

Well, apparently that was the choice. I for one think it's sad, and that
it will lead to trouble for Python somewhere down the road. Probably the
trouble will be in its capacity to be available on all platforms, and I
think it's sad to have such binding ties with newer Windows. Time will
tell, but I think Windows is a slowly sinking ship, not unlike the
eventual fate of the behemoth IBM. It's the way of all things, and
Microsoft is no more immune to natural processes than anything else is.

> > Ah, and you've got me there. I have no clue what an ABI tag is or
why 
> > it's significant,
> 
> ABI = Application Binary Interface. Essentially it's the 
> interface between the Python interpreter and code that's been 
> compiled from C, and it can change from one version of Python 
> to another.

Thanks! Another mysterious acronym demystified!

> The error message from pip could be more useful -- it would 
> be nice to know exactly *what* it is about the wheel that 
> doesn't match your system!

Ahem, yes that would be most useful. 

> Some things it could be:
> 
> * Wrong version of Python
> 
> * Wrong CPU architecture (32 vs 64 bit)
> 
> -- 
> Greg

Or it could be any number of other things, since neither of those two
are obviously the problem here.

I do know, from trying to get Microsoft setup devs to make their error
messages more helpful, that this problem too is more complicated than it
might appear at first glance. There are often a lot of things that can
cause a particular error to pop in a particular situation, and it's hard
for devs to write enough code to cover all the bases and give the end
user an appropriate response text. Most of the whacko error messages you
see are some developer's attempt to do something better, but they didn't
know all the trigger combinations for that particular message text.

Maybe some brilliant computer scientist will solve the error message
quagmire someday. Not holding my breath. 

Deborah