What is the difference between 32 and 64 bit Python on Windows 7 64 bit?

[Sturla Molden]

On 11/05/14 08:56, Ross Gayler wrote:

> It looks to me as though 32 and 64 bit versions of Python on 64 bit
> Windows are both really 32 bit Python, differing only in how they
> interact with Windows.

No! Pointers are 64 bit, Python integers (on Python 2.x) are 32 bit. 
Microsoft decided to use a 32 bit long in MSVC for backwards 
compatiblity, but also because the AMD64 (x86-64) architecture was 
designed to use a 64 address with a 32 bit offset. (A 64 bit long was 
originally slightly less efficient.) You can see the value of 64 bit 
Python e.g. if you allocate a lot of objects or if you try to mmap a 
huge file. With 32 bit Python you are limited to only 2 GB of virtual 
memory. In 64 bit Python you can in practice mmap as much as you want.

The element size of what you try to index also matters. While a C long 
and a Python int is 32 bit on Windows, 64-bit Python will use a 64-bit 
offset internally (Py_ssize_t and Py_intptr_t) even on Windows. The 32 
bit Python int just limits how many objects you can index from Python 
space before Python roll over to using long instead of int. It does not 
limit the amount of memory a Python int can index. In is only when you 
index an array of bytes you will see the roll-over from Python int to 
Python long at the 2 GB limit. Typically, object will be much larger 
than one byte.

Here are two examples:

- A one-dimensional NumPy array with dtype np.float64 can keep 16 GB of 
data before a 32 bit index is too small and Python starts to use long. 
A two-dimensional NumPy array with dtype np.float64 can keep 256 GB of 
data before a 32 bit index is too small.

- A Python list stores internally an array of pointers, each of which is 
64 bit. So just indexing those goes up to 16 GB of pointer data before 
the int rolls over. Then each of these pointers point to a Python 
object. A Python float on my computer (not Windows) is 24 bytes, which I 
got from sys.getsizeof(1.) So 2**32 of those are another 383 GB. So if I 
indexed a list of Python floats on this computer, Python could handle an 
almost 400 GB data structure with a 32 bit int as indexer without 
rolling over to long.

This is obviously way beyond anything the 2 GB limit on 32 bit Python 
allows.




Sturla