I lost nearly all my modules installing 3.7

[Elliott Roper]

On 28 Jun 2018, Terry Reedy wrote
(in article<mailman.176.1530208004.7721.python-list at python.org>):

> On 6/28/2018 1:08 PM, Elliott Roper wrote:
> > I have done something stupid. Don't know what.
>
> It appears that you ran 3.7 expecting that modules installed for 3.6
> would magically be available for 3.7.
Yes indeed. It worked for 3.4, 3.5, and 3.6
>
>
> There is a pip command for making an editable file of installed
> packages. Run that in 3.6, perhaps after updating everything.
>
>
> There is another pip command for using that file to install everything
> listed. Run that in 3.7.

I can't see the pip commands you mention for writing a file from 3.6 and 
reading it back for 3.7
Is it pip freeze -r <file>  followed by pip install -r<file>? If so, what is 
meant by 'the given requirements file' in the freeze options?
>
>
> > My $PATH looks like this
> > XXXMac:~ elliott$ echo $PATH
> > /Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Py
> > th
> > on.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/
> > 3.
> > 5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/elliott/b
> > in
> > > /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local
> > > /MacGPG2/bin:/Applications/Wireshark.app/Contents/MacOS:/usr/local/git/bin
> >
> > pip3 list or pip3.7 list gives me
> > Package Version
> > ---------- -------
> > pip 10.0.1
> > setuptools 39.0.1
>
> This is the content of the 3.7 site-packages.
>
> > > > > import numpy as np
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > ModuleNotFoundError: No module named 'numpy'
> > So it is not lying to me!!!
> >
> > pip list or pip3.6 list
> > gives the whole caboodle I was expecting with a far smaller version number
> > for setuptools.
>
> The content of the 3.6 site-packages directory.

When I look inside site-packages in ~/Library (see below) I see many packages 
that pip lists, but by no means all. F'instance numpy and scipy. They can be 
found in /Library's site-packages however.
>
>
> > My understanding is that the whole $PATH is searched in order to resolve an
> > import, but it isn't.
>
> The OS searches the OS path, which you listed above.
> Python searches its sys.path, which it creates when started.
> Run >>>  import sys; sys.path to see the contents.
> Unless macOS is more different than I think, you should see a 3.7
> site-packages when running 3.7.

Aha! That is most helpful

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(python prompt)>>>  import sys
.>>>sys.path
['', '/Library/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', # 
no such file 
'/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7', # 207 
items, none matching pip intstallable modules 
'/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-
dynload',# 69 items, none matching pip installable modules
'/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-
packages'] # just pip and setuptools are in here
.>>>  ^D

EiPro:~ elliott$ python3.6
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 05:52:31)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
.>>>  import sys
(python prompt)>>>  sys.path
['', '/Library/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', # 
no such file 
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6', # 207 
items, none matching pip intstallable modules 
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-
dynload',# 65 items, none matching pip installable modules
'/Users/elliott/Library/Python/3.6/lib/python/site-packages', # 103 items 
some matching items that appear in pip3.6 list 
'/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-
packages'] # numpy, scipy, pandas etc. are here. These might have been 
modules I needed sudo -H to update
.>>>

What I did not make clear that my 'elliott' user is not an Administrator. If 
I log into my admin account (which I hardly ever do), it thinks my Python3 is 
3.7, list there also shows the hugely abbreviated collection of modules 
namely pip and setuptools. pip list shows a list of modules so old as to be 
unrecognisable.

It looks like I have a tangled mess. Is there a way of getting rid of all the 
pythons except Apple's museum piece and starting again? It is a hobby for me. 
I have no need for backward compatibility. I think if I could install 3.7 
site wide from my non-admin account, I would be happiest. The standard 
install pretty much worked up to 3.6
pip
Would it be safe to delete everything on python's sys.path and re-install 
from the download .pkg
>
>
> > It might be relevant that I have had a bit of hassle installing module
> > updates in the past. I would get an error saying the module version being
> > replaced could not be deleted with permissions errors which I resolved with
> > a
> > bit of sudo -H.
> >
> > Python 3.6 is still working properly when invoked explicitly

Sorry, I am really out of my depth here.
Thanks so much for your help
-- 
To de-mung my e-mail address:- fsnospam$elliott$$ PGP Fingerprint: 1A96 3CF7 
637F 896B C810 E199 7E5C A9E4 8E59 E248