[ python-Bugs-1289136 ] distutils extension library path bug on cygwin

[SourceForge.net]

Bugs item #1289136, was opened at 2005-09-12 15:04
Message generated for change (Comment added) made by jwhitley
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1289136&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: John Whitley (jwhitley)
Assigned to: Nobody/Anonymous (nobody)
Summary: distutils extension library path bug on cygwin

Initial Comment:
A while back I reported a problem on the Cygwin mailing
list where all python extension packages would fail
"python setup.py install" at the link step due to a
mangled lib dir (-L) option.  distutils was producing a
link line with "-L.", instead of the desired
"-L/usr/lib/python2.4/config".  I've finally rooted out
the cause of this problem.

The relevant code is the if-block starting at line 188 of:
  /usr/lib/python2.4/distutils/command/build_ext.py

I've reproduced that block here for clarity of
discussion (indentation truncated for redability)

  if sys.platform[:6] == 'cygwin' or sys.platform[:6]
== 'atheos':
      if string.find(sys.executable, sys.exec_prefix)
!= -1:
          # building third party extensions
         
self.library_dirs.append(os.path.join(sys.prefix, "lib",
                                       "python" +
get_python_version(),
                                       "config"))
      else:
          # building python standard extensions
          self.library_dirs.append('.')

The test "string.find(...) != -1" attempts to test
whether "/usr" appears in the full executable name. 
This incorrectly fails in the case that /bin is in the
user's path before /usr/bin.  (i.e.
string.find("/bin/python","/usr") == -1)  Note that a
vagary of Cygwin is that /usr/bin is a Cygwin mount to
/bin.

The user-side workaround is to ensure that /usr/bin
appears in your path before /bin.  It looks like a new
and improved Cygwin special case test is needed to fix
this problem; I'm not sure offhand what the best case
would be.  Perhaps an outright test as follows would work:
   sys.executable.startswith(sys.exec_prefix) or
sys.executable.startswith(os.sep+"bin")


----------------------------------------------------------------------

>Comment By: John Whitley (jwhitley)
Date: 2005-10-28 10:29

Message:
Logged In: YES 
user_id=649097

Jason is correct that readlink won't help.  On Cygwin
installations,
/bin and /usr/bin are not symlinks, they're aliases in
Cygwin's default mount system.  (C:\cygwin on / (root) and
C:\cygwin\bin on /usr/bin)  The Python 2.4.1 docs says:
"readlink(path)  Return a string representing the path to
which the symbolic link points."  For good measure, I
confirmed that readlink doesn't give the desired results.

The bigger problem that I see is that this code is using
hard-coded automagic path detection to figure out something
that it should be told explicitly.  That is, "are we in the
special case of building python's own libraries/extensions."
 (versus trying to install an extension to an existing
python installation.)

To illustrate the problem, both my proposal and the original
code are broken for trying to build python in /usr/local. 
(e.g. build under /usr/local/src -- thinks it's an install
not a build due to "/usr" prefix...)


----------------------------------------------------------------------

Comment By: Jason Tishler (jlt63)
Date: 2005-10-28 05:21

Message:
Logged In: YES 
user_id=86216

Unfortunately, I don't think readlink will help.

I was thinking of comparing inodes, but AFAICT inodes
under Cygwin are not guaranteed to be unique for all
platforms and filesystems.

So, maybe John's check is the only reliable option?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-23 19:47

Message:
Logged In: YES 
user_id=33168

What does os.readlink(sys.executable) return?

Maybe that will help you find the true canonical name?

----------------------------------------------------------------------

Comment By: Jason Tishler (jlt63)
Date: 2005-09-16 05:04

Message:
Logged In: YES 
user_id=86216

John,

Thanks for the excellent analysis!

All,

Unfortunately, I'm not sure what is the best way to solve this
problem.  Does anyone have any suggestions? Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1289136&group_id=5470