[ python-Bugs-977934 ] Python compiler encodes path to source in .pyc

[SourceForge.net]

Bugs item #977934, was opened at 2004-06-23 04:17
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=977934&group_id=5470

Category: IDLE
Group: Python 2.3
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Jonathan Polley (jwpolley)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python compiler encodes path to source in .pyc

Initial Comment:
This bug was notices in version 2.0, but I was able to
reproduce it in version 2.3.4.  When a python module is
compiled, the path to that file is encoded in the .pyc
file.  This causes problems when a multi-platform
development environment is used, in my case it is a
hybrid UNIX/Windows platform.

To reproduce the problem, perform the following steps
from within IDLE:

1)  run python
2)  import crlf
3)  exit python
4)  copy the crlf.py and crlf.pyc files from
Tools/Scripts to another directory
5)  run python
6)  add the path to the copies of crlf.py* to the start
of the system path.
7)  import crlf
8)  print crlf.__file__ (this will yield the proper
path to the files)
9)  using the Open Module..., try to open the crlf
module.  You will get an error saying that the module
was not found.

This is a major problem when doing multi-platform
debugging.  If an exception is raised in a module that
was compiled on another system (with a different path
or OS), the debugger can not find the file to open it
so it can be debugged.  It also makes the "Open
Module..." menu option unreliable.

If you look in the .pyc file you will find the path to
the location where the file was originally generated.

----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2004-06-28 06:52

Message:
Logged In: YES 
user_id=21627

You will have to report errors on tools one by one, and some
of them may not be fixable. From this report, it is not
clear what precisely the error in IDLE is.

In any case, the behaviour that the .pyc compiler embeds the
source file path cannot and will not change. The mechanism
for __file__ is different because __file__, for a .pyc file,
refers to that very .pyc file, not to the source file. OTOH,
the debug information refers to the source file.

If the .pyc file says the source is in
/rfs/proj/cse/scripts, you should arrange that the source
file really is in that location. If you move the files to a
different machine, you will need to delete the .pyc files.

I do not completely agree that the traceback is erroneous.
If your complaint is that it refers to a non-existing file:
this could be fixed by not printing the name of the file if
it doesn't exist. However, I would not think that this would
be an improvement.

If you think that the traceback should magically guess where
the source file is: this is not implementable in a reliable way.

----------------------------------------------------------------------

Comment By: Jonathan Polley (jwpolley)
Date: 2004-06-28 06:34

Message:
Logged In: YES 
user_id=391068

This bug was reported because the encoding of the path in the 
.pyc generates errors in some tools (namely IDLE) as well as 
generates erroneous tracebacks when exceptions are raised.  
When some of the not very software savvy script writers see "/rfs/
proj/cse/scripts" in their traceback, instead of the "P:/scripts" that 
they expect, it leads to confusion.

I'm not sure why the __file__ and exception mechanism use 
different methods for determining the path to the file, but they 
do.  The same can be said for the value of having the fully 
qualified path encoded in the .pyc.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2004-06-27 23:06

Message:
Logged In: YES 
user_id=21627

I fail to see any kind of bug in this report. Yes, the path
to the source code is hard-coded in the .pyc file, but that
is intentional, and not a bug. If you want the tracebacks to
be correct on multiple installations, you have to recreate
the .pyc files every time.

----------------------------------------------------------------------

Comment By: Nick Bastin (mondragon)
Date: 2004-06-25 16:48

Message:
Logged In: YES 
user_id=430343

Changing this to be filed against IDLE, not the parser/compiler (should 
also fix the debugger as well).  There is no magic value that the compiler 
could put in  there that would make this right, so external tools will just 
have to deal with that.

----------------------------------------------------------------------

Comment By: Nick Bastin (mondragon)
Date: 2004-06-25 16:47

Message:
Logged In: YES 
user_id=430343

Changing this to be filed against IDLE, not the parser/compiler (should 
also fix the debugger as well).  There is no magic value that the compiler 
could put in  there that would make this right, so external tools will just 
have to deal with that.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=977934&group_id=5470