[Patches] [ python-Patches-708374 ] add offset to mmap
SourceForge.net
noreply at sourceforge.net
Sat Oct 28 20:12:29 CEST 2006
Patches item #708374, was opened at 2003-03-23 15:33
Message generated for change (Comment added) made by loewis
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=708374&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Modules
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Neal Norwitz (nnorwitz)
Summary: add offset to mmap
Initial Comment:
This patch is from Yotam Medini <yotamm at
mellanox.co.il> sent to me in mail.
It adds support for the offset parameter to mmap.
It ignores the check for mmap size "if the file is
character device. Some device drivers (which I happen
to use) have zero size in fstat buffer, but still one
can seek() read() and tell()."
I added minimal doc and tests.
----------------------------------------------------------------------
>Comment By: Martin v. Löwis (loewis)
Date: 2006-10-28 20:12
Message:
Logged In: YES
user_id=21627
I have update the patch (mmap3.diff) for the current trunk
(52498).
I've reviewed and tested the Windows implementation, and found
the original patch to be incorrect: For MapViewOfFile,
instead of
passing the size, offset+size must be passed. I also extended it
to support 64-bit offsets on a 64-bit system (64-bit offsets on
a 32-bit system still aren't supported).
I have doubts about the changes to find and tell: why is it
good that it includes the offset? In particular, for find,
I think it should return the index in the buffer, not the
offset in the file.
----------------------------------------------------------------------
Comment By: Yotam Medini (yotam)
Date: 2003-12-08 20:01
Message:
Logged In: YES
user_id=22624
Recent posts deal mainly about with size checks against
stat() call.
But the main issue here is the support for offset in mmap()ping.
I wonder what is the status of this so
it would become part of the official release.
If nobody cares to implement and test it for MS-Windows,
let's have it for Linux/Unix only.
----------------------------------------------------------------------
Comment By: A.M. Kuchling (akuchling)
Date: 2003-07-15 15:07
Message:
Logged In: YES
user_id=11375
The device driver check is now committed to CVS, both the
trunk and 2.2 maintenance branch.
----------------------------------------------------------------------
Comment By: A.M. Kuchling (akuchling)
Date: 2003-07-14 22:02
Message:
Logged In: YES
user_id=11375
According to my reading of the Single Unix Specification
page for sys/stat.h,
st_size only has a sensible value for regular files and for
symlinks. This implies that the size comparison should only
be done if S_ISREG() returns true. Patch attached as
mmap_reg.diff; I'll also check it in after running the tests.
----------------------------------------------------------------------
Comment By: A.M. Kuchling (akuchling)
Date: 2003-07-11 15:54
Message:
Logged In: YES
user_id=11375
There's nothing to test on Windows; the HAVE_FSTAT code is
inside #ifdef UNIX. If it works on Unix (and it seems to
for me, on Linux), then it should be checked in.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-06-27 05:19
Message:
Logged In: YES
user_id=33168
Yes, the check for block devices should go in now as a bug
fix I think. Can anyone test on windows? I attached a
patch for just this minimal change.
----------------------------------------------------------------------
Comment By: A.M. Kuchling (akuchling)
Date: 2003-06-26 19:05
Message:
Logged In: YES
user_id=11375
Shouldn't block devices also be excluded from the size check?
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-06-05 16:59
Message:
Logged In: YES
user_id=33168
Since this is an enhancement, it will not go into 2.2.x.
The patch is stuck right now. It needs to be tested on
Windows, but I can't do that (no Windows development
environment). If you can update the patch to work on both
Unix and Windows, we can apply the patch.
----------------------------------------------------------------------
Comment By: Yotam Medini (yotam)
Date: 2003-06-05 16:55
Message:
Logged In: YES
user_id=22624
Just wondering, what is the status of this patch.
I guess it did not make it to 2.2.3.
regards -- yotam
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2003-04-23 05:53
Message:
Logged In: YES
user_id=80475
This is much closer and solves the last problem.
But, it fails test_mmap with a windows errocode 8: not
enought memory.
Raymond
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-10 20:20
Message:
Logged In: YES
user_id=33168
http://msdn.microsoft.com/library/en-us/fileio/base/mapviewoffile.asp?frame=true
says the offset must be a multiple of the allocation
granularity which used to be hard-coded at 64k. So maybe if
we made the offset 64k it would work. The attached patch
makes this test change for windows only. (I think the Unix
offset must be a multiple of PAGESIZE.)
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2003-04-10 19:53
Message:
Logged In: YES
user_id=80475
I had posted the wrong traceback (before the rebuild).
The correct one shows Window Error #87.
Traceback (most recent call last):
File "test_mmap.py", line 357, in ?
test_main()
File "test_mmap.py", line 353, in test_main
test_offset()
File "test_mmap.py", line 37, in test_offset
m = mmap.mmap(f.fileno(), mapsize - PAGESIZE,
offset=PAGESIZE)
WindowsError: [Errno 87] The parameter is incorrect
[9363 refs]
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-10 19:14
Message:
Logged In: YES
user_id=33168
Note to self: Self, make sure to backport S_ISCHR() fix.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-10 19:10
Message:
Logged In: YES
user_id=33168
Hmmm, did Modules/mmapmodule.c get rebuilt? There is code
in the patch for new_mmap_object() to support the offset
keyword in both Unix & Windows. I don't see a problem in
the code/patch.
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2003-04-10 18:01
Message:
Logged In: YES
user_id=80475
It doesn't run:
C:\py23\Lib\test>python_d test_mmap.py
Traceback (most recent call last):
File "test_mmap.py", line 357, in ?
test_main()
File "test_mmap.py", line 353, in test_main
test_offset()
File "test_mmap.py", line 37, in test_offset
m = mmap.mmap(f.fileno(), mapsize - PAGESIZE,
offset=PAGESIZE)
TypeError: 'offset' is an invalid keyword argument for this
function
[9363 refs]
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-04-10 01:35
Message:
Logged In: YES
user_id=33168
Raymond, could you try to test this patch and see if it
works on Windows?
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-03-29 22:28
Message:
Logged In: YES
user_id=33168
Sounds fair. Attached is an updated patch which includes
windows support (I think). I cannot test on Windows.
Tested on Linux. Includes updates for doc, src, and test.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2003-03-28 01:12
Message:
Logged In: YES
user_id=21627
I think non-zero offsets need to be supported for Windows as
well.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2003-03-23 16:37
Message:
Logged In: YES
user_id=33168
Email received from Yotam:
I have downloaded and patched the 2.3a source. compiled
locally just this module, and it worked fine for my
application (with offset for character device file) I did
not run the released test though.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=708374&group_id=5470
More information about the Patches
mailing list