[ python-Bugs-1274069 ] bz2module.c compiler warning

[SourceForge.net]

Bugs item #1274069, was opened at 2005-08-26 09:25
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1274069&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: Tim Peters (tim_one)
>Assigned to: Reinhold Birkenfeld (birkenfeld)
Summary: bz2module.c compiler warning

Initial Comment:
On Python HEAD, there's a new warning while 
compiling bz2module.c (MSVC 7.1):

python\Modules\bz2module.c(1095) : warning 
C4244: '=' :
   conversion from 'Py_off_t' to 'int', possible loss of data

Looks like a legitimate complaint to me.  The line in 
question:

	readsize = offset-bytesread;

Those have types int, Py_off_t, and int respectively.  Is 
it true that offset-bytesread _necessarily_ fits in an int?  
If so, a comment should explain why, and a cast should 
be added to squash the warning.  Or If the difference 
doesn't necessarily fit in an int, the code is wrong.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2005-08-27 11:31

Message:
Logged In: YES 
user_id=80475

The latest patch gives me the following:

--------------------Configuration: bz2 - Win32
Debug--------------------
Compiling...
bz2module.c
C:\py25\Modules\bz2module.c(1143) : warning C4244:
'function' : conversion from '__int64 ' to 'long ', possible
loss of data
C:\py25\Modules\bz2module.c(1143) : warning C4761: integral
size mismatch in argument; conversion supplied
 lib /out:libbz2.lib blocksort.obj   huffman.obj    
crctable.obj    randtable.obj   compress.obj   
decompress.obj  bzlib.obj
Microsoft (R) Library Manager Version 6.00.8168
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.
 cl -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo -o bzip2
bzip2.c libbz2.lib setargv.obj
bzip2.c
 cl -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo -o
bzip2recover bzip2recover.c
bzip2recover.c
 type words1
Doing 6 tests (3 compress, 3 uncompress) ...
If there's a problem, things might stop at this point.
 
 .\bzip2 -1  < sample1.ref > sample1.rb2
 .\bzip2 -2  < sample2.ref > sample2.rb2
 .\bzip2 -3  < sample3.ref > sample3.rb2
 .\bzip2 -d  < sample1.bz2 > sample1.tst
 .\bzip2 -d  < sample2.bz2 > sample2.tst
 .\bzip2 -ds < sample3.bz2 > sample3.tst
All six of the fc's should find no differences.
If fc finds an error on sample3.bz2, this could be
because WinZip's 'TAR file smart CR/LF conversion'
is too clever for its own good. Disable this option.
The correct size for sample3.ref is 120,244. If it
is 150,251, WinZip has messed it up.
 fc sample1.bz2 sample1.rb2
Comparing files sample1.bz2 and sample1.rb2
FC: no differences encountered
 fc sample2.bz2 sample2.rb2
Comparing files sample2.bz2 and sample2.rb2
FC: no differences encountered
 fc sample3.bz2 sample3.rb2
Comparing files sample3.bz2 and sample3.rb2
****** sample3.bz2
BZh31AY&SYºÍ3É
·ïuU‰©´`†Û2     ªÉçI%@¯
‹¾Œàý£^‘1 ˯ðú£¦Ç“º™€•)„ëô·alèDh3H¯‘Ú\mIL´hiiÈ•B°Ró

****** sample3.rb2
BZh31AY&SY›îÜ>
ò#
óAŠJ3ù²ªÔ©zÉêè7UÎ{ýÍC2 
‹l
}ۇ******
 fc sample1.tst sample1.ref
Comparing files sample1.tst and sample1.ref
FC: no differences encountered
 fc sample2.tst sample2.ref
Comparing files sample2.tst and sample2.ref
FC: no differences encountered
 fc sample3.tst sample3.ref
Comparing files sample3.tst and sample3.ref
FC: no differences encountered
Linking...

bz2_d.pyd - 1 error(s), 2 warning(s)


----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-08-27 09:36

Message:
Logged In: YES 
user_id=1188172

Okay, next try. Someone with Windows should check this
before I check in.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2005-08-27 07:55

Message:
Logged In: YES 
user_id=21627

As Tim says: the cast deserves a comment, explaining why it
cannot overflow. Even when readsize is size_t, it might
still overflow, since Py_off_t might be wider than size_t.

Apart from that, the patch looks good - but I can't check
whether it silences the warning on Windows.

----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-08-27 07:27

Message:
Logged In: YES 
user_id=1188172

Attaching patch for f->pos and f->size as Py_off_t.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2005-08-27 05:30

Message:
Logged In: YES 
user_id=21627

Clearly, the difference is bound by buffersize (since, if it
is larger, readsize is set to buffersize). buffersize is
initialized to SMALLCHUNK, which is 8192, and never changed,
so yes, the difference fits to an int an all supported
platforms.

OTOH, the condition of the if statement is bogus:

    if ((size_t)offset-bytesread > buffersize)

offset is of type Py_off_t, which is a 64-bit type on most
platforms. Casting it to size_t causes truncation on 32-bit
platforms.

Furthermore, I think self->pos should be of type Py_off_t,
as it will wrap around for files larger >2GB on 32-bit
platforms; the same for self->size.

----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-08-26 10:05

Message:
Logged In: YES 
user_id=1188172

I think declaring readsize as size_t should fix the problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1274069&group_id=5470