[Python-Dev] issues with int/long on 64bit platforms - eg stringobject (PR#306)

[M.-A. Lemburg]

Trent Mick wrote:
> 
> > > Guido van Rossum wrote:
> > > >
> > > > The email below is a serious bug report.  A quick analysis shows that
> > > > UserString.count() calls the count() method on a string object, which
> > > > calls PyArg_ParseTuple() with the format string "O|ii".  The 'i'
> > > > format code truncates integers.  It probably should raise an overflow
> > > > exception instead.  But that would still cause the test to fail --
> > > > just in a different way (more explicit).  Then the string methods
> > > > should be fixed to use long ints instead -- and then something else
> > > > would probably break...
> > >
> MAL wrote:
> > > All uses in stringobject.c and unicodeobject.c use INT_MAX
> > > together with integers, so there's no problem on that side
> > > of the fence ;-)
> > >
> > > Since strings and Unicode objects use integers to describe the
> > > length of the object (as well as most if not all other
> > > builtin sequence types), the correct default value should
> > > thus be something like sys.maxlen which then gets set to
> > > INT_MAX.
> > >
> > > I'd suggest adding sys.maxlen and the modifying UserString.py,
> > > re.py and sre_parse.py accordingly.
> >
> Guido wrote:
> > Hm, I'm not so sure.  It would be much better if passing sys.maxint
> > would just WORK...  Since that's what people have been doing so far.
> >
> 
> Possible solutions (I give 4 of them):
> [...]

Here is another one... I don't really like it because I think that
silent truncations are a bad idea, but to make things "just work
it would help:

* Change PyArg_ParseTuple() to truncate the range(INT_MAX+1, LONG_MAX+1)
  to INT_MAX and the same for negative numbers when passing a
  Python integer to a "i" marked variable. This would map
  range(INT_MAX+1, LONG_MAX+1) to INT_MAX and thus sys.maxint
  would turn out as INT_MAX in all those cases where "i" is
  used as parser marker. Dito for negative values.

With this truncation passing sys.maxint as default argument
for length parameters would "just work" :-).

The more radical alternative would be changing the Python
object length fields to long -- I don't think this is
practical though (and probably not really needed unless
you intend to work with 3GB strings ;).

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/