[Python-Dev] Re: marshal / unmarshal

[Scott David Daniels]

Fredrik Lundh wrote:
>>pickle doesn't have the INF=>1.0 bug:
>>>>>import pickle
>>>>>pickle.loads(pickle.dumps(1e10000))
>>...
>>ValueError: invalid literal for float(): 1.#INF
>>>>>import cPickle
>>>>>cPickle.loads(cPickle.dumps(1e10000))
>>...
>>ValueError: could not convert string to float
>>>>>import marshal
>>>>>marshal.loads(marshal.dumps(1e10000))
>>1.0
> should I check in a fix for this?
> 
> the code in PyFloat_FromString contains lots of trickery to deal with more or less
> broken literals, and more or less broken C libraries.
> 
> unfortunately, and unlike most other functions with similar names, PyFloat_FromString
> takes a Python object, not a char pointer.  would it be a good idea to add a variant
> that takes a char*?  if so, should PyFloat_FromString use the new function, or are we
> avoiding that kind of refactoring for speed reasons these days?
> 
> any opinions?
> 
> </F> 
 From yesterday's sprint, we found a smallest-change style fix.
At the least a change like this will catch the unpacking:
in marshal.c (around line 500) in function r_object:
...
case TYPE_FLOAT:
	{
		char buf[256];
+		char *endp;
		double dx;
		n = r_byte(p);
		if (n == EOF || r_string(buf, (int)n, p) != n) {
			PyErr_SetString(PyExc_EOFError,
				"EOF read where object expected");
			return NULL;
		}
		buf[n] = '\0';
		PyFPE_START_PROTECT("atof", return 0)
-		dx = PyOS_ascii_atof(buf);
+		dx = PyOS_ascii_strtod(buf, &endptr);
		PyFPE_END_PROTECT(dx)
+		if buf + n != &endptr) {
+			PyErr_SetString(PyExc_ValueError,
+				"not all marshalled float text read");
+			return NULL;
+		}
		return PyFloat_FromDouble(dx);
	}


-- Scott David Daniels
Scott.Daniels at Acm.Org