[pypy-dev] Play with fuzzing

[Victor Stinner]

Hi,

I wrote a fuzzing "framework" called Fusil and this week I wrote a fuzzer for 
Python. The idea is quite simple: for a module,
 - list all functions, classes and class methods
 - call a function with random arguments (of random types)
 - instanciate a class with random arguments
 - if the class is created correctly, call methods with random arguments

Example:
--------------------- 8< -----------------------------------
print "Call 39/40: linuxaudiodev.open()"
try:
    linuxaudiodev.open(
        # argument 1/2
        u"\u62C0\uFBD7\uB46A\u55E0\uFB7B\uD392\u7CEE",
        # argument 2/2
        52.682,
    )
except Exception, err:
    print >>stderr, "ERROR: %s" % err
--------------------- 8< -----------------------------------

I tried it on CPython 2.5 and then on CPython trunk (future 2.6). I found some 
bugs:

http://bugs.python.org/issue3304 
  -> invalid call to PyMem_Free() in fileio_init()
http://bugs.python.org/issue3306
  -> audioop.findmax() crashs with negative length
http://bugs.python.org/issue3303
  -> invalid ref count on locale.strcoll() error
http://bugs.python.org/issue3302 
  -> segfault on gettext(None)
http://bugs.python.org/issue3301 
  -> DoS when lo is negative in bisect.insort_right()
http://bugs.python.org/issue3299
  -> invalid object destruction in re.finditer()

Most bugs crash with a segmentation fault, abort or a denial of service.

If you would like to try my fuzzer, use:
 (1) svn co http://fusil.hachoir.org/svn/trunk fusil
 (2) cd fusil
 (3) ./run_fusil.sh -p projects/python.py --fast --remove ALL

The option --fast goes faster, --remove does remove session directory even if 
Python generated some files, and "ALL" test all modules.

FUSIL IS NOT SAFE! So run it under a different using to avoid dangerous call 
to os.unlink().

Module list is hardcoded: it's the list of CPython modules written in C.

More informations about Fusil:
   http://fusil.hachoir.org/trac

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/