[pypy-issue] [issue1040] Crash on Ubuntu with threads and gethostbyname_ex

[Akira Li]

New submission from Akira Li <4kir4.1i+pypy at gmail.com>:

I've tried to run a simple script that calls gethostbyname_ex from several
threads. It either runs normally or crashes with sigbus, sometimes -- segfault:

  import socket
  from Queue import Queue
  from threading import Thread
  
  def resolve_hostnames(queue, f=socket.gethostbyname_ex):
      for host in iter(queue.get, None):
          try: # resolve hostname
              f(host) # ignore result
          except IOError:
              pass
  
  def main():
      hosts = "youtube.com google.com facebook.com live.com".split()*1000
  
      queue = Queue(20)
      num_threads = 10
      threads = [Thread(target=resolve_hostnames, args=(queue,))
                 for _ in range(num_threads)]
      for t in threads:
          t.daemon = True
          t.start()
  
      for host in hosts: queue.put(host) # feed hosts to threads
      for _ in threads:  queue.put(None) # signal no more hosts
      for t in threads:  t.join()        # wait for completion
  
  main()

I can't reproduce the error if num_threads=1, or f=gethostbyname, or f=getaddrinfo.

With gdb it fails 1 in 4 times.

Bus error:

  (gdb) run
  Starting program: /home/me/bin/pypy dns-queue.py
  [Thread debugging using libthread_db enabled]
  [New Thread 0x7ffff50a9700 (LWP 12557)]
  [New Thread 0x7ffff48a8700 (LWP 12558)]
  [New Thread 0x7ffff40a7700 (LWP 12559)]
  [New Thread 0x7ffff38a6700 (LWP 12560)]
  [New Thread 0x7ffff30a5700 (LWP 12561)]
  [New Thread 0x7ffff28a4700 (LWP 12562)]
  [New Thread 0x7ffff20a3700 (LWP 12563)]
  [New Thread 0x7ffff18a2700 (LWP 12564)]
  [New Thread 0x7ffff10a1700 (LWP 12565)]
  [New Thread 0x7ffff08a0700 (LWP 12566)]
  
  Program received signal SIGBUS, Bus error.
  [Switching to Thread 0x7ffff20a3700 (LWP 12563)]
  0x0000000001045d53 in ?? ()
  (gdb) bt
  #0  0x0000000001045d53 in ?? ()
  #1  0x0000000001048191 in ?? ()
  #2  0x0000000000c44b62 in ?? ()
  ..snip..
  #35 0x000000000077d255 in ?? ()
  #36 0x0000000000410e96 in ?? ()
  #37 0x00007ffff6079efc in start_thread (arg=0x7ffff20a3700) at
pthread_create.c:304
  #38 0x00007ffff5db489d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:112
  #39 0x0000000000000000 in ?? ()
  
Segmentation fault:

  (gdb) run
  Starting program: /home/me/bin/pypy dns-queue.py
  [Thread debugging using libthread_db enabled]
  [New Thread 0x7ffff50a9700 (LWP 13741)]
  [New Thread 0x7ffff48a8700 (LWP 13742)]
  [New Thread 0x7ffff40a7700 (LWP 13743)]
  [New Thread 0x7ffff38a6700 (LWP 13744)]
  [New Thread 0x7ffff30a5700 (LWP 13745)]
  [New Thread 0x7ffff28a4700 (LWP 13746)]
  [New Thread 0x7ffff20a3700 (LWP 13747)]
  [New Thread 0x7ffff18a2700 (LWP 13748)]
  [New Thread 0x7ffff10a1700 (LWP 13749)]
  [New Thread 0x7ffff08a0700 (LWP 13750)]
  
  Program received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x7ffff28a4700 (LWP 13746)]
  0x0000000000c44eec in ?? ()
  (gdb) bt
  #0  0x0000000000c44eec in ?? ()
  #1  0x00007ffff56b97ec in ?? ()
  #2  0x00007ffff28a33f0 in ?? ()
  #3  0x0000000000000000 in ?? ()

I've compiled pypy-1.8 from sources
https://bitbucket.org/pypy/pypy/get/release-1.8.tar.bz2  

  $ pypy -mplatform
  Linux-3.0.0-15-generic-x86_64-with-debian-wheezy-sid

  $ pypy --version
  Python 2.7.2 (2346207d99463f299f09f3e151c9d5fa9158f71b, Feb 10 2012, 21:14:54)
  [PyPy 1.8.0 with GCC 4.6.1]

issue995 and issue737 might be related

----------
files: dns-queue.py
messages: 3887
nosy: pypy-issue
priority: bug
status: unread
title: Crash on Ubuntu with threads and gethostbyname_ex

________________________________________
PyPy bug tracker <tracker at bugs.pypy.org>
<https://bugs.pypy.org/issue1040>
________________________________________