[IPython-dev] ZMQ Segfault - IPython cluster

[Dave Hirschfeld]

MinRK <benjaminrk <at> gmail.com> writes:

> 
> 
> Yes, this is expected. The default for FD_SETSIZE is 1024 in libzmq (up 
from the default of 64 on Windows), and can be increased at compile time. As 
maintainer of pyzmq, I am reluctant for the defaults of pyzmq's bundled 
libzmq to be different from the defaults for a traditionally built libzmq, 
but zeromq-dev may be open to pushing up the default.
>  
> 
> -MinRK
>  
> 
> <pedant> a segfault is bad memory access; assert != segfault </pedant>
> 


It was certainly unexpected for me! :P

There's also no mention of FD_SETSIZE on either the IPython or PyZMQ 
websites AFAICS.

It was a fairly nasty bug to track down because the remote process failed 
with no indication of why it did so. This is because Windows HPC 2008 only 
caches the first 4000 characters of stdout/stderr so the error message never 
got displayed (this has changed to a much more useful last 4k chars in 
2012). To actually see the message I had to remote on the node and start the 
ipcontroller process manually in a terminal. 

After actually managing to get an error message I then had to search the 
internet for what it meant and then recompile libzmq itself with VS2008 
which isn't exactly a cakewalk for the uninitiated.

I agree it should be fixed in the upstream library so I've opened an issue
there - we can see what they say there...

https://github.com/zeromq/libzmq/issues/1165


Thanks,
Dave