[IPython-dev] Qt console making great strides, please use it and let us know what works, what doesn't

[David Hirschfeld]

On Mon, Apr 11, 2011 at 9:58 PM, MinRK <benjaminrk at gmail.com> wrote:
> And what are the values of:
> zmq.EPROTONOSUPPORT
> and
> zmq.strerror(zmq.EPROTONOSUPPORT)
>
> ?
>
> The fact that the zmq error code doesn't match itself in your pyzmq
> test worries me, but maybe it's a Windows thing (though not on my
> Windows VMs).
>
> Under what circumstances are '.lib' files needed in addition to
> '.dll'?  I've never needed it.
>
> I'm also surprised that pyzmq didn't copy libzmq.dll along with it, as
> it is in the package_data for setup.  Can you post the complete output
> of a pyzmq build and install (after your libzmq.dll/lib fixes)?  I
> cannot replicate these problems on my XP VM.
>
> On Mon, Apr 11, 2011 at 13:42, Dave Hirschfeld
> <dave.hirschfeld at gmail.com> wrote:
>>
>> FWIW the results are:
>>
>> In [2]: import zmq
>>
>> In [3]: zmq.zmq_version()
>> Out[3]: '2.1.4'
>>
>> In [4]: zmq.pyzmq_version()
>> Out[4]: '2.1dev'
>>
>> from Python(x,y) 2.6.5.6
>>
>> -Dave
>>

In [5]: zmq.EPROTONOSUPPORT
Out[5]: 156384714

In [6]: zmq.strerror(zmq.EPROTONOSUPPORT)
Out[6]: 'Unknown error'

After building ZMQ 2.1.4 with VS2010 the contents of the lib and include
directories are:

C:\dev\src\pyzmq>ls C:/dev/src/zeromq-2.1.4/lib
libzmq.dll

C:\dev\src\pyzmq>ls C:/dev/src/zeromq-2.1.4/include
zmq.h  zmq.hpp  zmq_utils.h

i.e. no libzmq.lib in C:/dev/src/zeromq-2.1.4/lib.

At this point running the configure stage of pyzmq gives:

C:\dev\src\pyzmq>python setup.py configure --zmq=C:\dev\src\zeromq-2.1.4
running configure
******************************************
Configure: Autodetecting ZMQ settings...
   Custom ZMQ dir:       C:\dev\src\zeromq-2.1.4

c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-2.1.4\include -Izmq\utils -Izmq\core
-Izmq\devices /Tcdetect\vers.c /Fodetect\vers.obj vers.c

c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /nologo
/INCREMENTAL:NO /LIBPATH:C:\dev\src\zeromq-2.1.4\lib libzmq.lib detect\vers.obj
/OUT:detect\vers.exe /MANIFESTFILE:detect\vers.exe.manifest

LINK : fatal error LNK1181: cannot open input file 'libzmq.lib'
Fatal:
   Failed to compile ZMQ test program.  Please check to make sure:

   * You have a C compiler installed
   * A development version of Python is installed (including header files)
   * A development version of ZeroMQ >= 2.1.0 is installed (including header
     files)
   * If ZMQ is not in a default location, supply the argument --zmq=<path>
******************************************

-i.e. it's looking for libzmq.lib

In case it helps I patched line 241 of setup.py to save the exception
as exc and debugged it
in IPython:

> c:\dev\src\pyzmq\setup.py(255)run()
   254             print ("*"*42)
--> 255             self.erase_tempdir()
   256         self.config = config

ipdb> exc
LinkError(DistutilsExecError('command \'"c:\\Program Files (x86)\\Microsoft
Visual Studio 9.0\\VC\\BIN\\link.exe"\' failed with exit status 1181',),)
ipdb> settings
{'libraries': ['libzmq'],
'library_dirs': ['C:/dev/src/zeromq-2.1.4\\lib'],
'extra_compile_args': [],
'include_dirs': ['C:/dev/src/zeromq-2.1.4\\include',
                 'zmq\\utils', 'zmq\\core', 'zmq\\devices']}

I'm no expert but I think .lib files are compile/link time dependencies and
.dlls are only runtime dependencies so I don't think you can get around
copying the libzmq.lib file to the lib (C:/dev/src/zeromq-2.1.4/lib in my case)
directory.

Anyway, if you do that the configure runs fine:

C:\dev\src\pyzmq>python setup.py configure --zmq=C:\dev\src\zeromq-2.1.4
running configure
******************************************
Configure: Autodetecting ZMQ settings...
   Custom ZMQ dir:       C:\dev\src\zeromq-2.1.4

c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox
/MD /W3 /GS- /DNDEBUG -IC:\dev\src\zeromq-2.1.4\include -Izmq\utils -Izmq\core
-Izmq\devices /Tcdetect\vers.c /Fodetect\vers.obj vers.c

c:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /nologo
/INCREMENTAL:NO /LIBPATH:C:\dev\src\zeromq-2.1.4\lib libzmq.lib
detect\vers.obj /OUT:detect\vers.exe /MANIFESTFILE:detect\vers.exe.manifest

C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\mt.exe -nologo -manifest
detect\vers.exe.manifest -outputresource:detect\vers.exe;1

   ZMQ version detected: 2.1.4
******************************************

Running the install command:

C:\dev\src\pyzmq> python setup.py install 2>&1 | tee pyzmq.build.log

...seemed to run fine too. Lines 45 & 159 of the attached log would suggest
that the dll is being copied over:

45 copying zmq\libzmq.dll -> build\lib.win32-2.6\zmq
159 copying build\lib.win32-2.6\zmq\libzmq.dll  ->
    C:\dev\bin\Python26\Lib\site-packages\zmq

...and indeed libzmq.dll is there this time so I think we can chalk that one
up to user error, sorry.

This still leaves us with the zmq error:

Assertion failed: Socket operation on non-socket
(c:\dev\src\zeromq-2.1.4\src\zmq.cpp:632)

...whenever we try to use the parallel code.

I next patched line 29 of zmq/__init__.py to set `here` to point to the
output of the Visual Studio ZMQ project:

here = r'C:\dev\src\zeromq-2.1.4\lib'

and ran the ipcluster script from the Visual Studio debugger. As expected the
debugger stopped at line 632 of zmq.cpp in the zmq_poll function.
Unfortunately that's the limit of my capabilities :( I've attached the output
of the locals window at the point of the exception in case it helps.

-Dave
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyzmq.build.log
Type: application/octet-stream
Size: 25538 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110412/35944722/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: locals.png
Type: image/png
Size: 67424 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110412/35944722/attachment.png>