[Pythonmac-SIG] py2app unable to find cprocessors.so

[Paul Wiseman]

On 13 September 2012 07:18, Ronald Oussoren <ronaldoussoren at mac.com> wrote:

>
> On 10 Sep, 2012, at 16:37, Paul Wiseman <poalman at gmail.com> wrote:
>
> Ah,
>
> I've found out how to recreate the error
>
> If I create a main.py with nothing but 'import sqlalchemy'
>
> then use the following setup.py:
>
> from setuptools import setup
>
> setup(
>     version="1",
>     name="TestApp1",
>     app=["main.py"],
>     setup_requires=["py2app"]
> )
>
> setup(
>     version="1",
>     name="TestApp2",
>     app=["main.py"],
>     setup_requires=["py2app"]
> )
>
> If it doesn't produce the error it's probably because of this: "The
> "cprocessors" module in SQLAlchemy is written in C and compiles
> conditionally, based on if the current platform supports compiling it.   If
> not present, SQLAlchemy continues to work perfectly well as the hooks which
> attempt to import it will fall back to pure-Python functions instead." So
> you may have a cprocessors.py which I dont think you'd get the problem,
> only if it compiled the .so when sqlalchemy installed.
>
>
> I had the cprocessors extension in my build (that is, py2app mentioned in
> copied the extension)
>
>
> I get the error, but only when it builds the second app. In my main build
> script I make a few apps in the same script (I make 3 apps which get moved
> into the main app, any additional code in their site-packages.zip is moved
> into the main apps zip, I remove the "sub-apps" Contents/Resources/lib
> folder and symlink it at run time to the main apps lib folder.)
>
> Is this a bug or are you never supposed to run multiple setups in the same
> build? If not how can I achieve the above?
>
>
> Calling distutils.setup multiple times is at best untested with py2app,
> and I wouldn't be surprised if it causes problems due to state leaking from
> one build into the next.  A workaround would be to use the subprocess
> module to run the setup jobs in separate processes.
>
>
Isn't the problem that they share dist folders, not a process? if not where
does the state exist? Would I need to subprocess them from different
directories?


> BTW. I don't quite understand what you are trying to do with these 3 apps.
>  Are you building 3 apps that share a lot of code/resources and where you
> want two of the apps to link to the 3th one to reduce the amount of
> disk-space used?
>
>
Yea exactly, I have some smaller apps which are used for specific separate
jobs (one has a simple gui and generates and gathers log files from the
main app and zips them up should the main app ever fail to open for
instance), the jobs are all to do with the main app and all use a sub set
of code to the main app, so I put the apps in the Resources folder and
symlink the lib folder so I can include them with only using a little extra
disk space, but more importantly keeping the installer size down.


> Ronald
>

> On 10 September 2012 13:18, Ronald Oussoren <ronaldoussoren at mac.com>wrote:
>
>>
>> On 9 Sep, 2012, at 20:34, Paul Wiseman <poalman at gmail.com> wrote:
>>
>> Hey,
>>
>> When building an app that is using sqlalchemy I get this error:
>>
>> creating python loader for extension 'sqlalchemy.cprocessors'
>> error:
>> /Users/paul/Source/Python/build/bdist.macosx-10.6-intel/python2.7-standalone/app/temp/sqlalchemy/cprocessors.py:
>> No such file or directory
>>
>> I took a look in site packages and there is no cprocessors.py, but a
>> cprocessors.so - so maybe it is just looking for the wrong extension
>>
>> I tried adding "sqlalchemy.cprocessors" to the includes list in py2app
>> but that hasn't helped.
>>
>> I was wondering if I can fool it by dropping an empty cprocessors.py so
>> it will build, then swap it out afterwards for the so, but I'm sure there's
>> a better way and I'm not convinced that could even work.
>>
>> Surely py2app doesn't assume every extension is .py, or if it does can it
>> be changed?
>>
>>
>> Py2app does not assume that every extension is a python file. Given the
>> messasge I'd say that the error occurs in the code path that creates a
>> helper python file that actually loads the exention.
>>
>> A little background information: when py2app creates the application
>> bundle all modules are stored in a zipfile and loaded using python's
>> zipimporter. Extensions cannot be stored in the zipfiles because the
>> zipimporter doesn't support that. Py2app therefore creates a placeholder
>> python module in the zipfile that loads the extensions from a directory in
>> the application bundle.
>>
>> BTW. could you please create a sample project that demonstrates the
>> problem? I've tried to reproduce your problem on my machine and failed to
>> do so. I did run into another problem, py2app generated an incomplete
>> bundle due to confusion between a _collections submodule in SQLAlchemy and
>> the _collections extension in the stdlib; that's something I'm currently
>> trying to fix.
>>
>> Ronald
>>
>> _______________________________________________
>> Pythonmac-SIG maillist  -  Pythonmac-SIG at python.org
>> http://mail.python.org/mailman/listinfo/pythonmac-sig
>> unsubscribe: http://mail.python.org/mailman/options/Pythonmac-SIG
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythonmac-sig/attachments/20120913/b3fcdea5/attachment-0001.html>