From eric.feigenson at editshare.com Tue Jan 2 12:43:52 2018 From: eric.feigenson at editshare.com (eric.feigenson at editshare.com) Date: Tue, 2 Jan 2018 12:43:52 -0500 Subject: [Pythonmac-SIG] Building python-based Mac app with py2app, creating an installer, generates failing installer In-Reply-To: References: <45776666-107A-4761-882C-2AE0F9DC99F4@editshare.com> Message-ID: <0271B588-552C-486A-974E-424F4C132C6C@editshare.com> Thanks for your replies. I managed to come up with something that worked! I had to do a couple of things, for those that are interested? I needed to upgrade py2app to version 0.8, which fixed a problem in the version we were using (0.7.3) where file permissions were not preserved when copying files into the application bundle. As an aside, I tried the most recent version (0.14), but it created a new and different problem (which I can?t recall at the moment). In the pkgbuild command, I had been using ?root instead of ?component when creating the component package from the application. Using ?root made a package that expected to install its own python, which is why it was trying to mess with the /System Python.Framework directory. Using ?component made things work. :) Hope may this will help someone someday. -EricF > On Dec 23, 2017, at 10:38 AM, Sean Wayland wrote: > > The other thing I remember is that tkinter is incompatible with py2app.. > > On Dec 23, 2017 10:21 AM, "Sean Wayland" > wrote: > I tried this once. My vague memory of it is that py2app only works with one version of Python. Did you try using Python 3 instead? You might be able to use JUCE to create a standalone application instead of py2app > > On Dec 22, 2017 4:52 PM, > wrote: > I hope someone on this list can help? I haven?t been able to find many answers on the web. > > Environment: Mac OS 10.8 & 10.11, Python 2.7.6, py2app 0.14 > > I create a Mac application bundle from python, using `py2app`, on Mac OS 10.8. This seems to work fine. I then build an installer package using `pkgbuild` and `productbuild`. That appears to build with no errors, until I try to use the installer to install the package. > > I'm running the installer on MacOS 10.11 (it will also have to run on MacOS 10.8-10.13). The installation fails, with the following in the install.log: > > Dec 21 15:14:46 Erics-10-11-Mac-VM shove[30757]: [src=nonrestricted] /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/pyconfig.h: unable to remove flag 0x80000 (error 1) > Dec 21 15:14:46 Erics-10-11-Mac-VM shove[30757]: [source=file] failed _RelinkFile(/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/C/PKInstallSandboxManager/3BE46873-2E16-4787-81A4-335FB1DCB67B.activeSandbox/Root/Applications/MyApp.app/Contents/Resources/include/python2.7/pyconfig.h, /System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7/pyconfig.h): Operation not permitted > Dec 21 15:14:46 Erics-10-11-Mac-VM shove[30757]: srcPath = /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/C/PKInstallSandboxManager/3BE46873-2E16-4787-81A4-335FB1DCB67B.activeSandbox/Root/Applications/MyApp.app/Contents/Resources/include/python2.7/pyconfig.h NSFileOwnerAccountID=0 NSFileHFSTypeCode=0 NSFileSystemFileNumber=1025119 NSFileExtensionHidden=0 NSFileSystemNumber=16777219 NSFileSize=36515 NSFileGroupOwnerAccountID=0 NSFileOwnerAccountName=root NSFileCreationDate=2017-12-21 20:12:33 +0000 NSFilePosixPermissions=420 NSFileHFSCreatorCode=0 NSFileType=NSFileTypeRegular NSFileGroupOwnerAccountName=wheel NSFileReferenceCount=1 NSFileModificationDate=2017-12-21 20:12:33 +0000 > > The issue seems to be that the installer wants to copy (or do something with) `Contents/Resources/include/pyconfig.h` into `/System/Library/Frameworks/Python.framework...` but that operation is not permitted. It's presumably not permitted due to the MacOS System Integrity Protection on `/System/`. > > This raises a couple of questions: > > 1. What exactly is the installer trying to do by copying `pyconfig.h` into a System directory (which is expressly prohibited?) Besides, there's already a `pyconfig.h` there. > 3. What can I do to fix this? Somehow tell `py2app` not to put `pyconfig.h` in the `include` folder? Tell the installer to not copy it into the `System` folder? Is there a workaround? Disabling `SIP` is not an option. > > Note that I'm constrained to build on MacOS 10.8, and install on arbitrary systems from MacOS 10.8 - 10.3. > > Thanks for your help! > > -EricF > > _______________________________________________ > Pythonmac-SIG maillist - Pythonmac-SIG at python.org > https://mail.python.org/mailman/listinfo/pythonmac-sig > unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shawn.tolidano at compass.com Wed Jan 17 12:42:14 2018 From: shawn.tolidano at compass.com (Shawn Tolidano) Date: Wed, 17 Jan 2018 12:42:14 -0500 Subject: [Pythonmac-SIG] Problem with pip Message-ID: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> Hi, I have cffi==1.11.4 in my requirements file (and in my transitive file, by extension). I am running High Sierra, my python is 2.7.14 and comes from Brew. /usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ I create a venv and upgrade pip (to 9) and then: ./build-support/python/venvs/pydeps-wheel/bin/pip install -r 3rdparty/python/requirements.transitive.txt In this file are lots of requirements, but that?s not relevant (I don?t think). One of them is cffi==1.11.4. When I run this, this happens: Collecting cffi==1.11.4 (from -r 3rdparty/python/requirements.transitive.txt (line 22)) Downloading cffi-1.11.4-cp27-cp27m-macosx_10_6_intel.whl (238kB) 100% |????????????????????????????????| 245kB 2.2MB/s Note the 10.6. Also note this: ./build-support/python/venvs/pydeps-wheel/bin/python -c "import distutils.util; print(distutils.util.get_platform())" macosx-10.13-x86_64 So - to my knowledge, it has NO BUSINESS using the 10.6 as a compatible wheel and should instead download the source. So I empty my requirements file and leave ONLY cffi==1.11.4 I recompute (with pip freeze) and end up with cffi, setuptools, wheel, and pycparser. I run the SAME install command as above. And get this: Collecting cffi==1.11.4 (from -r 3rdparty/python/requirements.transitive.txt (line 2)) Using cached cffi-1.11.4.tar.gz And ultimately, it builds this: Dist/wheels/cffi-1.11.4-cp27-none-macosx_10_13_x86_64.whl 1) Why does it think 10.6 is a valid wheel? 2) How do I fix this? Thanks a lot in advance for your help! ------------------------- Shawn Tolidano Senior Engineer - DevOps 90 Fifth Avenue 6th Floor New York, NY 10011 m: (516) 668-5448 NEW YORK | BROOKLYN | EAST HAMPTON | BRIDGEHAMPTON | SOUTHAMPTON | SAG HARBOR | WASHINGTON DC | CHEVY CHASE | BOSTON | CAMBRIDGE | MIAMI | COCONUT GROVE | BEVERLY HILLS | MALIBU | PASADENA | MONTECITO | SANTA BARBARA | BASALT | ASPEN | SAN FRANCISCO -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.orponen at 4teamwork.ch Thu Jan 18 08:58:10 2018 From: j.orponen at 4teamwork.ch (Joni Orponen) Date: Thu, 18 Jan 2018 14:58:10 +0100 Subject: [Pythonmac-SIG] Problem with pip In-Reply-To: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> References: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> Message-ID: On Wed, Jan 17, 2018 at 6:42 PM, Shawn Tolidano wrote: > > So - to my knowledge, it has NO BUSINESS using the 10.6 as a compatible > wheel and should instead download the source. > The offical CPython distributions seem to be compiled on 10.6 for compatibility reasons. It is easier to compile CPython forwards-compatible than to manually patch pyconfig.h after configure to exclude the syscalls not available on your older target platforms. For the people compiling wheels for distribution it would make sense to mirror CPython for their distributions for similar reasons. Is there some newer syscall path you absolutely need to go through (for reasons of sandboxing requirements? a specification requirement for using getentropy()?) or what is the actual problem you are trying to fix here? -- Joni Orponen -------------- next part -------------- An HTML attachment was scrubbed... URL: From karstenwo at googlemail.com Thu Jan 18 11:05:05 2018 From: karstenwo at googlemail.com (Karsten Wolf) Date: Thu, 18 Jan 2018 17:05:05 +0100 Subject: [Pythonmac-SIG] Problem with pip In-Reply-To: References: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> Message-ID: <61F9C40E-3B17-4DDE-98E4-924FE07C1591@googlemail.com> > On 18.01.2018, at 14:58, Joni Orponen wrote: > > The offical CPython distributions seem to be compiled on 10.6 for compatibility reasons. It is easier to compile CPython forwards-compatible than to manually patch pyconfig.h after configure to exclude the syscalls not available on your older target platforms. With the following config line you compile a backwards compatible Python (here 64-bit only since I don't like fiddling around with hard to compile fat-libs). # for 2.7 env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue and it yields a Python that can be py2apped and run on 10.6 as long as you don't use newer Frameworks or libs that need OSX > 1.6 -karsten From j.orponen at 4teamwork.ch Thu Jan 18 13:18:54 2018 From: j.orponen at 4teamwork.ch (Joni Orponen) Date: Thu, 18 Jan 2018 19:18:54 +0100 Subject: [Pythonmac-SIG] Problem with pip In-Reply-To: <61F9C40E-3B17-4DDE-98E4-924FE07C1591@googlemail.com> References: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> <61F9C40E-3B17-4DDE-98E4-924FE07C1591@googlemail.com> Message-ID: On Thu, Jan 18, 2018 at 5:05 PM, Karsten Wolf via Pythonmac-SIG < pythonmac-sig at python.org> wrote: > > > On 18.01.2018, at 14:58, Joni Orponen wrote: > > > > The offical CPython distributions seem to be compiled on 10.6 for > compatibility reasons. It is easier to compile CPython forwards-compatible > than to manually patch pyconfig.h after configure to exclude the syscalls > not available on your older target platforms. > > With the following config line you compile a backwards compatible Python > (here 64-bit only since I don't like fiddling around with hard to compile > fat-libs). > > # for 2.7 > env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch > x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue > > and it yields a Python that can be py2apped and run on 10.6 as long as you > don't use newer Frameworks or libs that need OSX > 1.6 > I'm not sharing your experience. Adding -Wl,-no_weak_imports to LDFLAGS flags things up and those then actually fail hard at runtime. The easiest one to spot and verify yourself is getentropy() when using TLS. I'm having to patch out the detected support for those syscalls manually out of the generated pyconfig.h when compiling something backwards-compatibly for distribution. I'm also passing through -mmacosx-version-min in both LDFLAGS and CFLAGS as not all egg specific build systems pass in the environment correctly, but there is still some hope of them to use whatever it is that Python was built with as Python exposes that at runtime. There are eggs one still has to patch up manually for distribution. This is also useful to be in the know of for some of the dependencies of any CPython modules you might have to bundle as static libraries for distribution purposes. For the framework incompatibilities you can get to about the same detection ability if you use -Wunguarded-availability in CFLAGS. There is also -Wno-unguarded-availability-new which combines to provide the ability for not yet caring of non-fatal issues for the newest one (as in the one you are currently building on with the intention of being backwards-compatible only), but this is rather new [1]. I do not recommend -Werror and -Wunguarded-availability together when compiling backwards-compatible as you'd have to fix the use of all the deprecated, but not yet broken, bits in CPython itself. So for targeting 10.6 I'd use something roughly like (also allowing you to switch between Xcode versions via xcode-select, if needed): CC="xcrun cc" CXX="$CC" MACOSX_DEPLOYMENT_TARGET="10.6" XCODE_ROOT="$(xcode-select --print-path)" SYSROOT="$XCODE_ROOT/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" COMMON_FLAGS="--sysroot=$SYSROOT -mmacosx-version-min=$MACOSX_DEPLOYMENT_TARGET" CFLAGS="-I$SYSROOT/usr/include $COMMON_FLAGS -Wno-unguarded-availability-new -Wunguarded-availability" CPPFLAGS="$CFLAGS" LDFLAGS="-L$SYSROOT/usr/lib $COMMON_FLAGS -Wl,-no_weak_imports" In practice I'd not target such an old OS version when building backwards-compatibly as the associated workload would be rather large. Building on the latest version and supporting a couple of versions back down is still rather manageable and sane in my experience. Against this backdrop, I'm understanding of people, especially open source volunteers, choosing to build both CPython and their distributable wheels on 10.6 as the amount of effort to validate and support more permutations is rather large for almost no benefit. The expectation levels on the Windows side have risen and thus since Python 3.5 that stack has moved to build with a newer Visual Studio and is also offering a nicer userspace install. I'm taking this thread as a signal of the expectations of the macOS crowd having risen as well. A midway compromise point might be to coerce the community to start building on 10.11 as 10.12 was again a great hardware support divider such as 10.7 was and it's starting to have been long enough since 10.11. On the actual topic: If one explicitly needs some build-time details to be specific, one can just build the egg oneself. Otherwise whatever pip pulls in and is working is good enough for the job. -- Joni Orponen [1] https://reviews.llvm.org/D34264 -------------- next part -------------- An HTML attachment was scrubbed... URL: From pythonchb at gmail.com Thu Jan 18 20:22:15 2018 From: pythonchb at gmail.com (Christopher Barker) Date: Fri, 19 Jan 2018 01:22:15 +0000 Subject: [Pythonmac-SIG] Problem with pip In-Reply-To: References: <38EA7B97-CE3B-40A9-8C9C-52F3FBE9FFD7@compass.com> <61F9C40E-3B17-4DDE-98E4-924FE07C1591@googlemail.com> Message-ID: This would be a very good time to go over to Python-dev and see what?s being done for 3.7. And to help Ned if you want to see changes. -CHB On Thu, Jan 18, 2018 at 10:19 AM Joni Orponen wrote: > On Thu, Jan 18, 2018 at 5:05 PM, Karsten Wolf via Pythonmac-SIG < > pythonmac-sig at python.org> wrote: > >> >> > On 18.01.2018, at 14:58, Joni Orponen wrote: >> > >> > The offical CPython distributions seem to be compiled on 10.6 for >> compatibility reasons. It is easier to compile CPython forwards-compatible >> than to manually patch pyconfig.h after configure to exclude the syscalls >> not available on your older target platforms. >> >> With the following config line you compile a backwards compatible Python >> (here 64-bit only since I don't like fiddling around with hard to compile >> fat-libs). >> >> # for 2.7 >> env MACOSX_DEPLOYMENT_TARGET=10.6 CFLAGS="-arch x86_64 " LDFLAGS="-arch >> x86_64 " ./configure --enable-framework --enable-ipv6 --enable-toolbox-glue >> >> and it yields a Python that can be py2apped and run on 10.6 as long as >> you don't use newer Frameworks or libs that need OSX > 1.6 >> > > I'm not sharing your experience. > > Adding -Wl,-no_weak_imports to LDFLAGS flags things up and those then > actually fail hard at runtime. The easiest one to spot and verify yourself > is getentropy() when using TLS. I'm having to patch out the detected > support for those syscalls manually out of the generated pyconfig.h when > compiling something backwards-compatibly for distribution. > > I'm also passing through -mmacosx-version-min in both LDFLAGS and CFLAGS > as not all egg specific build systems pass in the environment correctly, > but there is still some hope of them to use whatever it is that Python was > built with as Python exposes that at runtime. There are eggs one still has > to patch up manually for distribution. This is also useful to be in the > know of for some of the dependencies of any CPython modules you might have > to bundle as static libraries for distribution purposes. > > For the framework incompatibilities you can get to about the same > detection ability if you use -Wunguarded-availability in CFLAGS. There is > also -Wno-unguarded-availability-new which combines to provide the ability > for not yet caring of non-fatal issues for the newest one (as in the one > you are currently building on with the intention of being > backwards-compatible only), but this is rather new [1]. > > I do not recommend -Werror and -Wunguarded-availability together when > compiling backwards-compatible as you'd have to fix the use of all the > deprecated, but not yet broken, bits in CPython itself. > > So for targeting 10.6 I'd use something roughly like (also allowing you to > switch between Xcode versions via xcode-select, if needed): > > CC="xcrun cc" > CXX="$CC" > > MACOSX_DEPLOYMENT_TARGET="10.6" > > XCODE_ROOT="$(xcode-select --print-path)" > SYSROOT="$XCODE_ROOT/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" > > COMMON_FLAGS="--sysroot=$SYSROOT > -mmacosx-version-min=$MACOSX_DEPLOYMENT_TARGET" > > CFLAGS="-I$SYSROOT/usr/include $COMMON_FLAGS > -Wno-unguarded-availability-new -Wunguarded-availability" > CPPFLAGS="$CFLAGS" > > LDFLAGS="-L$SYSROOT/usr/lib $COMMON_FLAGS -Wl,-no_weak_imports" > > In practice I'd not target such an old OS version when building > backwards-compatibly as the associated workload would be rather large. > Building on the latest version and supporting a couple of versions back > down is still rather manageable and sane in my experience. > > Against this backdrop, I'm understanding of people, especially open source > volunteers, choosing to build both CPython and their distributable wheels > on 10.6 as the amount of effort to validate and support more permutations > is rather large for almost no benefit. > > The expectation levels on the Windows side have risen and thus since > Python 3.5 that stack has moved to build with a newer Visual Studio and is > also offering a nicer userspace install. I'm taking this thread as a signal > of the expectations of the macOS crowd having risen as well. > > A midway compromise point might be to coerce the community to start > building on 10.11 as 10.12 was again a great hardware support divider such > as 10.7 was and it's starting to have been long enough since 10.11. > > On the actual topic: If one explicitly needs some build-time details to be > specific, one can just build the egg oneself. Otherwise whatever pip pulls > in and is working is good enough for the job. > > -- > Joni Orponen > > [1] https://reviews.llvm.org/D34264 > _______________________________________________ > Pythonmac-SIG maillist - Pythonmac-SIG at python.org > https://mail.python.org/mailman/listinfo/pythonmac-sig > unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuomov at iki.fi Sat Jan 20 21:17:50 2018 From: tuomov at iki.fi (Tuomo Valkonen) Date: Sun, 21 Jan 2018 02:17:50 +0000 Subject: [Pythonmac-SIG] Unmodified environment for subprocesses Message-ID: <6DA7B747-AE9A-4277-B40E-23F74789AD3A@iki.fi> I?m working on a frontend to Borg backup, based on rumps (http://rumps.readthedocs.io/en/latest/ ). For using the keychain, etc., it would be useful to turn the frontend into a standalone app. So I tried to do that with py2app, but when running my app this way, borg subprocesses fail: module ?borg? not found. If I pass a modified environment to subprocess.Popen, as follow, then things will work: env=os.environ.copy() del env['PYTHONPATH'] del env['PYTHONHOME'] But of course this also will fail on system where the variables are set in the default environment. My question then is, is there a clean way to restore the original environment, before py2app changed it? Tuomo -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuomov at iki.fi Sat Jan 20 20:48:03 2018 From: tuomov at iki.fi (Tuomo Valkonen) Date: Sun, 21 Jan 2018 01:48:03 +0000 Subject: [Pythonmac-SIG] Unmodified environment for subprocesses Message-ID: I?m working on a frontend to Borg backup, based on rumps (http://rumps.readthedocs.io/en/latest/ ). For using the keychain, etc., it would be useful to turn the frontend into a standalone app. So I tried to do that with py2app, but when running my app this way, borg subprocesses fail: module ?borg? not found. If I pass a modified environment to subprocess.Popen, as follow, then things will work: env=os.environ.copy() del env['PYTHONPATH'] del env['PYTHONHOME'] But of course this also will fail on system where the variables are set in the default environment. My question then is, is there a clean way to restore the original environment, before py2app changed it? Tuomo -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at barrys-emacs.org Mon Jan 22 14:16:25 2018 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 22 Jan 2018 19:16:25 +0000 Subject: [Pythonmac-SIG] Unmodified environment for subprocesses In-Reply-To: References: Message-ID: <7167D165-2EF0-487E-9E27-44F033D6A2C3@barrys-emacs.org> > On 21 Jan 2018, at 01:48, Tuomo Valkonen wrote: > > I?m working on a frontend to Borg backup, based on rumps (http://rumps.readthedocs.io/en/latest/ ). For using the keychain, etc., it would be useful to turn the frontend into a standalone app. So I tried to do that with py2app, but when running my app this way, borg subprocesses fail: module ?borg? not found. If I pass a modified environment to subprocess.Popen, as follow, then things will work: > > env=os.environ.copy() > del env['PYTHONPATH'] > del env['PYTHONHOME'] That is what I do. There might be more env to remove, depends on the details of you app code. > > But of course this also will fail on system where the variables are set in the default environment. I would not expect them in the default env. Only in the env setup by .bash_profile that you do not see in an App I recall. > My question then is, is there a clean way to restore the original environment, before py2app changed it? If you want to find out the details of the env you start with edit your Appto move its executable (named Info.plist) to one side and put a bash script in its place that records the command line and env into a file for you to look at. Then call you renamed executable as the last thing from the script. Your exe is in .app/Contents/MacOS/. Barry > > Tuomo > > _______________________________________________ > Pythonmac-SIG maillist - Pythonmac-SIG at python.org > https://mail.python.org/mailman/listinfo/pythonmac-sig > unsubscribe: https://mail.python.org/mailman/options/Pythonmac-SIG -------------- next part -------------- An HTML attachment was scrubbed... URL: