[IronPython] change in standard library behavior for compiled .exe/.dll???

Thu Oct 7 17:05:40 CEST 2010

I checked my 2.6 and 2.7 systems; neither has "os" as a builtin module, and
os has a lot more members than 'nt' on both, and seem to be the same lists
on both versions of IP.

In one of my previous notes in this thread, I listed a 'diff' of pyc.py from
the 2.6 to 2.7 versions; does anyone know why the changes were made, and,
more importantly, WHAT the effect of the changes is? I tried compiling on my
2.7 system, using the pyc.py from 2.6 and the .exe coughed up an exception,
but wasn't able to make much of it, but wondering if the changes in pyc.py
may be affecting what the compile module outputs?
Ken

On Wed, Oct 6, 2010 at 5:59 PM, Dino Viehland <dinov at microsoft.com> wrote:

>  Hurm, will __file__ will still be set even for pre-compiled DLLs so it’s
> not picking up a pre-compiled version.  What about sys.builtin_module_names
> on 2.6?  Does it have os listed in there somehow?  And does os have the same
> members at the nt module or does it have significantly more?  For example if
> I  do “set(dir(os)) - set(dir(nt))” there’s 40 more members in os (os just
> builds on top of the underlying nt/unix/other platform built-in module so if
> these are getting aliased somehow then os would have the same members).
>
>
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Ken MacDonald
> *Sent:* Wednesday, October 06, 2010 2:28 PM
>
> *To:* Discussion of IronPython
> *Subject:* Re: [IronPython] change in standard library behavior for
> compiled .exe/.dll???
>
>
>
> Funny, we did exactly that yesterday. When we run the .exe on the new
> version (2.7), we get a sys.path of [".", "...\current_dir\DLLs",
> "...\current_dir\Lib"], and the os.__file__ was "...\current_dir\os.py" -
> where I happened to have put os.py. We rebuilt our 2.6 app, and sys.path was
> the same, but os.__file__ was just plain "os" - no file path listed. Almost
> as if it was built into the application DLL, which is where we think it used
> to be....
> Ken
>
> On Wed, Oct 6, 2010 at 5:15 PM, Dino Viehland <dinov at microsoft.com> wrote:
>
> AFAIK nothing has changed between 2.7 and 2.6  w/ how we’d go about finding
> the standard library – and we certainly never had anything which would
> automatically include it.  Do you have something which works on 2.6 where
> you could print out os.__file__ (or some other module you’re importing from
> the std lib) and sys.path?  That’d at least tell us where/how we’re picking
> these up from in the previous version of the app and maybe we can work out
> what’s going on from there.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Ken MacDonald
> *Sent:* Wednesday, October 06, 2010 1:05 PM
> *To:* Discussion of IronPython
>
>
> *Subject:* Re: [IronPython] change in standard library behavior for
> compiled .exe/.dll???
>
>
>
> Hi Dino,
>
> On Wed, Oct 6, 2010 at 2:57 PM, Dino Viehland <dinov at microsoft.com> wrote:
>
> How are you distributing your app?  I’m assuming you’re going to have
> something like:
>
>
>
> MyApp\
>
> MyApp.exe
>
> MyApp.dll
>
> IronPython.dll
>
> IronPython.Modules.DLL
>
> …
>
>
>
> This is exactly how we have been distributing our app up to IP 2.6 with
> .NET 3.5. We also have another DLL with our resources in it; XAML, icons,
> image files, but no code per se. We have been distributing it this way to
> customer systems that do NOT have IronPython installed, or any sign of the
> std. library, or "os.py" specifically, and it's been working really, really
> well.
>
> We're trying to understand what changed moving to IP 2.7 and .NET 4.0 that
> we should have to care about distributing the std. library now. The way it
> was before was quite simple and robust; deliver a small handful of files and
> it just worked, very easy to keep track of. Now instead of distributing 5
> files, we suddenly need to distribute 500??? We've figured out that it
> certainly COULD work as described below, but it seems like a giant step
> backwards on several fronts, including the potential for folks to
> maliciously or accidentally tamper with the std lib. sources and affect the
> functioning of our app. So, how do we get back to the old/better
> functionality?
>
> On a *slightly *related note, our app imports some package directories in
> addition, e.g. "import ctypes". When python encounters a directory import,
> it looks for __init__.py in the directory, and derives the package import
> directions from there, as I understand it. However, I can't specify the
> "ctypes" directory as an argument to the pyc.py compile app; just causes it
> to croak. If I explicitly specified paths like "lib\ctypes\__init__.py" and
> the other files in the ctypes subdirectory, it seems like "import ctypes"
> would have no clue that the __init__.py that was compiled in had anything to
> do with the "ctypes" package, as the path names are presumably irrelevant to
> the compiler as long as they specify a python file. I'm considering mod'ing
> pyc.py to be able to incorporate a list of std lib modules to compile in:
> simple enough for the standalone files like "os.py", but the compile modules
> don't seem to be able to grok what to do with a package subdirectory.
> Ken
>
>
>
>
>
>
> You should be able to also distribute the standard library and just drop it
> into a Lib directory next to IronPython.dll:
>
> MyApp\
>
> MyApp.exe
>
> MyApp.dll
>
> IronPython.dll
>
> IronPython.Modules.DLL
>
> …
>
>                 Lib\
>
>                                 os.py
>
>
>
> That lib dir should be on sys.path at startup and so it should be available
> for importing.
>
>
>
> *From:* users-bounces at lists.ironpython.com [mailto:
> users-bounces at lists.ironpython.com] *On Behalf Of *Ken MacDonald
> *Sent:* Wednesday, October 06, 2010 11:42 AM
> *To:* Michael Foord
>
>
> *Cc:* Discussion of IronPython
> *Subject:* Re: [IronPython] change in standard library behavior for
> compiled .exe/.dll???
>
>
>
> Hi Michael,
>
>
> I started out on implementing this, but I am importing maybe a dozen of the
> std. library modules, which then import others, and so on. It appears that
> eventually, most of the std modules would have to be imported explicitly
> (perhaps 400 or so files) which might make for a somewhat cumbersome command
> line, incidentally also about 20K characters too long :-). I'm hoping to
> find a way to get this to work as well as it did under IP 2.5 / .NET 3.5.
>
> Noah: what kind of problems are YOU having with pyc.py under 4.0? Maybe one
> of us can suggest something if we have an understanding of what you're
> trying to do.
> Ken
>
> On Wed, Oct 6, 2010 at 7:08 AM, Michael Foord <michael at voidspace.org.uk>
> wrote:
>
> On 05/10/2010 22:27, Ken MacDonald wrote:
>
> I've been looking at the .exe's we built - using pyc.py - with IP 2.5/.Net
> 3.5 and IP2.7 / .NET 4.0. In the 2.7 .exe, it appears that the imports (like
> "os") are not being built into the .exe/.dll, but instead are required to be
> imported in source form, e.g. "os.py" must be somewhere on sys.path. In the
> IP 2.5 .exe's we had been building, they would run fine on machines without
> the IP standard library installed at all, in other words, with "os.py" not
> present on the machine at all. We did notice that the .exe in question went
> from being 2.9 MB in it's IP 2.5 incarnation, down to 1.2 MB in the IP 2.7
> version, and the newer version requires that the source code for the IP
> standard library be on the path. Is this a deliberate change in behavior? We
> never had to package the standard library source when we sent out .exe's to
> customers before
>
>
>
> Hmmm... I'm pretty sure I always had to explicitly compile and bundle the
> standard library with previous versions of Python. Odd. Anyway, the simple
> solution is to ensure that you add any standard library modules you use to
> the set you compile and ship.
>
>
>
> All the best,
>
> Michael Foord
>
>
> >"os" is not an assembly but a Python module from the standard library. You
> need to ensure >that the Python standard library (or the parts that you use
> and their dependencies) is on the >path.
>
>
> All the best,
>
> Michael Foord
>
> and how do I ensure it gets found from my .exe - is there a specific env.
> variable, or the Windows %PATH% e.v., or something I haven't AddReference'd
> to????
> Thanks,
> Ken
>
>
>
> _______________________________________________
>
> Users mailing list
>
>  Users at lists.ironpython.com
>
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
> --
>
> http://www.voidspace.org.uk/blog
>
>
>
> READ CAREFULLY. By accepting and reading this email you agree,
>
> on behalf of your employer, to release me from all obligations
>
> and waivers arising from any and all NON-NEGOTIATED agreements,
>
> licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
>
> confidentiality, non-disclosure, non-compete and acceptable use
>
> policies (”BOGUS AGREEMENTS”) that I have entered into with your
>
> employer, its partners, licensors, agents and assigns, in
>
> perpetuity, without prejudice to my ongoing rights and privileges.
>
> You further represent that you have the authority to release me
>
> from any BOGUS AGREEMENTS on behalf of your employer.
>
>
>
>
>
> --
>
> http://www.voidspace.org.uk/blog
>
>
>
> READ CAREFULLY. By accepting and reading this email you agree,
>
> on behalf of your employer, to release me from all obligations
>
> and waivers arising from any and all NON-NEGOTIATED agreements,
>
> licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
>
> confidentiality, non-disclosure, non-compete and acceptable use
>
> policies (”BOGUS AGREEMENTS”) that I have entered into with your
>
> employer, its partners, licensors, agents and assigns, in
>
> perpetuity, without prejudice to my ongoing rights and privileges.
>
> You further represent that you have the authority to release me
>
> from any BOGUS AGREEMENTS on behalf of your employer.
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20101007/2132f84c/attachment.html>