From ncoghlan at gmail.com Fri May 19 06:24:58 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 19 May 2017 20:24:58 +1000 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: <1495111849.11457.2.camel@localhost> References: <1495111849.11457.2.camel@localhost> Message-ID: On 18 May 2017 at 22:50, wrote: > Greetings, > > This has been already sent to python-ideas, but since I got no > response, so I'm re-sending it to this SIG. I would welcome any > comments. Sorry about that - I suspect you caught a lot of other folks in the middle of getting ready for PyCon travel, and I put it aside to have a closer look when I had more time. > I'm a student that has been working lately on feature of the runpy > module that I have been quite interested in: execution of extension > modules using the -m switch. This is very cool, and one of the things we were hoping to enable with the multi-phase initialisation changes :) > Currently this requires access to the module's code, so it only works > for modules written in Python. > I have a proof-of-concept implementation that adds a new > ExtensionFileLoader method called "exec_as_main". > The runpy module then checks if the loader has this method, and if so, > calls it instead of getting the the code and running that. > > This new method calls into the _imp module, which executes the module > as a script. > I can see two ways of doing this. Both expect that the module uses PEP > 489 multi-phase initialization. The main reason I didn't immediately reply is that I had a vague recollection of thinking this could be done *without* a new method on loaders, but I needed to refresh my memory of our plans in that regard. I've now done that, and I'm pretty sure the unwritten plan was to change runpy to do something like the following: spec = importlib.find_spec(modname) created = spec.loader.create_module() if created is not None: raise RuntimeError("Cannot use customised module instance as __main__") spec.loader.exec_module(main_mod) That's oversimplified quite a bit, but it gives the general idea. > The first way is having a new PyModuleDef_Slot called Py_mod_main, > which names a function to execute when run as main. > > The second way is running a module's Py_mod_exec inside the __main__ > module's namespace, as it's done for normal modules. > The module would then do a `if __name__ == "__main__"` check. > This is possible for modules that don't define Py_mod_create: they > expect a default module object to be created for them, so we can pass > the __main__ module to their Py_mod_exec function. > This way would mean that, for example, modules written in Cython would > behave like their Python counterparts. And that's *precisely* the idea behind allowing this to work with existing loaders, as long as they return None from create_module(). > Another possibility would be to use both, allowing both easy Cython- > style modules and a dedicated slot for modules that need custom > Py_mod_create. I'm OK with continuing to have cases like the latter rely on a helper module that imports the one that needs a custom module instance. Given PEP 489, those can even be defined in the same shared library: https://www.python.org/dev/peps/pep-0489/#multiple-modules-in-one-library Independently of extension module initialisation, I also have an idea for taking another go at providing "autorun" capabilities for main modules, where defining a dunder functions with a particular name will execute it after __main__ finishes running (I'll do a separate post about that). > My proof of concept uses another combination: it requires Py_mod_main > and runs it in the __main__ namespace. But that can change based on > discussion here. > > Link to the implementation: https://github.com/Traceur759/cpython/tree/ > main_c_modules > Diff from master: https://github.com/python/cpython/compare/master...Tr > aceur759:main_c_modules > > You can quickly test it with: > $ ./python -m _testmultiphase > This is an extension module named __main__ Once again, very cool! If we don't have one already, could you file a 3.7 RFE for this on bugs.python.org? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From encukou at gmail.com Fri May 19 07:43:19 2017 From: encukou at gmail.com (Petr Viktorin) Date: Fri, 19 May 2017 13:43:19 +0200 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: References: <1495111849.11457.2.camel@localhost> Message-ID: <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> On 05/19/2017 12:24 PM, Nick Coghlan wrote: > On 18 May 2017 at 22:50, wrote: >> Greetings, >> >> This has been already sent to python-ideas, but since I got no >> response, so I'm re-sending it to this SIG. I would welcome any >> comments. > ... >> >> This new method calls into the _imp module, which executes the module >> as a script. >> I can see two ways of doing this. Both expect that the module uses PEP >> 489 multi-phase initialization. > > The main reason I didn't immediately reply is that I had a vague > recollection of thinking this could be done *without* a new method on > loaders, but I needed to refresh my memory of our plans in that > regard. > > I've now done that, and I'm pretty sure the unwritten plan was to > change runpy to do something like the following: > > spec = importlib.find_spec(modname) > created = spec.loader.create_module() > if created is not None: > raise RuntimeError("Cannot use customised module instance as __main__") > spec.loader.exec_module(main_mod) > > That's oversimplified quite a bit, but it gives the general idea. The problem here is that for extension modules, `spec.loader.create_module()` returns None. It can't: the PyModuleDef is attached to the returned module, and that's where the Py_mod_exec function is stored. This is unlike with source modules, where the code is always looked up by module name. So I see these ways to make things work: - Make spec.loader.create_module() return None if Py_mod_create is missing, and either store the PyModuleDef on the loader (which doesn't really fit in with how importlib works), or re-load it from the .so every time (which seems wasteful and hacky). - Make exec_module take two modules ? the module in whose namespace to run, and the module whose code should run. Or make it take a module and a spec of a different module. This would be an API change, affecting all third-party loaders, so it's out. - Add a new loader method taking two modules (or module and spec) as above - Add a new loader method to explicitly run as main From ncoghlan at gmail.com Sat May 20 00:36:49 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 20 May 2017 14:36:49 +1000 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> References: <1495111849.11457.2.camel@localhost> <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> Message-ID: On 19 May 2017 at 21:43, Petr Viktorin wrote: > On 05/19/2017 12:24 PM, Nick Coghlan wrote: >> >> On 18 May 2017 at 22:50, wrote: >>> >>> Greetings, >>> >>> This has been already sent to python-ideas, but since I got no >>> response, so I'm re-sending it to this SIG. I would welcome any >>> comments. >> >> > ... >>> >>> >>> This new method calls into the _imp module, which executes the module >>> as a script. >>> I can see two ways of doing this. Both expect that the module uses PEP >>> 489 multi-phase initialization. >> >> >> The main reason I didn't immediately reply is that I had a vague >> recollection of thinking this could be done *without* a new method on >> loaders, but I needed to refresh my memory of our plans in that >> regard. >> >> I've now done that, and I'm pretty sure the unwritten plan was to >> change runpy to do something like the following: >> >> spec = importlib.find_spec(modname) >> created = spec.loader.create_module() >> if created is not None: >> raise RuntimeError("Cannot use customised module instance as >> __main__") >> spec.loader.exec_module(main_mod) >> >> That's oversimplified quite a bit, but it gives the general idea. > > > The problem here is that for extension modules, > `spec.loader.create_module()` returns None. I'm guessing this was meant to be "doesn't return None". I thought I was forgetting something, and that would be it :) > It can't: the PyModuleDef is > attached to the returned module, and that's where the Py_mod_exec function > is stored. This is unlike with source modules, where the code is always > looked up by module name. > > So I see these ways to make things work: > - Make spec.loader.create_module() return None if Py_mod_create is missing, > and either store the PyModuleDef on the loader (which doesn't really fit in > with how importlib works), or re-load it from the .so every time (which > seems wasteful and hacky). > - Make exec_module take two modules ? the module in whose namespace to run, > and the module whose code should run. Or make it take a module and a spec of > a different module. This would be an API change, affecting all third-party > loaders, so it's out. > - Add a new loader method taking two modules (or module and spec) as above > - Add a new loader method to explicitly run as main As a third variant on the last two options: add a new optional "exec_in_namespace" method - that could potentially be useful for generalising reload and lazy loading support, as well as making it easier for pdb, profile, etc, to support non-traditional modules. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From encukou at gmail.com Mon May 22 05:33:50 2017 From: encukou at gmail.com (Petr Viktorin) Date: Mon, 22 May 2017 11:33:50 +0200 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: References: <1495111849.11457.2.camel@localhost> <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> Message-ID: <55aa2054-baae-5b2c-dd3b-3eb5d395a683@gmail.com> On 05/20/2017 06:36 AM, Nick Coghlan wrote: > On 19 May 2017 at 21:43, Petr Viktorin wrote: >> On 05/19/2017 12:24 PM, Nick Coghlan wrote: >>> >>> On 18 May 2017 at 22:50, wrote: >>>> >>>> Greetings, >>>> >>>> This has been already sent to python-ideas, but since I got no >>>> response, so I'm re-sending it to this SIG. I would welcome any >>>> comments. >>> >>> >> ... >>>> >>>> >>>> This new method calls into the _imp module, which executes the module >>>> as a script. >>>> I can see two ways of doing this. Both expect that the module uses PEP >>>> 489 multi-phase initialization. >>> >>> >>> The main reason I didn't immediately reply is that I had a vague >>> recollection of thinking this could be done *without* a new method on >>> loaders, but I needed to refresh my memory of our plans in that >>> regard. >>> >>> I've now done that, and I'm pretty sure the unwritten plan was to >>> change runpy to do something like the following: >>> >>> spec = importlib.find_spec(modname) >>> created = spec.loader.create_module() >>> if created is not None: >>> raise RuntimeError("Cannot use customised module instance as >>> __main__") >>> spec.loader.exec_module(main_mod) >>> >>> That's oversimplified quite a bit, but it gives the general idea. >> >> >> The problem here is that for extension modules, >> `spec.loader.create_module()` returns None. > > I'm guessing this was meant to be "doesn't return None". I thought I > was forgetting something, and that would be it :) > >> It can't: the PyModuleDef is >> attached to the returned module, and that's where the Py_mod_exec function >> is stored. This is unlike with source modules, where the code is always >> looked up by module name. >> >> So I see these ways to make things work: >> - Make spec.loader.create_module() return None if Py_mod_create is missing, >> and either store the PyModuleDef on the loader (which doesn't really fit in >> with how importlib works), or re-load it from the .so every time (which >> seems wasteful and hacky). >> - Make exec_module take two modules ? the module in whose namespace to run, >> and the module whose code should run. Or make it take a module and a spec of >> a different module. This would be an API change, affecting all third-party >> loaders, so it's out. >> - Add a new loader method taking two modules (or module and spec) as above >> - Add a new loader method to explicitly run as main > > As a third variant on the last two options: add a new optional > "exec_in_namespace" method - that could potentially be useful for > generalising reload and lazy loading support, as well as making it > easier for pdb, profile, etc, to support non-traditional modules. That won't be possible, since Py_mod_exec expects a module argument. The main extra thing a module has in addition to its namespace dict is the C-level module state (which I don't think is handled in the current PoC ? Marcel, can you add that?) In the face of C module state, I think asking extension authors to always handle reloading correctly is too much. But the other use cases should be possible. From encukou at gmail.com Mon May 22 10:51:55 2017 From: encukou at gmail.com (Petr Viktorin) Date: Mon, 22 May 2017 16:51:55 +0200 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: <55aa2054-baae-5b2c-dd3b-3eb5d395a683@gmail.com> References: <1495111849.11457.2.camel@localhost> <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> <55aa2054-baae-5b2c-dd3b-3eb5d395a683@gmail.com> Message-ID: <8d0c1856-48f6-779b-e6c3-b139f5ad50f2@gmail.com> On 05/22/2017 11:33 AM, Petr Viktorin wrote: > On 05/20/2017 06:36 AM, Nick Coghlan wrote: >> On 19 May 2017 at 21:43, Petr Viktorin wrote: >>> On 05/19/2017 12:24 PM, Nick Coghlan wrote: >>>> >>>> On 18 May 2017 at 22:50, wrote: >>>>> >>>>> Greetings, >>>>> >>>>> This has been already sent to python-ideas, but since I got no >>>>> response, so I'm re-sending it to this SIG. I would welcome any >>>>> comments. >>>> >>>> >>> ... >>>>> >>>>> >>>>> This new method calls into the _imp module, which executes the module >>>>> as a script. >>>>> I can see two ways of doing this. Both expect that the module uses PEP >>>>> 489 multi-phase initialization. >>>> >>>> >>>> The main reason I didn't immediately reply is that I had a vague >>>> recollection of thinking this could be done *without* a new method on >>>> loaders, but I needed to refresh my memory of our plans in that >>>> regard. >>>> >>>> I've now done that, and I'm pretty sure the unwritten plan was to >>>> change runpy to do something like the following: >>>> >>>> spec = importlib.find_spec(modname) >>>> created = spec.loader.create_module() >>>> if created is not None: >>>> raise RuntimeError("Cannot use customised module instance as >>>> __main__") >>>> spec.loader.exec_module(main_mod) >>>> >>>> That's oversimplified quite a bit, but it gives the general idea. >>> >>> >>> The problem here is that for extension modules, >>> `spec.loader.create_module()` returns None. >> >> I'm guessing this was meant to be "doesn't return None". I thought I >> was forgetting something, and that would be it :) >> >>> It can't: the PyModuleDef is >>> attached to the returned module, and that's where the Py_mod_exec >>> function >>> is stored. This is unlike with source modules, where the code is always >>> looked up by module name. >>> >>> So I see these ways to make things work: >>> - Make spec.loader.create_module() return None if Py_mod_create is >>> missing, >>> and either store the PyModuleDef on the loader (which doesn't really >>> fit in >>> with how importlib works), or re-load it from the .so every time (which >>> seems wasteful and hacky). >>> - Make exec_module take two modules ? the module in whose namespace >>> to run, >>> and the module whose code should run. Or make it take a module and a >>> spec of >>> a different module. This would be an API change, affecting all >>> third-party >>> loaders, so it's out. >>> - Add a new loader method taking two modules (or module and spec) as >>> above >>> - Add a new loader method to explicitly run as main >> >> As a third variant on the last two options: add a new optional >> "exec_in_namespace" method - that could potentially be useful for >> generalising reload and lazy loading support, as well as making it >> easier for pdb, profile, etc, to support non-traditional modules. > > That won't be possible, since Py_mod_exec expects a module argument. > The main extra thing a module has in addition to its namespace dict is > the C-level module state (which I don't think is handled in the current > PoC ? Marcel, can you add that?) > In the face of C module state, I think asking extension authors to > always handle reloading correctly is too much. But the other use cases > should be possible. > Marcel had to leave for the day. To prevent losing a PyCon sprint day, I've fixed up his latest PoC and pushed it here: Branch: https://github.com/encukou/cpython/tree/main_c_modules_namespace Diff: https://github.com/encukou/cpython/compare/master...encukou:main_c_modules_namespace?expand=1 These changes get rid of Py_mod_main, and add an optional exec_in_module method to loaders. This method initializes a given module using a given spec. Does this approach look good? From ncoghlan at gmail.com Mon May 22 21:19:27 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 23 May 2017 11:19:27 +1000 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: <8d0c1856-48f6-779b-e6c3-b139f5ad50f2@gmail.com> References: <1495111849.11457.2.camel@localhost> <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> <55aa2054-baae-5b2c-dd3b-3eb5d395a683@gmail.com> <8d0c1856-48f6-779b-e6c3-b139f5ad50f2@gmail.com> Message-ID: On 23 May 2017 at 00:51, Petr Viktorin wrote: > On 05/22/2017 11:33 AM, Petr Viktorin wrote: >> That won't be possible, since Py_mod_exec expects a module argument. >> The main extra thing a module has in addition to its namespace dict is the >> C-level module state (which I don't think is handled in the current PoC ? >> Marcel, can you add that?) >> In the face of C module state, I think asking extension authors to always >> handle reloading correctly is too much. But the other use cases should be >> possible. That restriction makes sense to me. > Marcel had to leave for the day. To prevent losing a PyCon sprint day, I've > fixed up his latest PoC and pushed it here: > > Branch: https://github.com/encukou/cpython/tree/main_c_modules_namespace > Diff: > https://github.com/encukou/cpython/compare/master...encukou:main_c_modules_namespace?expand=1 > > These changes get rid of Py_mod_main, and add an optional exec_in_module > method to loaders. This method initializes a given module using a given > spec. > > Does this approach look good? +1 from me (my only comments relate to the fact that some of the helper functions descriptions in runpy will need updating), and I think it retains forward compatibity with an "autorun" idea I came up with during an email discussion with Brandon Rhodes a few months back. While the latter is still an entirely speculative concept, I'll post a thread about it so you can double-check that we're still leaving that possibility open for the future. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From ncoghlan at gmail.com Mon May 22 22:02:34 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 23 May 2017 12:02:34 +1000 Subject: [Import-SIG] Idea: Autorun functionality for Python modules (redux) Message-ID: (Note: Posting to import-sig as this isn't something I'm actively planning to pursue myself any time soon, but I want to ensure we don't accidentally block this possibility while working on the proposal to make it possible to run extension modules as Python scripts. ) PEP 299 is an old rejected PEP proposing a special "__main__()" function for Python modules: https://www.python.org/dev/peps/pep-0299/ Three main points were cited in its rejection: - the name clash with "import __main__" - the lack of a clear strategy for supporting both newer versions of Python that supported automatic execution of a suitably named function as well as older versions that required an "if __name__ == '__main__':" block - the status quo wasn't seen as particular broken and "it would be more familiar to C/C++ programmers" wasn't a compelling argument for adding a second way to do it In an email discussion with Brandon Rhodes a few months back, he lamented the apparent intransigence of the core developers on this front, and I pointed out that nobody had ever actually made a follow-up proposal that specifically addressed the rationale applied in rejecting PEP 299, and put together a sketch of what such a proposal might look like. The first two technical points can be handled by: 1. Using `__run__` as the special function name 2. Setting "__main__.__autorun__ = True" prior to main module execution, and allowing a module to delete it or set `__autorun__ = False` to turn off the default autorun behaviour With those two special attributes defined, the autorun protocol would be: if getattr(main_module, "__autorun__", False): try: runmain = main_module.__run__ except AttributeError: pass else: import sys sys.exit(runmain(sys.argv) Scripts that want to optionally invoke "__run__" explicitly for compatibility with older Python versions can then check "__autorun__" to see whether or not they need to start the application themselves: if __name__ == "__main__" and not globals().get("__autorun__"): import sys sys.exit(__run__(sys.argv)) With the technical objections handled, we can then ask what concrete benefits a `def __run__(argv):` model might offer over the existing `if name == "__main__":` model: def __run__(argv): """CLI documentation goes here""" return 0 1. __run__ can go at the *top* of the script, rather than at the end, giving a conventional "CLI function followed by supporting definitions" structure 2. you gain access to sys.argv without having to import sys (testing & REPL friendly!) 3. you can set the process return code without having to call sys.exit (testing & REPL friendly!) 4. you can attach CLI docs to __run__, rather than forcing them into the module level docstring 4. introspection tools can more readily discover modules that expose a command line interface 5. command line scripts written this way are automatically easier to test, since they're written in a functional style (argv goes in, return code comes out) 6. the call-and-response functional structure is also likely to provide a better long term base building block for interoperable CLI frameworks Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From encukou at gmail.com Tue May 23 06:47:52 2017 From: encukou at gmail.com (Petr Viktorin) Date: Tue, 23 May 2017 12:47:52 +0200 Subject: [Import-SIG] Idea: Autorun functionality for Python modules (redux) In-Reply-To: References: Message-ID: <600a1cd7-0bab-d987-0468-e9aa1caa6969@gmail.com> On 05/23/2017 04:02 AM, Nick Coghlan wrote: > (Note: Posting to import-sig as this isn't something I'm actively > planning to pursue myself any time soon, but I want to ensure we don't > accidentally block this possibility while working on the proposal to > make it possible to run extension modules as Python scripts. ) Thanks for sharing another piece of the master plan :) > PEP 299 is an old rejected PEP proposing a special "__main__()" > function for Python modules: https://www.python.org/dev/peps/pep-0299/ > > Three main points were cited in its rejection: > > - the name clash with "import __main__" > - the lack of a clear strategy for supporting both newer versions of > Python that supported automatic execution of a suitably named function > as well as older versions that required an "if __name__ == > '__main__':" block > - the status quo wasn't seen as particular broken and "it would be > more familiar to C/C++ programmers" wasn't a compelling argument for > adding a second way to do it > > In an email discussion with Brandon Rhodes a few months back, he > lamented the apparent intransigence of the core developers on this > front, and I pointed out that nobody had ever actually made a > follow-up proposal that specifically addressed the rationale applied > in rejecting PEP 299, and put together a sketch of what such a > proposal might look like. > > The first two technical points can be handled by: > > 1. Using `__run__` as the special function name > 2. Setting "__main__.__autorun__ = True" prior to main module > execution, and allowing a module to delete it or set `__autorun__ = > False` to turn off the default autorun behaviour > > With those two special attributes defined, the autorun protocol would be: > > if getattr(main_module, "__autorun__", False): > try: > runmain = main_module.__run__ > except AttributeError: > pass > else: > import sys > sys.exit(runmain(sys.argv) > > Scripts that want to optionally invoke "__run__" explicitly for > compatibility with older Python versions can then check "__autorun__" > to see whether or not they need to start the application themselves: > > if __name__ == "__main__" and not globals().get("__autorun__"): > import sys > sys.exit(__run__(sys.argv)) I have two nitpicks: 1) __main__.__autorun__ is quite a weird place to store interpreter-level configuration. 2) AFAIU, `if __name__ == "__main__"` is an obscure implementation detail (answer to "what should be the name of a nameless module?"), reified by people relying on it. It would be nice to try to downgrade it to being just an implementation detail again. So instead of __main__.__autorun__, I'd prefer: 2. Setting `sys.autorun_module = "__main__"` prior to main module execution, and allowing a module to change it to use another module, or delete it to turn off the default autorun behaviour. The autorun protocol would become: main_module_name = getattr(sys, 'autorun_module', None) if main_module_name is not None: main_module = __import__(main_module_name) try: runmain = main_module.__run__ except AttributeError: pass else: import sys sys.exit(runmain(sys.argv) and the compatibility shim would be: if __name__ == getattr(sys, "autorun_module", "__main__"): import sys sys.exit(__run__(sys.argv)) (This runs the shim in all Python versions, which I think is fine: sys.exit would prevent the future Python's own autorun from running.) From ncoghlan at gmail.com Thu May 25 11:17:56 2017 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 26 May 2017 01:17:56 +1000 Subject: [Import-SIG] Idea: Autorun functionality for Python modules (redux) In-Reply-To: <600a1cd7-0bab-d987-0468-e9aa1caa6969@gmail.com> References: <600a1cd7-0bab-d987-0468-e9aa1caa6969@gmail.com> Message-ID: On 23 May 2017 at 20:47, Petr Viktorin wrote: > I have two nitpicks: > 1) __main__.__autorun__ is quite a weird place to store interpreter-level > configuration. Fair point when it comes to the idea as posted, but I wasn't actually intending for it to be interpreter level configuration: Instead, I'd expect runpy.run_path() and run_module() to *always* set `__autorun__=True`, to indicate they were going to implicitly invoke any `__run__` method definitions, without having to mess with the name of the module they're executing. Thus, __autorun__ would be a "Running a module as a script" setting, and "__main__.__autorun__" would get set by virtue of it being a module getting run as a script, rather than by virtue of it specifically being the main module. > 2) AFAIU, `if __name__ == "__main__"` is an obscure implementation detail > (answer to "what should be the name of a nameless module?"), reified by > people relying on it. It would be nice to try to downgrade it to being just > an implementation detail again. Unfortunately, __main__ also gets used as "the place to access application wide config settings" (by way of the "import __main__" approach), so I don't think we're ever going to get away from that name twiddling for the actual __main__ module. I *do* think we can get rid of (or at least deemphasise) it for the runpy APIs though - I just missed that I'd left that aspect of the design rationale out of the initial post. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia From gmarcel.plch at gmail.com Tue May 23 09:57:28 2017 From: gmarcel.plch at gmail.com (gmarcel.plch at gmail.com) Date: Tue, 23 May 2017 13:57:28 -0000 Subject: [Import-SIG] Running C extension modules using -m switch In-Reply-To: References: <1495111849.11457.2.camel@localhost> <9058da1b-b1cb-f123-721a-e66c28066593@gmail.com> <55aa2054-baae-5b2c-dd3b-3eb5d395a683@gmail.com> <8d0c1856-48f6-779b-e6c3-b139f5ad50f2@gmail.com> Message-ID: <1495547844.21970.2.camel@localhost> On Tue, 2017-05-23 at 11:19 +1000, Nick Coghlan wrote: > On 23 May 2017 at 00:51, Petr Viktorin wrote: > > Marcel had to leave for the day. To prevent losing a PyCon sprint > > day, I've > > fixed up his latest PoC and pushed it here: > >? > > Branch: https://github.com/encukou/cpython/tree/main_c_modules_name space > > Diff: > > https://github.com/encukou/cpython/compare/master...encukou:main_c_ modules_namespace?expand=1 > >? > > These changes get rid of Py_mod_main, and add an optional > > exec_in_module > > method to loaders. This method initializes a given module using a > > given > > spec. > >? > > Does this approach look good? >? > +1 from me (my only comments relate to the fact that some of the > helper functions descriptions in runpy will need updating), and I > think it retains forward compatibity with an "autorun" idea I came up > with during an email discussion with Brandon Rhodes a few months > back. >? > While the latter is still an entirely speculative concept, I'll post > a > thread about it so you can double-check that we're still leaving that > possibility open for the future. Alright, thanks. Here's the pull request: https://github.com/python/cpython/pull/1761