[Python-Dev] PyCapsule_Import semantics, relative imports, module names etc.

[Larry Hastings]

PyCapsule_Import() is a simple helper function, a slightly-updated analogue
to PyCObject_Import().  It's not particularly sophisticated, and I'm not
surprised it's bewildered by complicated scenarios like relative imports in
subpackages.  For now all I can recommend is that you not try and torture
PyCapsule_Import().  And, as always... patches welcome.


/arry

On Sat, Jul 25, 2015 at 1:41 AM, John Dennis <jdennis at redhat.com> wrote:

> While porting several existing CPython extension modules that form a
> package to be 2.7 and 3.x compatible the existing PyObject_* API was
> replaced with PyCapsule_*. This introduced some issues the existing CPython
> docs are silent on. I'd like clarification on a few issues and wish to
> raise some questions.
>
> 1. Should an extension module name as provided in PyModule_Create (Py3) or
> Py_InitModule3 (Py2) be fully package qualified or just the module name? I
> believe it's just the module name (see item 5 below) Yes/No?
>
> 2. PyCapsule_Import does not adhere to the general import semantics. The
> module name must be fully qualified, relative imports are not supported.
>
> 3. PyCapsule_Import requires the package (e.g. __init__.py) to import
> *all* of it's submodules which utilize the PyCapsule mechanism preventing
> lazy on demand loading. This is because PyCapsule_Import only imports the
> top level module (e.g. the package). From there it iterates over each of
> the module names in the module path. However the parent module (e.g.
> globals) will not contain an attribute for the submodule unless it's
> already been loaded. If the submodule has not been loaded into the parent
> PyCapsule_Import throws an error instead of trying to load the submodule.
> The only apparent solution is for the package to load every possible
> submodule whether required or not just to avoid a loading error. The
> inability to load modules on demand seems like a design flaw and change in
> semantics from the prior use of PyImport_ImportModule in combination with
> PyObject. [One of the nice features with normal import loading is setting
> the submodule name in the parent, the fact this step is omitted is what
> causes PyCapsule_Import to fail unless all submodules are unconditionally
> loaded). Shouldn't PyCapsule_Import utilize PyImport_ImportModule?
>
> 4. Relative imports seem much more useful for cooperating submodules in a
> package as opposed to fully qualified package names. Being able to import a
> C_API from the current package (the package I'm a member of) seems much
> more elegant and robust for cooperating modules but this semantic isn't
> supported (in fact the leading dot syntax completely confuses
> PyCapsule_Import, doc should clarify this).
>
> 5. The requirement that a module specifies it's name as unqualified when
> it is initializing but then also has to use a fully qualified package name
> for PyCapsule_New, both of which occur inside the same initialization
> function seems like an odd inconsistency (documentation clarification would
> help here). Also, depending on your point of view package names could be
> considered a deployment/packaging decision, a module obtains it's fully
> qualified name by virtue of it's position in the filesystem, something at
> compile time the module will not be aware of, another reason why relative
> imports make sense. Note the identical comment regarding _Py_PackageContext
> in  modsupport.c (Py2) and moduleobject.c (Py3) regarding how a module
> obtains it's fully qualified package name (see item 1).
>
> Thanks!
>
> --
> John
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/larry%40hastings.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150726/3b6c0ed8/attachment.html>