[Python-Dev] ElementTree - Why not part of the core? (fwd)

[Fredrik Lundh]

Martin v. Löwis wrote:

> >>That's primarily for the author of the software to decide, at this
> >>point. Fredrik Lundh would have to offer it for contribution first.
> >
> > I've already done that, as others have noted.  Everything I release
> > under a Python-compatible license is available for bundling with the
> > python core.
>
> I see a difference here, though, between "offer for contribution"
> and "make available for bundling". To my knowledge, you never said
> (until now) "I would like to incorporate ElementTree into the Python CVS
> repository, and thus have it become part of future Python releases".

Well, I'm offering you to bundle a well-defined version of ElementTree
with Python.  In practice, the plan I proposed means that we'll be shipping
a *copy* of ET with Python, not that ET development will move over to
python.org.

At any time, it should be possible to say "Python release X.Y.Z includes
ElementTree release A.B.C".

(this doesn't rule out bug fixes in the trunk, of course, but work on new
features should take place elsewhere)

> So how can the trunk get smaller, yet the distribution larger?

That's a separate issue; I'll have to get back to this at a later time.

> >   - I check in an existing elementtree release in a separate location in
> >     the svn.python.org source tree. e.g.
> >
> >         svn.python.org/kits/elementtree-1.2.6-20050316
> >
> >     this will make it clear that this is external software, and it also
> >     provides a reference point for tracking down local changes
>
> Ah, so you want what CVS calls a "vendor branch": code that is
> externally maintained, and imported from time to time.

Exactly.  But I'm not sure "branch" is really accurate here; it's more like
"snapshot".  Stable releases are added to the "vendor" tree, and relevant
files are are then copied to the appropriate location in the release tree.

> The other issue is, of course, the question whose job it is to actually
> perform the updates. Would you expect to do that yourself, or would
> you expect somebody else does that?

I can deal with this.

> I'm still troubled that you keep saying that sgmlop "was forked". I
> had not been PyXML maintainer long enough to remember the precise
> history of things, but it was certainly the case that you could have
> updated it all the time - you still have write permission to the
> PyXML repository.

Perhaps, but there's a limit to how much downstream use you can expect
anyone to monitor (cf. the Seigenthaler story).  But I should point out that
I don't think the forking was intentional; it just happened.

> >   - we decide what elementtree modules to include, and where to place
> >     them, and copy them to the python trunk.
> >
> >         (suggestion: either directly under xml, or under xml.etree)
>
> Would there be a reason not to include the entire elementtree package?
> Either xml.etree, or xml.tree would be fine with me, -0 for putting
> it directly into xml.

Since all the relevant module names start with "Element", putting it directly
under xml wouldn't be too bad.  But an xml subpackage is better, and prior
art says "etree".

I think that limiting this to ElementTree, ElementPath, and perhaps Element-
Include would be a good start.

> >   - I convert the pythondoc pages for the included modules to match the
> >     library reference format (someone will have to help with the markup
> >     here)
>
> Would you then start using the tex sources as your primary sources, or
> would this conversion need to be done every time the package is updated?

The reference documentation is autogenerated from markup in the source
file, so yes, some kind of conversion has to be done for each new release.

> >   - when new stable releases appear upstream, add to kits and copy
> >     relevant modules.  update/tweak docs as necessary.
>
> The "tweak docs" part sounds somewhat worrying. Of course, you could run
> "svn diff" on the old and new version, to see what doc strings have
> changed or appeared - but that might be quite some work.

Luckily, it can be partially automated.  And ET doesn't change very quickly.

> >   - delegate incoming bug reports / patches to the upstream maintainer.
>
> Would it be sufficient to set you as the "Assigned To" in the SF
> tracker?

Sure.

And maybe PEP 291 could be updated to cover both compatibility with older
Python versions and other compatibility issues.

> > and, optionally
> >
> >   - sort out expat bundling issues, and include cElementTree as well
> >     (using the same approach as above).
>
> Not sure what this would be; we probably can look at it again when
> we are done with the first part.

The problem is that cElementTree is, by default, statically linked against
its own (unmodified) copy of expat.  The same applies to pyexpat.  I think
it would be better if there was only one copy of expat in

(one way to do this would be to add an "function pointer table" to pyexpat
that contains pointers to selected portions of the expat API, and then add
an indirection level to cElementTree)

> > whaddya think?
>
> Overall, sounds like a good plan.

Just say "go", and I'll start working on this.

</F>