[Mailman-Developers] Hi from a student interested in a GSoC project

[Barry Warsaw]

On Apr 11, 2013, at 02:22 PM, Elias Assarsson wrote:

>> * MM2's configuration file is a Python file which really must be imported
>> in order to get a valid set of values.  MM3's configuration file is a stack
>> of .ini-style files.

>I am trying to find and understand the configuration files so that I know
>what that that needs to be migrated and to what form. Is the MM2
>configuration you refer to mainly Mailman/mm_cfg.py and MM3 configuration
>files src/mailman/config/* and /src/mailman/<listname>/*?

Yes, in a deployed Mailman 2, mm_cfg.py will contain the system configuration
settings.  These override the settings from Defaults.py so a good way to
explore is to use `bin/withlist` and `import mm_cfg` at the Python prompt.

In MM3, we use lazr.config, which is essentially a configparser type package
that allows for stacks of configurations, with pushing and popping values on
this stack.

http://pythonhosted.org/lazr.config/

The src/mailman/config/schema.cfg file is at the bottom of the stack and
defines the schema, obviously :).  From there, src/mailman/config/mailman.cfg
essentially instantiates this schema, providing all the defaults.  In the
testing environment, src/mailman/testing/testing.cfg gets pushed on top of
that (and many tests push and pop micro-overrides).  In a deployed system, the
site's mailman.cfg is on the top of the stack and so can override anything.
There are various places this mailman.cfg file is searched; see
src/mailman/core/initialize.py for all the gory details.

List-specific configurations live in the various config.db files for MM2.
These are pickles.  In MM3, everything's in the database.

>Given the two points above it seems that handling the migration from within
>Python is the best choice (rather than using Augeas which is C based or
>Config::Model which is Perl based).

I think so.  Pickles in particular are Python-specific.  You could generate an
intermediate format, but I'm not sure it's worth it.

>Obviously one wants to use whatever feature of Python that can ease the
>process. You seem to be using configparser in MM3. I dunno if there are any
>other Python tool one should look at in helping migration. Maybe one should
>investigate this further.

>>   * What about importing archives?

>I have tried to find information on how archives are stored in MM2 and MM3
>but failed to find any. What is a good source to learn about this?

Importing archives will either be trivial or impossible :).  At the lowest
level, parsing the MM2 .mbox file and generating maildirs would work, but
there are existing tools to do that, so probably little for GSoC to worry
about.  Much more interesting would be to parse the Pipermail database files
and try to build a reverse mapping from URL to Message-ID so that these could
be preserved when the archive is regenerated for HyperKitty or whatever.

>> There is some moderate beginnings in the 3.0 tree, but none of it is
>> functional in all likelihood.  Take a look at src/mailman/bin/export.py
>> and src/mailman/commands/cli_import.py.

>So those are the files that are supposed to handle migration for which the
>project is to make them handle a complete migration from MM2 to MM3?

Well, I'd say they were more some unfinished work in that direction.

-Barry