From msapiro at value.net  Wed Feb  1 03:57:33 2006
From: msapiro at value.net (Mark Sapiro)
Date: Tue, 31 Jan 2006 18:57:33 -0800
Subject: [Mailman-Developers] [Mailman-Users] any info on this
	reportedexploit?
In-Reply-To: <43DEE399.3010605@is.kochi-u.ac.jp>
Message-ID: <PC17402006013118573303437d2dde98@msapiro>

Tokio Kikuchi wrote:
>
>Well, the logic may be unclear but calculate_attachment_dir() tries 
>again to guess the real date of message arrival because it may be called 
>from bin/arch.  I think the code should be cleaned up but since we are 
>now dealing with the email parsedate bug and it should be safe to limit 
>our patch to this purpose.


I ran into a problem with this while testing something else. I posted a
message using bin/inject and also had scrub_nondigest true. The
message got shunted in Scrubber's calculate_attachments_dir because
parts = msg.get_unixfrom().split() threw an exception because
msg.get_unixfrom() was None.

Anyway, I wasn't concerned about that per se, but rather about how the
process had gotten that far. It turns out that email.Utils.parsedate()
returns a 9-tuple with tm_yday = 0 which is not acceptable to
time.strftime() in Python 2.4, but it is OK for time.mktime().

When safe_strftime() returns None, we get to
        datestr = msgdata.get('X-List-Received-Date')
        if datestr:
            datedir = safe_strftime(fmt, datestr)

This is a problem too - first, X-List-Received-Date is a message
header, not message metadata and second it is a formatted date which
is not the right argument for strftime so safe_strftime() returns None
again and we fall back to unixfrom.

Perhaps the workaround patch for parsedate should be something like

+def parsedate(data):
+    if not data:
+        return None
+    t = _parsedate(data)
++   if not t[7]:
++       t = t[:7] + (1,) +t[8:]
+    try:
+        time.mktime(t)
+    except OverflowError:
+        return None
+    return t

and similarly for parsedate_tz

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From msapiro at value.net  Wed Feb  1 04:23:46 2006
From: msapiro at value.net (Mark Sapiro)
Date: Tue, 31 Jan 2006 19:23:46 -0800
Subject: [Mailman-Developers] [Mailman-Users] any info on
	thisreportedexploit?
In-Reply-To: <PC17402006013118573303437d2dde98@msapiro>
Message-ID: <PC1740200601311923460968880ab216@msapiro>

Mark Sapiro wrote:

>++   if not t[7]:
>++       t = t[:7] + (1,) +t[8:]

On second thought, that should be

    if t and not t[7]
        t = t[:7] + (1,) +t[8:]

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From msapiro at value.net  Wed Feb  1 04:26:11 2006
From: msapiro at value.net (Mark Sapiro)
Date: Tue, 31 Jan 2006 19:26:11 -0800
Subject: [Mailman-Developers] [Mailman-Users] any info
	onthisreportedexploit?
In-Reply-To: <PC1740200601311923460968880ab216@msapiro>
Message-ID: <PC174020060131192611087585b760d6@msapiro>

>Mark Sapiro wrote:
>
>>++   if not t[7]:
>>++       t = t[:7] + (1,) +t[8:]
>
>On second thought, that should be
>
>    if t and not t[7]
>        t = t[:7] + (1,) +t[8:]

I seem to be having a lot of trouble with this. Make that

    if t and not t[7]:
        t = t[:7] + (1,) +t[8:]


-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From msapiro at value.net  Wed Feb  1 23:13:25 2006
From: msapiro at value.net (Mark Sapiro)
Date: Wed, 1 Feb 2006 14:13:25 -0800
Subject: [Mailman-Developers] [Mailman-Users] Permission of
	data/bounce-events-?????.pck
In-Reply-To: <20060202030024.3C76.IMACAT@mail.imacat.idv.tw>
Message-ID: <PC1740200602011413250750edbcbe87@msapiro>

imacat wrote:
>
>    I noted that in the source of mailman 2.1.7 there are 2 lines in
>bin/mailmanctl:
>
>line 421-422
>        # Clear our file mode creation umask
>        os.umask(0)
>
>    Is this intended?  Is it the reason why data/bounce-events-?????.pck
>are world-writable?

It looks like you're right. I don't know if there is/was a good reason
or not. I'm cross posting this reply to Mailman-Developers. Maybe
someone there knows the reason for this.

Note that many places in the Mailman code, umask is saved and set for a
particular purpose and then restored, but BounceRunner doesn't do this
when creating the bounce-events-*.pck. I don't know why.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From imacat at mail.imacat.idv.tw  Thu Feb  2 08:31:47 2006
From: imacat at mail.imacat.idv.tw (imacat)
Date: Thu, 02 Feb 2006 15:31:47 +0800
Subject: [Mailman-Developers] [Mailman-Users] Permission of
	data/bounce-events-?????.pck
In-Reply-To: <PC1740200602011413250750edbcbe87@msapiro>
References: <20060202030024.3C76.IMACAT@mail.imacat.idv.tw>
	<PC1740200602011413250750edbcbe87@msapiro>
Message-ID: <20060202153135.F2DC.IMACAT@mail.imacat.idv.tw>

    I could use a little explanation on this issue.

    I was checking my system for world-writable files, and found lots of
data/bounce-events-?????.pck that are world-writable:

imacat at rinse ~ % ls -lt /var/lib/mailman/data | grep 'rw. '
-rw-rw-rw-  1 list list       0 2006-01-31 01:26 bounce-events-02258.pck
-rw-rw-rw-  1 list list    4786 2005-12-26 17:36 bounce-events-26086.pck
...
imacat at rinse ~ %

    I felt weird, so I looked into the mailman 2.1.7 source, and found
such lines in bin/mailmanctl:

line 421-422
        # Clear our file mode creation umask
        os.umask(0)

    I checked the umask when running the init script (copied from
scripts/mailman) on boot, inserted "touch /var/tmp/mailman.umask.test"
and reboot.  The result is sane, but a new world-writable
bounce-events-?????.pck was created altogeter.  Apparently
bin/mailmanctl did not inherit the umask from its parent init script:

imacat at rinse ~ % ls -l /var/tmp/mailman.umask.test
-rw-r--r--  1 root root 0 2006-02-02 02:52 /var/tmp/mailman.umask.test
imacat at rinse ~ % ls -l /var/lib/mailman/data/bounce-events-*.pck
-rw-rw-rw-  1 list list 0 2006-02-02 02:41 /var/lib/mailman/data/bounce-events-02211.pck
imacat at rinse ~ %

    I searched the archive.  I know that these long-gone
data/bouce-events-?????.pck can be safely removed.  But I'm a little
worried about this "world-writable" thing.  Is it intended?  Mark told
me these files are safe.  That's fine.  But I still find it confusing:

     1. Even if they are safe, it'll still create confusion when someone
tries to dump them for debugging purpose, if tempered.

     2. Even if they are safe, will other files created by
bin/mailmanctl, may be accidently, be world-writable, too?

     3. Will other processes raised by bin/mailmanctl, may be accidently,
inherit this umask and create world-writable files on surprise, too?

     4. Finally, is that necessary to clear the umask to 0?

    I'm not quite familiar with python, so forgive me if I didn't send any
patch on this.  It may not be appropriate, either, for the whole umask
operation seems to be quite complicated in bin/mailmanctl and an
improper patch may ruin it.

On Wed, 1 Feb 2006 14:13:25 -0800
Mark Sapiro <msapiro at value.net> wrote:
> imacat wrote:
> >    I noted that in the source of mailman 2.1.7 there are 2 lines in
> >bin/mailmanctl:
> >
> >line 421-422
> >        # Clear our file mode creation umask
> >        os.umask(0)
> >
> >    Is this intended?  Is it the reason why data/bounce-events-?????.pck
> >are world-writable?
> 
> It looks like you're right. I don't know if there is/was a good reason
> or not. I'm cross posting this reply to Mailman-Developers. Maybe
> someone there knows the reason for this.
> 
> Note that many places in the Mailman code, umask is saved and set for a
> particular purpose and then restored, but BounceRunner doesn't do this
> when creating the bounce-events-*.pck. I don't know why.



--
Best regards,
imacat ^_*' <imacat at mail.imacat.idv.tw>
PGP Key: http://www.imacat.idv.tw/me/pgpkey.txt

<<Woman's Voice>> News: http://www.wov.idv.tw/
Tavern IMACAT's: http://www.imacat.idv.tw/
TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060202/8cbbb9af/attachment.pgp 

From tkikuchi at is.kochi-u.ac.jp  Thu Feb  2 08:44:49 2006
From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi)
Date: Thu, 02 Feb 2006 16:44:49 +0900
Subject: [Mailman-Developers] [Mailman-Users] any
	info	onthisreportedexploit?
In-Reply-To: <PC174020060131192611087585b760d6@msapiro>
References: <PC174020060131192611087585b760d6@msapiro>
Message-ID: <43E1B871.9050700@is.kochi-u.ac.jp>

Mark Sapiro wrote:
>>Mark Sapiro wrote:
>>
>>
>>>++   if not t[7]:
>>>++       t = t[:7] + (1,) +t[8:]
>>
>>On second thought, that should be
>>
>>   if t and not t[7]
>>       t = t[:7] + (1,) +t[8:]
> 
> 
> I seem to be having a lot of trouble with this. Make that
> 
>     if t and not t[7]:
>         t = t[:7] + (1,) +t[8:]
> 
> 
I changed the patch including this one.  Please check it again.
https://sourceforge.net/tracker/index.php?func=detail&aid=1419490&group_id=103&atid=300103


-- 
Tokio Kikuchi, tkikuchi@ is.kochi-u.ac.jp
http://weather.is.kochi-u.ac.jp/

From msapiro at value.net  Thu Feb  2 20:17:56 2006
From: msapiro at value.net (Mark Sapiro)
Date: Thu, 2 Feb 2006 11:17:56 -0800
Subject: [Mailman-Developers] [Mailman-Users] any info on this reported
	exploit?
In-Reply-To: <43E1B871.9050700@is.kochi-u.ac.jp>
Message-ID: <PC1740200602021117560593aa6330a9@msapiro>

Tokio Kikuchi wrote:
>> 
>I changed the patch including this one.  Please check it again.
>https://sourceforge.net/tracker/index.php?func=detail&aid=1419490&group_id=103&atid=300103


I've applied the revised patch and I'm testing. So far it looks good.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From barry at python.org  Fri Feb  3 05:27:59 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 02 Feb 2006 23:27:59 -0500
Subject: [Mailman-Developers] [Mailman-Users] any
	info	onthisreportedexploit?
In-Reply-To: <PC174020060131192611087585b760d6@msapiro>
References: <PC174020060131192611087585b760d6@msapiro>
Message-ID: <1138940879.10783.41.camel@geddy.wooz.org>

On Tue, 2006-01-31 at 19:26 -0800, Mark Sapiro wrote:

> I seem to be having a lot of trouble with this. Make that
> 
>     if t and not t[7]:
>         t = t[:7] + (1,) +t[8:]

If you look at email 3.0, the 7th item returned from parsedate() and
parsedate_tz() is always 1, obviously so that the value is acceptable to
time.mktime() and time.strftime(), or None of course.

I think the right thing to do is to fix email 2.5 to behave the same
way.  This is safe because that time is documented as not being
meaningful, so we might as well make it safe <wink>.

I'd rather do that than apply the current SF patch to Mailman.  One
thing I don't like about the patch is that it calls time.strftime() to
try to catch errors and return None.  I don't think that's necessary if
we change the email package as described above.

So let's fix and release an email 2.5.7 for Mailman 2.1.8.  Even though
there are more things I'd like to fix in the email 2.5 package, I'd
rather get this out in time to integrate in Mailman 2.1.8.  I can do
that pretty quickly.

Thoughts?
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060202/fe028daa/attachment.pgp 

From msapiro at value.net  Sun Feb  5 06:16:44 2006
From: msapiro at value.net (Mark Sapiro)
Date: Sat, 4 Feb 2006 21:16:44 -0800
Subject: [Mailman-Developers] [Mailman-Users] Permission of
	data/bounce-events-?????.pck
In-Reply-To: <PC1740200602011413250750edbcbe87@msapiro>
Message-ID: <PC1740200602042116440046a9204c3a@msapiro>

imacat wrote:
>>
>>    I noted that in the source of mailman 2.1.7 there are 2 lines in
>>bin/mailmanctl:
>>
>>line 421-422
>>        # Clear our file mode creation umask
>>        os.umask(0)
>>
>>    Is this intended?  Is it the reason why data/bounce-events-?????.pck
>>are world-writable?

There doesn't appear to be a good reason. This has been changed for
Mailman 2.1.8 so that the 'default' umask will be 007 and also the
specific creation of the bounce-events queue file will have no
permission for 'other'.

The changes to bin/mailmanctl and Mailman/Queue/BounceRunner.py have
been committed to CVS and can be seen (soon) at
<http://cvs.sourceforge.net/viewcvs.py/mailman/mailman/>.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan


From imacat at mail.imacat.idv.tw  Sun Feb  5 22:39:36 2006
From: imacat at mail.imacat.idv.tw (imacat)
Date: Mon, 06 Feb 2006 05:39:36 +0800
Subject: [Mailman-Developers] [Mailman-Users] Permission of
	data/bounce-events-?????.pck
In-Reply-To: <PC1740200602042116440046a9204c3a@msapiro>
References: <PC1740200602011413250750edbcbe87@msapiro>
	<PC1740200602042116440046a9204c3a@msapiro>
Message-ID: <20060206053924.2CB6.IMACAT@mail.imacat.idv.tw>

On Sat, 4 Feb 2006 21:16:44 -0800
Mark Sapiro <msapiro at value.net> wrote:
> imacat wrote:
> >>    I noted that in the source of mailman 2.1.7 there are 2 lines in
> >>bin/mailmanctl:
> >>line 421-422
> >>        # Clear our file mode creation umask
> >>        os.umask(0)
> >>    Is this intended?  Is it the reason why data/bounce-events-?????.pck
> >>are world-writable?
> 
> There doesn't appear to be a good reason. This has been changed for
> Mailman 2.1.8 so that the 'default' umask will be 007 and also the
> specific creation of the bounce-events queue file will have no
> permission for 'other'.

    It's so great to hear that. ^_^  I shall be waiting for the release.

--
Best regards,
imacat ^_*' <imacat at mail.imacat.idv.tw>
PGP Key: http://www.imacat.idv.tw/me/pgpkey.txt

<<Woman's Voice>> News: http://www.wov.idv.tw/
Tavern IMACAT's: http://www.imacat.idv.tw/
TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060206/3fe12639/attachment.pgp 

From dario at ita.chalmers.se  Mon Feb 13 14:40:07 2006
From: dario at ita.chalmers.se (=?ISO-8859-1?Q?Dario_Lopez-K=E4sten?=)
Date: Mon, 13 Feb 2006 14:40:07 +0100
Subject: [Mailman-Developers] Status of XMLRPC in mailman
In-Reply-To: <1133199874.25354.9.camel@localhost.localdomain>
References: <1129307125.6026.10.camel@localhost.localdomain>	<1129335154.32365.56.camel@geddy.wooz.org>	<1130886730.30605.70.camel@localhost.localdomain>	<1131485036.7487.25.camel@localhost.localdomain>	<1131488690.20278.23.camel@geddy.wooz.org>
	<1133199874.25354.9.camel@localhost.localdomain>
Message-ID: <43F08C37.5050102@ita.chalmers.se>

Hello world,

I have recently found a need to administer, or rather integrate, mailman 
  (which runs on one machine) with a homegrown corporate directory 
system that we run on another machine. The load and layout of the 
systems forbid that we put both of them on the same machines, thus 
creating a need to administer mailman remotely.

Enter xml-rpc :-)

I am curious about the status of xml-rpc support in mailman, I recall 
that there was a discussion about a big patch adding xml-rpc 
capabilities to mailman back in october-november of last year.

Any news on that front?

Thanks,

/dario

-- 
-- -------------------------------------------------------------------
Dario Lopez-K?sten, IT Systems & Services Chalmers University of Tech.
Lyrics applied to programming & application design:
"emancipate yourself from mental slavery" - redemption song, b. marley


From jag at fsf.org  Mon Feb 13 17:18:33 2006
From: jag at fsf.org (Joshua Ginsberg)
Date: Mon, 13 Feb 2006 11:18:33 -0500
Subject: [Mailman-Developers] Status of XMLRPC in mailman
In-Reply-To: <43F08C37.5050102@ita.chalmers.se>
References: <1129307125.6026.10.camel@localhost.localdomain>
	<1129335154.32365.56.camel@geddy.wooz.org>
	<1130886730.30605.70.camel@localhost.localdomain>
	<1131485036.7487.25.camel@localhost.localdomain>
	<1131488690.20278.23.camel@geddy.wooz.org>
	<1133199874.25354.9.camel@localhost.localdomain>
	<43F08C37.5050102@ita.chalmers.se>
Message-ID: <1139847513.31591.3.camel@localhost.localdomain>

See:

http://sourceforge.net/tracker/index.php?func=detail&aid=1352333&group_id=103&atid=300103

We are continuing to improve the patch -- I've got a fairly extensive
modification in the works that will break a bit of the XMLRPC API...
however the patch works and is perfectly usable...

-jag


On Mon, 2006-02-13 at 14:40 +0100, Dario Lopez-K?sten wrote:
> Hello world,
> 
> I have recently found a need to administer, or rather integrate, mailman 
>   (which runs on one machine) with a homegrown corporate directory 
> system that we run on another machine. The load and layout of the 
> systems forbid that we put both of them on the same machines, thus 
> creating a need to administer mailman remotely.
> 
> Enter xml-rpc :-)
> 
> I am curious about the status of xml-rpc support in mailman, I recall 
> that there was a discussion about a big patch adding xml-rpc 
> capabilities to mailman back in october-november of last year.
> 
> Any news on that front?
> 
> Thanks,
> 
> /dario
> 
-- 
Joshua Ginsberg <jag at fsf.org>
Free Software Foundation - Senior Systems Administrator
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/mailman-developers/attachments/20060213/b9f7e224/attachment.pgp 

From dario at ita.chalmers.se  Mon Feb 13 20:46:01 2006
From: dario at ita.chalmers.se (=?ISO-8859-1?Q?Dario_Lopez-K=E4sten?=)
Date: Mon, 13 Feb 2006 20:46:01 +0100
Subject: [Mailman-Developers] Status of XMLRPC in mailman
In-Reply-To: <1139847513.31591.3.camel@localhost.localdomain>
References: <1129307125.6026.10.camel@localhost.localdomain>	<1129335154.32365.56.camel@geddy.wooz.org>	<1130886730.30605.70.camel@localhost.localdomain>	<1131485036.7487.25.camel@localhost.localdomain>	<1131488690.20278.23.camel@geddy.wooz.org>	<1133199874.25354.9.camel@localhost.localdomain>	<43F08C37.5050102@ita.chalmers.se>
	<1139847513.31591.3.camel@localhost.localdomain>
Message-ID: <43F0E1F9.5060505@ita.chalmers.se>

Joshua Ginsberg said the following on 2006-02-13 17:18:
> See:
> 
> http://sourceforge.net/tracker/index.php?func=detail&aid=1352333&group_id=103&atid=300103
> 
> We are continuing to improve the patch -- I've got a fairly extensive
> modification in the works that will break a bit of the XMLRPC API...
> however the patch works and is perfectly usable...
> 
> -jag
> 
> 

ah, great, thanks!

/dario

-- 
-- -------------------------------------------------------------------
Dario Lopez-K?sten, IT Systems & Services Chalmers University of Tech.
Lyrics applied to programming & application design:
"emancipate yourself from mental slavery" - redemption song, b. marley


From mads at kiilerich.com  Mon Feb 20 00:39:05 2006
From: mads at kiilerich.com (Mads Kiilerich)
Date: Mon, 20 Feb 2006 00:39:05 +0100
Subject: [Mailman-Developers] Mix of 8-bit ascii and unicode in Archive
	marshals
Message-ID: <43F90199.3000808@kiilerich.com>

Hi,

I just upgraded from 2.1.5 to 2.1.7, using a Fedora beta rpm. 
http://sourceforge.net/tracker/index.php?func=detail&aid=1405790&group_id=103&atid=300103 
has been applied.

Now I see crashes in Archiver like shown below. My problem is similar to 
but different from 
http://archives.free.net.ph/message/20060116.022354.167a1fae.en.html

It turns out to be because both 8-bit ascii and unicode strings appears 
as first element in key tuples in the 2006-February-author marshal. When 
sort tries to compare these unrelated encodings python don't know what 
to do. Neither do I...

How should the problem be solved? Should keys be stored as 8-bit ascii 
or as unicode?

If anybody can give a hint I'll try to come up with a solution ;-)

/Mads


Feb 18 17:57:44 2006 (29051) Uncaught runner exception: 'ascii' codec 
can't decode byte 0xe5 in position 1: ordinal not in range(128)
Feb 18 17:57:44 2006 (29051) Traceback (most recent call last):
   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 111, in _oneloop
     self._onefile(msg, msgdata)
   File "/usr/lib/mailman/Mailman/Queue/Runner.py", line 167, in _onefile
     keepqueued = self._dispose(mlist, msg, msgdata)
   File "/usr/lib/mailman/Mailman/Queue/ArchRunner.py", line 73, in _dispose
     mlist.ArchiveMail(msg)
   File "/usr/lib/mailman/Mailman/Archiver/Archiver.py", line 217, in 
ArchiveMail
     h.close()
   File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 327, in close
     self.update_dirty_archives()
   File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 543, in 
update_dirty_archives
     self.update_archive(i)
   File "/usr/lib/mailman/Mailman/Archiver/HyperArch.py", line 1125, in 
update_archive
     self.__super_update_archive(archive)
   File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 447, in 
update_archive
     self._update_simple_index(hdr, archive, arcdir)
   File "/usr/lib/mailman/Mailman/Archiver/pipermail.py", line 460, in 
_update_simple_index
     msgid = self.database.first(archive, hdr)
   File "/usr/lib/mailman/Mailman/Archiver/HyperDatabase.py", line 296, 
in first
     key, msgid = index.first()
   File "/usr/lib/mailman/Mailman/Archiver/HyperDatabase.py", line 113, 
in first
     self.__sort() # guarantee that the list is sorted
   File "/usr/lib/mailman/Mailman/Archiver/HyperDatabase.py", line 73, 
in __sort
     self.sorted.sort()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 1: 
ordinal not in range(128)

Feb 18 17:57:44 2006 (29051) SHUNTING: 
1140281864.286916+117720f8b561f5c15f1f799c2dd1fce5881078cb

From msapiro at value.net  Mon Feb 20 01:33:17 2006
From: msapiro at value.net (Mark Sapiro)
Date: Sun, 19 Feb 2006 16:33:17 -0800
Subject: [Mailman-Developers] Mix of 8-bit ascii and unicode in
	Archivemarshals
In-Reply-To: <43F90199.3000808@kiilerich.com>
Message-ID: <PC1740200602191633170484ac103015@msapiro>

Mads Kiilerich wrote:
>
>Now I see crashes in Archiver like shown below. My problem is similar to 
>but different from 
>http://archives.free.net.ph/message/20060116.022354.167a1fae.en.html
>
>It turns out to be because both 8-bit ascii and unicode strings appears 
>as first element in key tuples in the 2006-February-author marshal. When 
>sort tries to compare these unrelated encodings python don't know what 
>to do. Neither do I...
>
>How should the problem be solved? Should keys be stored as 8-bit ascii 
>or as unicode?


This appears to be the same problem as in the thread at
<http://mail.python.org/pipermail/mailman-users/2006-February/049333.html>
which was apparently worked around by rebuilding the archive with
'bin/arch --wipe'.

I haven't looked at this in any detail, but it looks like maybe we
changed the encoding of author and/or subject so if you upgrade in the
middle of a month, you get this problem in the database.

-- 
Mark Sapiro <msapiro at value.net>       The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan