[Mailman-Users] Importing large archives ... design limit hit, and possible bug

[LuKreme]

On Saturday, June 1, 2002, at 09:21 PM, Scott Courtney wrote:
> On Saturday 01 June 2002 10:59 pm, LuKreme wrote:
>> Out of curiosity, how did you split the mbox?  I have about 1200 emails 
>> I
>> want to add to the archive.
>
> I wrote a little "awk" program to split them into 80-message chunks. Here 
> is
> the source code:

Ah.. awk.  I hate awk.  Good thing you wrote it. :)

> I have now run this (successfully) on my largest archive, which contained
> 987 messages.

Worked fine on my 1001 message archive

This loaded almost all the emails succesfully.  I STILL got the

*** malloc[11027]: error: Can't allocate region
Traceback (most recent call last):
   File "bin/arch", line 129, in ?
     main()

So, looking at the file more carefully I noticed that the NEXT email had 
"rom " isntea of "From " and that about 200 emails in this archive all had 
that.  All the emails that were mutlipart, so I think html-trap munged the 
"From " header.

All the emails got loaded (thanks!) but I'm still getting errors when it's 
trying to finish.

******
Updating index files for archive [2002-June]
   Date
   Subject
   Author
   Thread
Computing threaded index
Updating HTML for article 52
article file /Users/mailman/archives/private/list/2002-June/000052.html is 
missing!
Updating HTML for article 53
article file /Users/mailman/archives/private/list/2002-June/000053.html is 
missing!
Updating HTML for article 54
article file /Users/mailman/archives/private/list/2002-June/000054.html is 
missing!
Updating HTML for article 55
article file /Users/mailman/archives/private/list/2002-June/000055.html is 
missing!
Updating HTML for article 27
Updating HTML for article 56
Updating HTML for article 57
Traceback (most recent call last):
   File "bin/arch", line 129, in ?
     main()
   File "bin/arch", line 119, in main
     archiver.close()
   File "/Users/mailman/Mailman/Archiver/pipermail.py", line 306, in close
     self.write_TOC()
   File "/Users/mailman/Mailman/Archiver/HyperArch.py", line 905, in 
write_TOC
     toc.write(self.html_TOC())
   File "/Users/mailman/Mailman/Archiver/HyperArch.py", line 647, in html_TOC
     d = {"listname": self.maillist.real_name,
   File "/Users/mailman/Mailman/Archiver/HyperArch.py", line 76, in sizeof
     size = os.path.getsize(filename)
   File "/usr/local/lib/python2.2/posixpath.py", line 139, in getsize
     st = os.stat(filename)
OSError: [Errno 2] No such file or directory: 
'/Users/mailman/archives/private/list.mbox/list.mbox'
*******

sigh.  GOing through the process again.

--
You are responsible for your rose.
"Rule 5: Get Kirsten Dunst wet" (p 74 of Time 20-05-02)