[Mailman-Users] bin/arch failing

[Bob]

Hi:

I am having a problem regenerating my archive.

Everything was going along fine, with daily updates being added to the
archive. I then wanted to refresh the entire archive and ran bin/arch
--wipe list1 

it failed. 

Archive articles go from 0000 to 66990

list1.mbox is: 526313754 Aug 21 20:19 list1.mbox

The server NetBSD 3.1, has 2GB memory

bin/arch first went through every article

Week-of-Mon-20070702
#65136 <20070703134009.74f75a8e at viola.tamara-b.org>
figuring article archives
[...]

then after many 

Updating HTML for article 4584
Updating HTML for article 4585
Updating HTML for article 4586
Updating index files for archive [Week-of-Mon-20050321]
  Date
  Subject
  Author
  Thread
Computing threaded index
Updating HTML for article 4587
Updating HTML for article 4588
Updating HTML for article 4589
Updating HTML for article 4590
Updating HTML for article 4591
Updating HTML for article 4592
Updating HTML for article 4593

[...]

Updating HTML for article 9353
Updating index files for archive [Week-of-Mon-20041122]
Traceback (most recent call last):
  File "bin/arch", line 200, in ?
    main()
  File "bin/arch", line 190, in main
    archiver.close()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 322,
in close self.update_dirty_archives()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 538,
in update_dirty_archives self.update_archive(i)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperArch.py", line 1121,
in update_archive self.__super_update_archive(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 439,
in update_archive self.__set_parameters(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 354,
in __set_parameters firstdate = self.database.firstdate(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
209, in firstdate self.__openIndices(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
240, in __openIndices self.__closeIndices()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
265, in __closeIndices index.close()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
187, in close fp.write(marshal.dumps(self.dict))
MemoryError

So I ran a subset of the archive:

bin/arch -s 10000 -e 66990 list1

And after a while:
Updating HTML for article 11689
Updating HTML for article 11690
Updating index files for archive [Week-of-Mon-20050815]
Traceback (most recent call last):
  File "bin/arch", line 200, in ?
    main()
  File "bin/arch", line 190, in main
    archiver.close()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 322,
in close self.update_dirty_archives()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 538,
in update_dirty_archives self.update_archive(i)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperArch.py", line 1121,
in update_archive self.__super_update_archive(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 439,
in update_archive self.__set_parameters(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 354,
in __set_parameters firstdate = self.database.firstdate(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
209, in firstdate self.__openIndices(archive)
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
240, in __openIndices self.__closeIndices()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
265, in __closeIndices index.close()
  File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line
187, in close fp.write(marshal.dumps(self.dict))
MemoryError

====================
_dirty_archives ??? something corrupt?

Is there a fix for this?


I can successfully run about the last 10,000 

bin/arch -s 50000 -e 66990 list1

But that is about 1/7th of the archive
Can I produce a complete archive in chunks of 10,000? How?

Best regards
Bob