[Mailman-Developers] Memory error when running arch
James Madill
James.Madill@duke.edu
Mon, 30 Jul 2001 13:18:23 -0400
This is a multipart message in MIME format.
--=_alternative 005F161D85256A99_=
Content-Type: text/plain; charset="us-ascii"
Last night, when running arch against one of the mailing lists on my
Mailman server, the following error was displayed on the terminal:
Updating HTML for article 12815
Updating HTML for article 12816
Updating HTML for article 12817
Traceback (most recent call last):
File "./arch", line 129, in ?
main()
File "./arch", line 119, in main
archiver.close()
File "/mailman/Mailman/Archiver/pipermail.py", line 304, in
close
self.update_dirty_archives()
File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in
update_dirty_archives
self.__super_update_dirty_archives()
File "/mailman/Mailman/Archiver/pipermail.py", line 513, in
update_dirty_archives
self.update_archive(i)
File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in
update_archive
self.__super_update_archive(archive)
File "/mailman/Mailman/Archiver/pipermail.py", line 422, in
update_archive
self._update_thread_index(archive, arcdir)
File "/mailman/Mailman/Archiver/pipermail.py", line 479, in
_update_thread_index
self.update_article(arcdir, article, a1, a3)
File "/mailman/Mailman/Archiver/HyperArch.py", line 1115, in
update_article
f.write(article.as_html())
File "/mailman/Mailman/Archiver/HyperArch.py", line 298, in
as_html
return self.html_tmpl % d
MemoryError
$
When I reran it this morning after checking the OS for errors, the same
problem occured:
Updating HTML for article 12490
Updating HTML for article 12491
Updating HTML for article 12492
Updating index files for archive [2001-February]
Date
no mem for new parser
Traceback (most recent call last):
File "./arch", line 129, in ?
main()
File "./arch", line 119, in main
archiver.close()
File "/mailman/Mailman/Archiver/pipermail.py", line 304, in
close
self.update_dirty_archives()
File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in
update_dirty_archives
self.__super_update_dirty_archives()
File "/mailman/Mailman/Archiver/pipermail.py", line 513, in
update_dirty_archives
self.update_archive(i)
File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in
update_archive
self.__super_update_archive(archive)
File "/mailman/Mailman/Archiver/pipermail.py", line 420, in
update_archive
self._update_simple_index(hdr, archive, arcdir)
File "/mailman/Mailman/Archiver/pipermail.py", line 437, in
_update_simple_index
article = self.database.getArticle(self.archive, msgid)
File "/mailman/Mailman/Archiver/HyperDatabase.py", line 279, in
getArticle
article = self.__cache[msgid] = pickle.loads(buf)
MemoryError
$
The three cron jobs also appeared to fail today as well:
Your "cron" job on arachnia
/usr/local/bin/python -S /mailman/cron/qrunner
produced the following output:
Traceback (most recent call last):
File "/mailman/cron/qrunner", line 85, in ?
from Mailman import MailList
File "/mailman/Mailman/MailList.py", line 43, in ?
from Mailman.MailCommandHandler import MailCommandHandler
MemoryError
--
Your "cron" job on arachnia
/usr/local/bin/python -S /mailman/cron/gate_news
produced the following output:
Traceback (most recent call last):
File "/mailman/cron/gate_news", line 40, in ?
from Mailman import MailList
File "/mailman/Mailman/MailList.py", line 44, in ?
from Mailman.HTMLFormatter import HTMLFormatter
File "/mailman/Mailman/HTMLFormatter.py", line 27, in ?
from Mailman.htmlformat import *
MemoryError
--
Your "cron" job on arachnia
/usr/local/bin/python -S /mailman/cron/senddigests
produced the following output:
Traceback (most recent call last):
File "/mailman/cron/senddigests", line 65, in ?
main()
File "/mailman/cron/senddigests", line 42, in main
send_list_digest(mlist)
File "/mailman/cron/senddigests", line 46, in send_list_digest
mlist.Lock()
File "/mailman/Mailman/MailList.py", line 1343, in Lock
self.Load()
File "/mailman/Mailman/MailList.py", line 903, in Load
raise Errors.MMCorruptListDatabaseError, e
Mailman.Errors.MMCorruptListDatabaseError: bad marshal data
The machine is a Solaris 7 box with 256Meg of RAM
Mailman version is 2.0.6
Python version is 2.1
The list in question has about 16,000 archived messages
arch has no problem running against lists with fewer messages in the
archives. It looks like the arch script might be overflowing some kind of
Python memory buffer.
Any ideas, solutions?
-- James
o o o o o o o . . . _______________________ _______=======_T___
o _____ |James Madill | |Duke Univ Med Ctr|
>.][__n_n_| D[ ====|____ |james.madill@duke.edu| | (919) 286-6384 |
(________|__|_[____/____]_|_____________________|_|_________________|
_/oo O-O-O ` oo oo 'o^o^o o^o^o` 'o^o o^o`
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
<http://www.duke.edu/~madil001/>
--=_alternative 005F161D85256A99_=
Content-Type: text/html; charset="us-ascii"
<br><font size=2 face="sans-serif">Last night, when running arch against one of the mailing lists on my Mailman server, the following error was displayed on the terminal:</font>
<br>
<br><font size=2 face="sans-serif"> Updating HTML for article 12815</font>
<br><font size=2 face="sans-serif"> Updating HTML for article 12816</font>
<br><font size=2 face="sans-serif"> Updating HTML for article 12817</font>
<br><font size=2 face="sans-serif"> Traceback (most recent call last):</font>
<br><font size=2 face="sans-serif"> File "./arch", line 129, in ?</font>
<br><font size=2 face="sans-serif"> main()</font>
<br><font size=2 face="sans-serif"> File "./arch", line 119, in main</font>
<br><font size=2 face="sans-serif"> archiver.close()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 304, in close</font>
<br><font size=2 face="sans-serif"> self.update_dirty_archives()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in update_dirty_archives</font>
<br><font size=2 face="sans-serif"> self.__super_update_dirty_archives()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 513, in update_dirty_archives</font>
<br><font size=2 face="sans-serif"> self.update_archive(i)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in update_archive</font>
<br><font size=2 face="sans-serif"> self.__super_update_archive(archive)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 422, in update_archive</font>
<br><font size=2 face="sans-serif"> self._update_thread_index(archive, arcdir)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 479, in _update_thread_index</font>
<br><font size=2 face="sans-serif"> self.update_article(arcdir, article, a1, a3) </font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 1115, in update_article</font>
<br><font size=2 face="sans-serif"> f.write(article.as_html())</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 298, in as_html</font>
<br><font size=2 face="sans-serif"> return self.html_tmpl % d</font>
<br><font size=2 face="sans-serif"> MemoryError</font>
<br><font size=2 face="sans-serif"> $</font>
<br>
<br><font size=2 face="sans-serif">When I reran it this morning after checking the OS for errors, the same problem occured:</font>
<br>
<br><font size=2 face="sans-serif"> Updating HTML for article 12490</font>
<br><font size=2 face="sans-serif"> Updating HTML for article 12491</font>
<br><font size=2 face="sans-serif"> Updating HTML for article 12492</font>
<br><font size=2 face="sans-serif"> Updating index files for archive [2001-February]</font>
<br><font size=2 face="sans-serif"> Date</font>
<br><font size=2 face="sans-serif"> no mem for new parser</font>
<br><font size=2 face="sans-serif"> Traceback (most recent call last):</font>
<br><font size=2 face="sans-serif"> File "./arch", line 129, in ?</font>
<br><font size=2 face="sans-serif"> main()</font>
<br><font size=2 face="sans-serif"> File "./arch", line 119, in main</font>
<br><font size=2 face="sans-serif"> archiver.close()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 304, in close</font>
<br><font size=2 face="sans-serif"> self.update_dirty_archives()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 949, in update_dirty_archives</font>
<br><font size=2 face="sans-serif"> self.__super_update_dirty_archives()</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 513, in update_dirty_archives</font>
<br><font size=2 face="sans-serif"> self.update_archive(i)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperArch.py", line 952, in update_archive</font>
<br><font size=2 face="sans-serif"> self.__super_update_archive(archive)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 420, in update_archive</font>
<br><font size=2 face="sans-serif"> self._update_simple_index(hdr, archive, arcdir)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/pipermail.py", line 437, in _update_simple_index</font>
<br><font size=2 face="sans-serif"> article = self.database.getArticle(self.archive, msgid)</font>
<br><font size=2 face="sans-serif"> File "/mailman/Mailman/Archiver/HyperDatabase.py", line 279, in getArticle</font>
<br><font size=2 face="sans-serif"> article = self.__cache[msgid] = pickle.loads(buf)</font>
<br><font size=2 face="sans-serif"> MemoryError</font>
<br><font size=2 face="sans-serif"> $ </font>
<br>
<br><font size=2 face="sans-serif">The three cron jobs also appeared to fail today as well:</font>
<br>
<br><font size=2 face="sans-serif"> </font><font size=2><tt>Your "cron" job on arachnia</tt></font>
<br><font size=2><tt> /usr/local/bin/python -S /mailman/cron/qrunner</tt></font>
<br>
<br><font size=2><tt> produced the following output:</tt></font>
<br>
<br><font size=2><tt> Traceback (most recent call last):</tt></font>
<br><font size=2><tt> File "/mailman/cron/qrunner", line 85, in ?</tt></font>
<br><font size=2><tt> from Mailman import MailList</tt></font>
<br><font size=2><tt> File "/mailman/Mailman/MailList.py", line 43, in ?</tt></font>
<br><font size=2><tt> from Mailman.MailCommandHandler import MailCommandHandler </tt></font>
<br><font size=2><tt> MemoryError</tt></font>
<br>
<br><font size=2 face="sans-serif">--</font>
<br>
<br><font size=2 face="sans-serif"> </font><font size=2><tt>Your "cron" job on arachnia<br>
/usr/local/bin/python -S /mailman/cron/gate_news<br>
<br>
produced the following output:<br>
<br>
Traceback (most recent call last):<br>
File "/mailman/cron/gate_news", line 40, in ?<br>
from Mailman import MailList<br>
File "/mailman/Mailman/MailList.py", line 44, in ?<br>
from Mailman.HTMLFormatter import HTMLFormatter <br>
File "/mailman/Mailman/HTMLFormatter.py", line 27, in ?<br>
from Mailman.htmlformat import *<br>
MemoryError<br>
</tt></font>
<br><font size=2 face="sans-serif">--</font>
<br>
<br><font size=2 face="sans-serif"> </font><font size=2><tt>Your "cron" job on arachnia<br>
/usr/local/bin/python -S /mailman/cron/senddigests<br>
<br>
produced the following output:<br>
<br>
Traceback (most recent call last):<br>
File "/mailman/cron/senddigests", line 65, in ?<br>
main()<br>
File "/mailman/cron/senddigests", line 42, in main<br>
send_list_digest(mlist)<br>
File "/mailman/cron/senddigests", line 46, in send_list_digest<br>
mlist.Lock()<br>
File "/mailman/Mailman/MailList.py", line 1343, in Lock<br>
self.Load()<br>
File "/mailman/Mailman/MailList.py", line 903, in Load<br>
raise Errors.MMCorruptListDatabaseError, e<br>
Mailman.Errors.MMCorruptListDatabaseError: bad marshal data<br>
</tt></font>
<br><font size=2><tt>The machine is a Solaris 7 box with 256Meg of RAM</tt></font>
<br><font size=2><tt>Mailman version is 2.0.6</tt></font>
<br><font size=2><tt>Python version is 2.1</tt></font>
<br><font size=2><tt>The list in question has about 16,000 archived messages</tt></font>
<br>
<br><font size=2><tt>arch has no problem running against lists with fewer messages in the archives. It looks like the arch script might be overflowing some kind of Python memory buffer.</tt></font>
<br>
<br><font size=2><tt>Any ideas, solutions?</tt></font>
<br><font size=3><tt><br>
-- James<br>
<br>
</tt></font><font size=3 color=#b0b0b0><tt>o o o o o o o . . .</tt></font><font size=3><tt> </tt></font><font size=3 color=blue><tt>_______________________</tt></font><font size=3><tt> </tt></font><font size=3 color=red><tt>_______=======_</tt></font><font size=3><tt>T</tt></font><font size=3 color=red><tt>___</tt></font><font size=3><tt><br>
</tt></font><font size=3 color=#b0b0b0><tt>o</tt></font><font size=3><tt> _____ </tt></font><font size=3 color=blue><tt>|</tt></font><font size=3 color=#ff00ff><tt>James Madill </tt></font><font size=3 color=blue><tt>|</tt></font><font size=3><tt> </tt></font><font size=3 color=red><tt>|</tt></font><font size=3 color=#00aa00><tt>Duke Univ Med Ctr</tt></font><font size=3 color=red><tt>|</tt></font><font size=3><tt><br>
</tt></font><font size=3 color=#f0c02c><tt>></tt></font><font size=3><tt>.][__n_n_| D[ ====|____ </tt></font><font size=3 color=blue><tt>|</tt></font><font size=3 color=#ff00ff><tt>james.madill@duke.edu</tt></font><font size=3 color=blue><tt>|</tt></font><font size=3><tt> </tt></font><font size=3 color=red><tt>|</tt></font><font size=3 color=#00aa00><tt> (919) 286-6384 </tt></font><font size=3 color=red><tt>|</tt></font><font size=3><tt><br>
(________|__|_[____/____]_</tt></font><font size=3 color=blue><tt>|_____________________|</tt></font><font size=3><tt>_</tt></font><font size=3 color=red><tt>|_________________|</tt></font><font size=3><tt><br>
_/oo O-O-O ` oo oo 'o^o^o o^o^o` 'o^o o^o`<br>
</tt></font><font size=3 color=#b0802c><tt>-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-</tt></font><font size=3><tt><br>
<http://www.duke.edu/~madil001/></tt></font>
<br>
--=_alternative 005F161D85256A99_=--