[Mailman-Users] SOLVED: How to purge correctly old messages ?

[BERTHOLD Jean]

Hello John,

I tried your command line and now it is ok.

I tried different solutions for editing large file under vi:

1. Add swap file (it is not working)
2. modify temporary storage for vi: set directory=/... (it is not working)
3. split my big file in several files it is the best solution

mailman at vanuatu # split -b 50m etrans-echanges.mbox 

mailman at vanuatu # ls -l x*
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:30 xaa
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:24 xab
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:24 xac
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:24 xad
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xae
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xaf
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xag
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xah
-rw-r--r--   1 mailman  mailman    65510 Feb  1 15:56 xai
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xaj
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xak
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:25 xal
-rw-r--r--   1 mailman  mailman  52428800 Feb  1 15:26 xam
-rw-r--r--   1 mailman  mailman  1810704 Feb  1 15:26 xan
mailman at vanuatu # 

After this split command it was possible to remove what I want.

mailman at vanuatu # cat xai xaj xak xal xam xan > TMP.mbox


mailman at vanuatu # du -sh *mbox
 202M   TMP.mbox
 652M   etrans-echanges.mbox
mailman at vanuatu # 

and mv TMP to etrans...

mailman at vanuatu # ../bin/arch --wipe etrans-echanges
...
...
...
Updating HTML for article 2604
Updating HTML for article 2605
Updating HTML for article 2606
Updating HTML for article 2607
Updating HTML for article 2608
Updating HTML for article 2609
Traceback (most recent call last):
  File "../bin/arch", line 187, in ?
    main()
  File "../bin/arch", line 177, in main
    archiver.close()
  File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 304, in close
    self.update_dirty_archives()
  File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 518, in update_dirty_archives
    self.update_archive(i)
  File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1032, in update_archive
    self.__super_update_archive(archive)
  File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 426, in update_archive
    self._update_thread_index(archive, arcdir)
  File "/usr/local/mailman/Mailman/Archiver/pipermail.py", line 480, in _update_thread_index
    self.update_article(arcdir, article, a1, a3)
  File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 1223, in update_article
    f.write(article.as_html())
  File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 426, in as_html
    lang=self._lang, mlist=self._mlist)
  File "/usr/local/mailman/Mailman/Archiver/HyperArch.py", line 195, in quick_maketext
    return Utils.uncanonstr(text, lang)
  File "/usr/local/mailman/Mailman/Utils.py", line 758, in uncanonstr
    u = unicode(s, charset)
MemoryError


Do you have an idea about this error ?

if I run my browser, only the two latest day of February are dislplayed:

----------------------------------
Les Archives de Etrans-Echanges
vous pouvez obtenir plus d' informations à propos de cette liste ou vous pouvez  télécharger les archives complètes ( 212 MB ). 

Archive Vue par: Version téléchargeable 
2 February 2004: [ Thread ] [ Sujet ] [ Auteur ] [ Date ]  [ Gzip'd Text 2 KB ] 
1 February 2004: [ Thread ] [ Sujet ] [ Auteur ] [ Date ]  [ Gzip'd Text 24 KB ] 
------------------------------------

however I kept archives from 20031031 (31 October 2003) to 2 February 2004.
I would like to display all messages between these dates...


Are my files too big ? 

mailman at vanuatu # du -sh etrans*
1015M   etrans-echanges
 203M   etrans-echanges.mbox
mailman at vanuatu # 



Thnks again

EOS Holding
Jean Berthold
Administrateur Unix & Oracle
Unité Gestion de l'infrastructure
Team Systèmes
Ch. de Mornex 10
Tél. +41(0)21 341 24 58
Fax +41(0)21 341 20 49
jean.berthold <mailto:jean.berthold at eosholding.ch>
 
...Unix is like a wigwam - no windows, no gates, apache inside...







-----Message d'origine-----
De : John DeCarlo [mailto:jdecarlo at mitre.org]
Envoyé : jeudi, 29. janvier 2004 16:07
À : BERTHOLD Jean
Cc : mailman-users at python.org
Objet : Re: [Mailman-Users] How to purge correctly old messages ?


Jean,

BERTHOLD Jean wrote:

> mailman at vanuatu # pwd
> /usr/local/mailman/archives/private/etrans-echanges.mbox
> mailman at vanuatu # ls -l
> total 1327072
> -rw-rw-r--   1 mailman  mailman  679109463 Jan 29 14:40 etrans-echanges.mbox
> mailman at vanuatu # vi etrans-echanges.mbox	
> mailman at vanuatu # vi etrans-echanges.mbox
> "etrans-echanges.mbox" Tmp file too large

>> You can also try and figure out which message in the mbox file you want 
>> to start archiving (instead of deleting them from the file) and run
>> 
>>   .../bin/arch --start=N --wipe etrans-echange
>> 
>> At least this should get you started.

I didn't pay very close attention to the size originally.

So my next guess is to try the "--start=N" option I mentioned above. 
See if you can figure out approximately how many messages you want to 
discard, by looking at the archives.

People with more Linux experience can suggest better approaches.

You can get fairly close to the number of messages in the .mbox file by 
running grep and wc:

grep "Return-Path:" etrans-echange.mbox | wc -l

This will tell you how many lines in the .mbox file have Return-Path: in 
them.

If not, just use trial and error.  Start with some number like half of 
the number you got above and if you have too many messages left in the 
archives, go up as needed.

-- 

John DeCarlo, My Views Are My Own