From janssen at parc.com Mon Nov 15 19:48:39 2010 From: janssen at parc.com (Bill Janssen) Date: Mon, 15 Nov 2010 10:48:39 PST Subject: [Archiver-dev] UpLib and archiving In-Reply-To: <20101019144634.2400f766@mission> References: <83790.1287349272@parc.com> <20101018142859.4553f808@mission> <92627.1287450806@parc.com> <20101019144634.2400f766@mission> Message-ID: <11275.1289846919@parc.com> The UpLib site (http://uplib.parc.com/) is now running a Mercurial repository and Mailman mailing lists. Now to figure out how to hook UpLib up to Mailman... Bill From barry at python.org Mon Nov 15 19:59:54 2010 From: barry at python.org (Barry Warsaw) Date: Mon, 15 Nov 2010 13:59:54 -0500 Subject: [Archiver-dev] UpLib and archiving In-Reply-To: <11275.1289846919@parc.com> References: <83790.1287349272@parc.com> <20101018142859.4553f808@mission> <92627.1287450806@parc.com> <20101019144634.2400f766@mission> <11275.1289846919@parc.com> Message-ID: <20101115135954.1d1872bf@mission> On Nov 15, 2010, at 10:48 AM, Bill Janssen wrote: >The UpLib site (http://uplib.parc.com/) is now running a Mercurial >repository and Mailman mailing lists. Now to figure out how to hook >UpLib up to Mailman... Nice. I'll grab a copy and try to take a look soon. If you need help with the latter and want some higher bandwidth chats, look me up on #mailman on freenode. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From janssen at parc.com Mon Nov 15 21:35:34 2010 From: janssen at parc.com (Bill Janssen) Date: Mon, 15 Nov 2010 12:35:34 PST Subject: [Archiver-dev] Pipermail archive ifc to UpLib Message-ID: <15448.1289853334@parc.com> I thought I'd see how hard it would be to mock up an UpLib extension which would provide a Pipermail-like view of an UpLib email archive. Easy to get started: import sys, os, re, urllib, time from uplib.plibUtil import note, format_date from uplib.webutils import htmlescape from uplib.basicPlugins import show_title, STANDARD_BACKGROUND_COLOR from uplib.emailParser import Thread, figure_thread_elements, SHOW_BLOCK def render_list (repo, response, params): elist = params.get("list") hits = repo.do_query('+categories:"email/%s"' % elist) dates = sorted([doc.get_date() for score, doc in hits]) earliest = dates[0] latest = dates[-1] fp = response.open() fp.write("

%s

\n" % htmlescape(elist)) fp.write("

%s to %s, %d msgs\n" % (earliest, latest, len(hits))) fp.write("

\n") for year in range(latest[0], earliest[0]-1, -1): k1 = [x for x in dates if (x[0] == year)] if k1: for month in range(12, 1, -1): k2 = [x for x in k1 if (x[1] == month)] if k2: fp.write("") fp.write("' % query) fp.write("" % len(k2)) fp.write("\n") fp.write("
%s: " % htmlescape(format_date("%s/%s" % (month, year)))) # threads query = 'categories:"email/%s" AND date:[%s/1/%s TO %s/32/%s]' % (elist, month, year, month, year) query = urllib.quote_plus(query) fp.write('[Threads] ' % query) fp.write('[Date] %d messages
") def by_thread (repo, response, params): query = params.get("query") response.redirect("/action/basic/email_threads?query=%s&sort-order=by-latest-inverted" % urllib.quote_plus(query)) def by_date (repo, response, params): query = params.get("query") response.redirect("/action/basic/email_threads?query=%s&sort-order=by-start" % urllib.quote_plus(query)) This assumes that the email is already labelled with UpLib categories of the form "email/LISTNAME", and the parameter "list" specifies which LISTNAME you are looking for. Adding search by person would be interesting, because UpLib has a built-in notion of Person, with support for name aliases and such, so you could slice that several different ways. Here's one way: try: person = find_and_load_extension("Person") except ImportError: person = None def by_author (repo, response, params): def display_name(author): if person: n = person.Name(author) if n.reasonable(): return unicode(n) return author hits = [doc for score, doc in repo.do_query(params.get("query"))] hits = sorted(hits, key=lambda x: x.get_metadata("date")) authors = list(set([(doc.get_metadata("authors") or doc.get_metadata("email-sender")) for doc in hits])) authors = sorted([(display_name(author), author) for author in authors if author], key=lambda x: x[0].lower()) fp = response.open() fp.write("%s\n" % htmlescape(params.get("query"))) fp.write('\n') fp.write('\n') fp.write('\n') fp.write(SHOW_BLOCK); fp.write('\n') fp.write('\n' % STANDARD_BACKGROUND_COLOR) for display_name, author in authors: docs = [x for x in hits if ((x.get_metadata("authors") == author) or (x.get_metadata("email-sender") == author))] fp.write("

%s:

\n") fp.write("\n")