Performance Issues with Threaded Python Network Server

[Joao Prado Maia]

> > Well, I think you misunderstood the issue here. Outlook 
> Express does a
> XOVER
> > 1-1500 (or whatever the range is) to get the header 
> information about
> all of
> > the articles, and then does a ARTICLE 'number' on every 
> article on the
> group
> > to get the actual body of the article. This only happens if the user
> selects
> > to have all messages downloaded to his hard disk.
> >
> Aah, I don't habitually download all articles, but pull them down as I
> read.

Me neither, but the user that actually suggested this feature for my
community site runs in a slow connection and likes to have the whole message
history on his computer. Go figure :)

> > It's not my fault. Outlook Express is the one asking for 
> 1500 articles
> one
> > at a time because the user wants to download the information of all
> articles
> > to his hard disk.
> Although if you set OE to download only new messages it should track
> what it already has and only pull down the new ones, right? Not much
> point dragging stuff down multiple times.

Indeed, Outlook Express will be smart enough to remember the articles that
it already has, and will only download the new ones. In any case, it is
still a problem if multiple clients ask for the same thing at once - instant
DoS on my silly NNTP server ;)


> > > > The NNTP server gets its information from a MySQL database
> > > (and no, MySQL
> > > is
> > > > not the bottleneck as far as I know, since 'top' shows the
> > > NNTP server
> > > > consuming 35% of CPU, not MySQL), formats the output by using
> string
> > > > replacement (aka "%s %s <%s@%s>" % (v,x,z,y)) and writes to
> > > the 'wfile'
> > > file
> > > > descriptor.
> > > >
> > > Your do_XOVER code doesn't look outrageously bad.
> > >
> >
> Closer inspection of the code suggests it's quite possibly the
> format_body() function in the back end that's gobbling up 
> your CPU. Try
> returning the argument, using only articles that don't need
> dot-escaping, and see if that's your problem.
> 

I will do that, thanks for the tip :)

-- Joao


--
Joao Prado Maia
Software QA
Bioinformatics Dept.
Lexicon Genetics, Inc.


*************************************************************************** 
 The contents of this communication are intended only for the addressee and
may contain confidential and/or privileged material. If you are not the
intended recipient, please do not read, copy, use or disclose this
communication and notify the sender.  Opinions, conclusions and other
information in this communication that do not relate to the official
business of my company shall be understood as neither given nor endorsed by
it.  
***************************************************************************