Non blocking socket server and storage engine

[Chris Rebert]

On Wed, Nov 19, 2008 at 2:36 AM, kdeveloper
<Krzysztof.Retel at googlemail.com> wrote:
> Hello Pythonists,
>
> I am building a non blocking socket server for incomming UDP packets.
> The server needs to run at least three threads:
> 1. getting data and pushing to "some" storage (at the moment I use
> queue),
> 2. acknowledge the package received
> 3. retrieve the information from the storage and insert it in DB.
>
> The problem I have is that when I use queue it stores more packets in
> the queue than it actually receives. An example: sent 99 UDP packets
> and queue stored 600-750 entries (?) Why? I have no idea. I have
> impression that I still do not understand completely how does the
> queue work in python.

No, I believe rather you don't completely understand UDP.

Quoting from Wikipedia (http://en.wikipedia.org/wiki/User_Datagram_Protocol):
"UDP does not guarantee reliability or ordering in the way that TCP
does. Datagrams may arrive out of order, ***appear duplicated***, or
go missing without notice."

I think that might at least partially account for your duplicate entries.

Cheers,
Chris
-- 
Follow the path of the Iguana...
http://rebertia.com

>
> Another issue is that I want the server to run very rupidly. The
> server needs to get at least 100 packets per second, and the same
> amount of acknowledges has to be sent back.
>
> The questions are:
> 1. What is the best storage engine for Python and multithreading
> server?
> 2. Can I use queue for such problem?
> 3. How does actually queue work in Python? (I know how it should work
> generally, but somehow it doesn't work as I expect)
>
> Any hints & helps? Would be very grateful
>
> Cheers
> K
> --
> http://mail.python.org/mailman/listinfo/python-list
>