PythonD: 4Suite or Twisted?

[Moshe Zadka]

On 05 Jun 2003, David Bolen <db3l at fitlinxx.com> wrote:

> However, the last time we specifically tried Twisted out for one of
> our applications - a distributed processing system - was probably
> around January/February - it's true that this was probably 1.0 (or
> 0.99.4 which was really close to 1.0).

In 1.0, UDP support was marked "unstable" for a reason -- the developers
had misgivings about the API. We have since changed it :)

> At that time, a big item that
> showed up was that the perspective broker, a key reason for looking at
> Twisted, didn't support UDP.

As far as I know, Perspective Broker is not slated to ever do UDP.
Unreliable transports are really bad for remote-object protocols which
are based on calling methods. I believe it is fairly easy, however,
to transport Jelly over UDP, if all you need is "unreliable high-level
messaging".

> The developer doing the testing also had
> problems getting some of the higher level objects and/or samples to
> function when trying to convert them from TCP to UDP

I'm sorry, converting from TCP to UDP doesn't make much sense to me.
The protocols have completely different characteristics. 

> I checked back last month (version 1.0.4) and saw the
> twisted.internet.udp class was still marked as "unstable" in the
> documentation and IReactorUDP as semi-stable.  This still seems true
> with 1.0.5 on the web site, and there aren't many UDP related comments
> in the change log.

Stability only changes before minor (not patch) releases. Specifically,
I predict that UDP will be stable by the time 1.1 rolls out, and with
roughly the same API. In other words, I consider the UDP stable enough
to develop against.

> All of this is really nice and well-structured, at least without UDP.
> UDP is twisted.internet.protocol.DatagramProtocol, but that doesn't
> even inherit from BaseProtocol, much less be a subclass of protocol.
> And because it's not a subclass of Protocol it doesn't get to play
> equally in a lot of the higher level support of Twisted.  Once you
> start getting into the higher order framework objects that layer on
> top of all of the factory and protocol work beneath, UDP starts to
> feel like the odd man out.

Um, it's not as much that UDP "feels" like the odd man out, but it is
the odd man out. SSL and UNIX domain sockets are reliable stream transports.
UDP is not.

> As a trivial example, take the "echoserv.py" example.  I can't see why
> that example can't become a UDP example just by changing the
> "listenTCP" call to "listenUDP".  However, because the example server
> is a subclass of Protocol, and the UDP DatagramProtocol class is going
> to call "datagramReceived" rather than Protocol's "dataReceived" it
> doesn't work.

This is correct. However, for any protocol *except* echo, this actually
makes sense. "datagramReceived" implies that the data packet has semantic
meaning, while "dataReceived" implies that the data packet is a part of
a stream, and may contain half the end of a previous message and half
the beginning of the next one. This is a profound semantic difference,
which cannot, and should not, be blurred by Twisted.

> I think my conclusions at the time were that the Protocol class was
> really a StreamProtocol class in disguise

Correct. Protocol is StreamProtcol.

> and there was probably no
> top level "protocol" abstraction in the system.

Also correct. However, I'm not sure I see the need for such an
abstraction.

> Datagram and Stream
> protocols can be like night and day at the protocol level, but there's
> also a lot of commonality that ought to be able to permit some of the
> higher level framework functionality to work in either case.

I disagree. If Perspective Broker (say) was to support UDP, it would
need an implementation of reliability on top of that. That is a whole
'nother level of support.

> The documentation doesn't tend to highlight TCP versus UDP
> differences, but just uses the factory and protocol level, so it's not
> generally clear where something won't work in both cases.

Nothing works in both cases. I think that's pretty clear.
We have a HOWTO explaining how to write UDP code.

> I just
> ended up feeling like trying to use UDP for a complex application with
> Twisted (as opposed to a dedicated UDP protocol implementation like
> DNS) was swimming against the tide.  I think that's still a valid
> state of affairs at the moment.

You're right, but it's not the Twisted tide you're swimming against.
Most of the time, UDP is the wrong solution, period. When it is,
for example for media-streaming, it needs to be treated differently.
There is no sense of writing reliable transports on top of UDP when
TCP exists.
 
-- 
Moshe Zadka -- http://moshez.org/
Buffy: I don't like you hanging out with someone that... short.
Riley: Yeah, a lot of young people nowadays are experimenting with shortness.
Agile Programming Language -- http://www.python.org/