TPCServer and xdrlib

[Laszlo Nagy]

>> I'm trying to write a multi threaded TPC server. I have used xmlrpc
>>     
>
>  How exactly did you come to the conclusion that your server must be
> multi threaded ?
>   
I don't think that it is important. But if you are interested:

- yes, the server will probably be I/O bound, not CPU bound
- I'm have experience with thread programming, but not with twisted

>> - I have to send larger amounts of data, the overhead of converting to
>> XML and parsing XML back would be too much pain
>>     
>
>  - What's the expected amount of data you have to transfer ?
>   
I cannot predict. But I will be trasferring image files which would be 
silly to do with XML.
>  - What's the expected network bandwidth ?
>   
It cannot be determined in advance.
>  - What's the expected acceptable transfer time ?
>   
Not known.
>  - How many users are expected to be transfering data at the same time ?
>   
The server should be scaleable up to hundreds of users. (I'm just trying 
to answer your questions, if that helps to answer mine.)
>  Did you consider gzipping your XML (or YAML) packets ? Would the
> transfer time be acceptable in this case ?
>   
No. "Image binary data -> base64encode -> XML -> gzip" - looks very 
silly. It cannot be efficient. Do you have better ideas?
>> BTW I do not care about the clients - they must trust the server side.
>>     
>
>  Oh, he said he _doesn't care about the clients_ ! ;-)
>   
I meant *safety* here: clients are going to download program updates 
from the server. So if they do not trust the server then they should not 
use it. The server is different: it must be safe against external 
attacks. Maybe it was my bad English? Sorry for the misunderstanding.
>  In general I would avoid that. Try to better estimate the speed
> requirements, to see if you really need do to this.
>   
I cannot predict "acceptable speed" requirements, but I can tell that 
there will be some clients downloading 100MB report files from the 
server, so I presume that I will need a progress bar. I think that I 
need to develop my own protocol for this, and probably the underlying 
layer should use binary representation.
>> Before I start re-inventing the wheel:
>>
>> - Is there another (already existing) higher level framework that I can
>> try? It should be safe and fast, that is the only restriction.
>>     
>
>  There's "Twisted".
>  http://twistedmatrix.com/projects/core/documentation/howto/servers.html
>   
Yes, I tried twisted before and I did not like it. It forces me to 
things that I do not want to do. (I cannot tell what it was - it was two 
years ago.)
>> - Do you think that it is a good idea to use xdrlib? I haven't seen
>> projects using it directly. For me it is like the rotor module was - it
>>     
>
>  It's probably the best way to send binary stuff over the network.
> But, again, I would avoid doing so.
>   
It is NOT the best way. Just to tell one example: big endian / little 
endian integers. Definitely I need some encoding.

(But if you are right and this is the best way, why would you avoid?)

  L