client to upload big files via https and get progress info

[Sean DiZazzo]

On May 13, 9:39 am, News123 <news1... at free.fr> wrote:
> Hi Aaaz,
>
> Aahz wrote:
> > In article <4bea6b50$0$8925$426a7... at news.free.fr>,
> > News123  <news1... at free.fr> wrote:
> >> I'd like to perform huge file uploads via https.
> >> I'd like to make sure,
> >> - that I can obtain upload progress info (sometimes the nw is very slow)
> >> - that (if the file exceeds a certain size) I don't have to
> >>  read the entire file into RAM.
>
> > Based on my experience with this, you really need to send multiple
> > requests (i.e. "chunking").  There are ways around this (you can look
> > into curl's resumable uploads), but you will need to maintain state no
> > matter what, and I think that chunking is the best/simplest.
>
> I agree I need  chunking. (the question is just on which level of the
> protocol)
>
> I just don't know how to make a chunkwise file upload or what library is
> best.
>
> Can you recommend any libraries or do you have a link to an example?
>
> I'd like to avoid to make separate https post requests for the chunks
> (at least if the underlying module does NOT support keep-alive connections)
>
> I made some tests with high level chunking (separate sequential https
> post requests).
> What I noticed is a rather high penalty in data throughput.
> The reason is probably, that each request makes its own https connection
> and that either the NW driver or the TCP/IP stack doesn't allocate
> enough band width to my request.
>
> Therefore I'd like to do the chunking on a 'lower' level.
> One option would be to have a https module, which supports keep-alive,
>
> the other would be  to have a library, which creates a http post body
> chunk by chunk.
>
> What do others do for huge file uploads
> The uploader might be connected via ethernet, WLAN, UMTS, EDGE, GPRS. )
>
> N

You could also just send the file in one big chunk and give yourself
another avenue to read the size of the file on the server.  Maybe a
webservice that you call with the name of the file that returns it's
percent complete, or it could just return bytes on disk and you do the
math on the client side.  Then you just forget about the transfer and
query the file size whenever you want to know...or on a schedule.

~Sean