HTTP or alternative upload for large files

[Jeremy Bowers]

robin at execulink.com wrote:
> I am attempting to upload large files (think 500 MB) through a web
> form using a Python CGI process.

Building on a previous reply:

You'll have to poke around the internals of Apache (or find somebody who 
can) and see if theres a way to intercept the stream before it's done 
being processed; perhaps PHP or something can do that, depending on how 
it's configured.

The other possibility is to create your own webserver, in Python if you 
like, that can stream the data from a client any way you like. It is not 
hard to create a web server, even from scratch, but the HTTP server 
modules in Python may be able to help. (They may also make the same 
assumption that Apache does about being able to load the whole request 
before passing it on to you.)

If they do not help, it is easy to create a server. Basically, put this 
server on another port, and have people only upload files to it. This is 
simple, but you can find information on it elsewhere.

The point of doing this is you'll get a socket, which you can pull 
information off of and do whatever you want, rather then a .5GB lump 
that's causing timeouts (and what happens if multiple people do this at 
once?).