[Flask] File upload breaks for files with size >64kb on ARM based apache wsgi servers
Michael Bayer
lists at mjbayer.de
Thu Oct 4 06:23:16 EDT 2018
Hi,
short update:
- On the ARM device, when I use the flask development wsgi server, I do
not have any issues
- Using Apache: Even with LogLevel "trace8" I do not see any logs
regarding the issue. I compared the traces from a successful upload with
a not successful one. Main difference is: The not successful one prints
out information about Content-Type and Content-Length and mod_wsgi
starts a new process. Both lines are missing in the logs of the
successful transmission.
Here the logs of a successful <64k file upload:
[] Headers received from client:, referer: http://192.168.178.38/trackdb/add
[] Host: 192.168.178.38, referer: http://192.168.178.38/trackdb/add
[] User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
Firefox/52.0, referer: http://192.168.178.38/trackdb/add
[] Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,
referer: http://192.168.178.38/trackdb/add
[] Accept-Language: de-DE, referer: http://192.168.178.38/trackdb/add
[] Accept-Encoding: gzip, deflate, referer:
http://192.168.178.38/trackdb/add
[] Referer: http://192.168.178.38/trackdb/add, referer:
http://192.168.178.38/trackdb/add
[] DNT: 1, referer: http://192.168.178.38/trackdb/add
[] Connection: keep-alive, referer: http://192.168.178.38/trackdb/add
[] Upgrade-Insecure-Requests: 1, referer:
http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/trackdb/, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.html, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.cgi, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.pl, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.php, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.xhtml, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/index.htm, referer: http://192.168.178.38/trackdb/add
[] Content-Type 'text/html; charset=utf-8' ..., referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'application/xml', referer:
http://192.168.178.38/trackdb/add
[] Content-Type condition for 'deflate' did not match, referer:
http://192.168.178.38/trackdb/add
[] Content-Type 'text/html; charset=utf-8' ..., referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'application/rss+xml', referer:
http://192.168.178.38/trackdb/add
[] Content-Type condition for 'deflate' did not match, referer:
http://192.168.178.38/trackdb/add
[] Content-Type 'text/html; charset=utf-8' ..., referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'application/x-javascript', referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'application/javascript', referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'application/ecmascript', referer:
http://192.168.178.38/trackdb/add
[] Content-Type condition for 'deflate' did not match, referer:
http://192.168.178.38/trackdb/add
[] Content-Type 'text/html; charset=utf-8' ..., referer:
http://192.168.178.38/trackdb/add
[] ... did not match 'text/css', referer: http://192.168.178.38/trackdb/add
[] Content-Type condition for 'deflate' did not match, referer:
http://192.168.178.38/trackdb/add
[] Content-Type 'text/html; charset=utf-8' ..., referer:
http://192.168.178.38/trackdb/add
[] ... matched 'text/html', referer: http://192.168.178.38/trackdb/add
[] Content-Type condition for 'deflate' matched, referer:
http://192.168.178.38/trackdb/add
[] Response sent with status 200, headers:, referer:
http://192.168.178.38/trackdb/add
[] Date: Thu, 04 Oct 2018 09:20:13 GMT, referer:
http://192.168.178.38/trackdb/add
[] Server: Apache/2.4.25 (Raspbian), referer:
http://192.168.178.38/trackdb/add
[] Vary: Accept-Encoding, referer: http://192.168.178.38/trackdb/add
[] Content-Encoding: gzip, referer: http://192.168.178.38/trackdb/add
[] Keep-Alive: timeout=5, max=99, referer:
http://192.168.178.38/trackdb/add
[] Connection: Keep-Alive, referer: http://192.168.178.38/trackdb/add
[] Transfer-Encoding: chunked, referer: http://192.168.178.38/trackdb/add
[] Content-Type: text/html; charset=utf-8, referer:
http://192.168.178.38/trackdb/add
[] core_output_filter: flushing because of FLUSH bucket
Here the logs of a not successfull >64k file upload:
[] Request received from client: POST /trackdb/add HTTP/1.1
[] Headers received from client:, referer: http://192.168.178.38/trackdb/add
[] Host: 192.168.178.38, referer: http://192.168.178.38/trackdb/add
[] User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
Firefox/52.0, referer: http://192.168.178.38/trackdb/add
[] Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8,
referer: http://192.168.178.38/trackdb/add
[] Accept-Language: de-DE, referer: http://192.168.178.38/trackdb/add
[] Accept-Encoding: gzip, deflate, referer:
http://192.168.178.38/trackdb/add
[] Referer: http://192.168.178.38/trackdb/add, referer:
http://192.168.178.38/trackdb/add
[] DNT: 1, referer: http://192.168.178.38/trackdb/add
[] Connection: keep-alive, referer: http://192.168.178.38/trackdb/add
[] Upgrade-Insecure-Requests: 1, referer:
http://192.168.178.38/trackdb/add
[] Content-Type: multipart/form-data;
boundary=---------------------------1459554725190952609947987496,
referer: http://192.168.178.38/trackdb/add
[] Content-Length: 80688, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/trackdb/add, referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of Require all granted: granted,
referer: http://192.168.178.38/trackdb/add
[] AH01626: authorization result of <RequireAny>: granted, referer:
http://192.168.178.38/trackdb/add
[] request authorized without authentication by access_checker_ex hook:
/add, referer: http://192.168.178.38/trackdb/add
[] mod_wsgi (pid=4976): Create interpreter '127.0.1.1|/trackdb'.
[] mod_wsgi (pid=4976, process='', application='127.0.1.1|/trackdb'):
Loading WSGI script '/var/www/trackdb/flaskapp.wsgi'., referer:
http://192.168.178.38/trackdb/add
[] Response sent with status 400, headers:, referer:
http://192.168.178.38/trackdb/add
[] Date: Thu, 04 Oct 2018 09:19:27 GMT, referer:
http://192.168.178.38/trackdb/add
[] Server: Apache/2.4.25 (Raspbian), referer:
http://192.168.178.38/trackdb/add
[] Content-Length: 192, referer: http://192.168.178.38/trackdb/add
[] Connection: close, referer: http://192.168.178.38/trackdb/add
[] Content-Type: text/html, referer: http://192.168.178.38/trackdb/add
[] core_output_filter: flushing because of FLUSH bucket
Michael
Am 03.10.2018 um 15:41 schrieb Michael Bayer:
>
> Am 02.10.2018 um 23:39 schrieb Geert Stappers:
>> On Tue, Oct 02, 2018 at 05:05:47PM +0200, Michael Bayer wrote:
>>> Hi everyone,
>>>
>>> I discovered a funny behavior of my flask app, when running on ARM
>>> based apache wsgi servers. Could you please give me some hints to
>>> debug this?
>>>
>>> I have a flask application, that supports file upload.
>>> It runs on an ARM based debian stable server with apache wsgi.
>>> If I upload small files, everything is fine. The problem appears,
>>> when I upload files with a file size >64kb. The famous "Bad request
>>> - The browser (or proxy) sent a request that this server could not
>>> understand." pops up.
>>>
>>> Things I tried out so far:
>>> - Deploying my flask app to my productive ARM based webserver (debian
>>> stable): Here I discovered the issue.
>>> - Running in flask's development wsgi server on my Development PC:
>>> No problem, I can upload files with any size I want.
>>> - Clean debian stable installation in a virtualbox on my Development
>>> PC (AMD64) same packages as my productive server, but AMD64 instead
>>> of armhf (apache, mod_wsgi): Again no problems at all.
>>> - Clean installation of raspian on a spare ARM Raspberry Pi (based on
>>> debian 9) with apache, mod_wsgi: Here comes the problem again!
>>> - Increase apache LogLevel to debug -> no error logged
>>> - Activate FileLogging inside my flask app -> no error logged
>>>
>>> Have you seen anything like this?
>> No.
>>
>> Thing that intriges me, is that 64K is a 16-bit limit.
>> ARMHF versus AMD64 is 32-bit vs 64-bit ...
> Thanks for your answer!
> Yes, this is strange. I also don't have an explanation
>>
>>> I unfortunately do not have much experience with web
>>> servers/development
>>> so I'm stuck with my debugging approaches :/
>>>
>>> What could I do besides that?
>> Consider it a none flask problem, it will allow you to look at Apache,
>> uWSGI, Python and other links in the chain. Is a webproxy involved??
>
> There is no proxy in the network.
>
>
>>> What information do you need from my side?
>>> I can provide any code/configuration you need.
>> That will help reproducing the problem.
> I created a minimal example to reproduce the error:
> https://github.com/mjbayer/flask-upload-issue-arm
>
> This repo contains code, configuration and example files that can be
> used for upload.
>
> Here is a demo Raspberry Pi (just for that purpose) to try it out:
> http://rgb.mjbayer.de/trackdb/
> (I could also give ssh access to the Raspi if required)
>
> If you run the code in flasks development wsgi server or apache
> mod_wsgi on AMD64, you will not see the issue.
>
>
> Michael
>
>
>
> _______________________________________________
> Flask mailing list
> Flask at python.org
> https://mail.python.org/mailman/listinfo/flask
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/flask/attachments/20181004/80a98bdd/attachment-0001.html>
More information about the Flask
mailing list