[New-bugs-announce] [issue17355] http tests testing more than the error code are fragile

[karl]

New submission from karl:

Some of the tests of the HTTP Test suite are checking for the full status-line, instead of just the error code.


Why is it an issue?

1. The only mandatory part in the status-line is the error code. The phrase is optional. For example the response is made of 3 parts.

   HTTP/1.1 400 Bad Request
   ('HTTP/1.1', '400', 'Bad Request')

Only 400 is the mandatory part of the error code and is testing the error.

2. It creates dependencies on how we conceive, modify the construction of the status-line and error messages.

3. Some tests are testing messages with different "optional texts".

http://hg.python.org/cpython/file/3.3/Lib/test/test_httpservers.py#l640
Test for "HTTP/1.1 400 Line Too Long\r\n".
Instead of just testing "400". (A specific message could be put in the body, but that's informational)

http://hg.python.org/cpython/file/3.3/Lib/test/test_httpservers.py#l633
Test for "HTTP/1.1 414 Request-URI Too Long\r\n"
instead of just testing "414", btw the spec says for the optional message "URI Too Long"
http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-22#section-6.5.12

Testing the message, IMHO, should be only, for testing the construction of the message. The tests here are made for testing the right error code, but they try to do more.

Hope it helps :)

Threading: related to http://bugs.python.org/issue12921#msg183520

----------
messages: 183521
nosy: karlcow, orsenthil
priority: normal
severity: normal
status: open
title: http tests testing more than the error code are fragile
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue17355>
_______________________________________