[issue30458] CRLF Injection in httplib

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan <tir.karthi at gmail.com> added the comment:

See also https://bugs.python.org/issue36276 for a similar report. I think it's better to raise an error instead of encoding CRLF characters in URL similar to headers.

I feel either of the issue and more preferably issue36276 closed as a duplicate of this one. Copy of msg337968 with reference to details about similar report in golang : 

For reference an exact report on golang repo : https://github.com/golang/go/issues/30794 . This seemed to have been fixed in latest golang release 1.12 and commit https://github.com/golang/go/commit/829c5df58694b3345cb5ea41206783c8ccf5c3ca . The commit introduces a check for CTL characters and throws an error for URLs something similar to Python does for headers now at bf3e1c9b80e9.

func isCTL(r rune) bool {
	return r < ' ' || 0x7f <= r && r <= 0x9f
}

if strings.IndexFunc(ruri, isCTL) != -1 {
	return errors.New("net/http: can't write control character in Request.URL")
}

So below program used to work before go 1.12 setting a key on Redis but now it throws error : 

package main

import "fmt"
import "net/http"

func main() {
	resp, err := http.Get("http://127.0.0.1:6379?\r\nSET test failure12\r\n:8080/test/?test=a")
	fmt.Println(resp)
	fmt.Println(err)
}


➜  go version
go version go1.12 darwin/amd64
➜  go run urllib_vulnerability.go
<nil>
parse http://127.0.0.1:6379?
SET test failure12
:8080/test/?test=a: net/url: invalid control character in URL

Looking more into the commit there seemed to be a solution towards escaping characters with https://github.com/golang/go/issues/22907 . The fix seemed to have broke Google's internal tests [0] and hence reverted to have the above commit where only CTL characters were checked and raises an error. I think this is a tricky bug upon reading code reviews in the golang repo that has around 2-3 reports with a fix committed to be reverted later for a more conservative fix and the issue was reopened to target go 1.13 .

Thanks a lot for the report @ragdoll.guo

[0] https://go-review.googlesource.com/c/go/+/159157/2#message-39c6be13a192bf760f6318ac641b432a6ab8fdc8

----------
nosy: +xtreak

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue30458>
_______________________________________