view http info when browsing
John J. Lee
jjl at pobox.com
Tue Sep 7 19:25:45 EDT 2004
santoshamb at yahoo.com (ben) writes:
> I'm trying to write a web client script in python to log onto a web
> page and pull some information off of it. The page has quite a few
> behind the scenes http things going on that are making it difficult to
> write the python script. I think if I could see the raw http data
> that comes to my browser life would be much easier. Does anybody know
> of programs that would run alongside my browser and show the http data
> that is send back and forth?
>
> I've looked at plugins like httpwatch and iewatch. httpwatch looks
> good but it costs quite a bit of money, and the trial is not fully
> functional. iewatch has a free trial, but they don't show cookie
> info, which is important.
>
> Does anybody know a good way of view the raw http?
[...]
from http://wwwsearch.sourceforge.net/bits/GeneralFAQ.html :
| I want to see what my web browser is doing, but standard network
| sniffers like ethereal or netcat (nc) don't work for HTTPS. How do I
| sniff HTTPS traffic?
|
| Three good options:
|
| Mozilla plugin: livehttpheaders.
| ieHTTPHeaders does the same for MSIE.
| Use lynx -trace, and filter out the junk with a script.
|
| I'm told you can also use a proxy like proxomitron (never tried it
| myself). There's also a commercial MSIE plugin.
URLs:
http://www.ethereal.com/
http://livehttpheaders.mozdev.org/
http://www.blunck.info/iehttpheaders.html
http://lynx.browser.org/
http://www.proxomitron.info/
http://www.simtec.ltd.uk/
There are plugins that show what session & permanent cookies your
browser has stored, too. Both Mozilla and Konqueror (& Safari and
Opera too, I guess?) do this as standard, but for IE:
http://www.codeproject.com/shell/cookiespy.asp?df=100&forumid=1989&exp=0&select=21591
John
More information about the Python-list
mailing list