[Baypiggies] urllib2, basic HTTP authentication, and tr.im
Damon McCormick
damonmc at gmail.com
Wed Mar 11 23:37:37 CET 2009
I've used a tool called Charles <http://www.charlesproxy.com/> for http,
and Wireshark <http://www.wireshark.org/> is the standard for general
protocol analysis. Both run on OS X.
-Damon
On Wed, Mar 11, 2009 at 3:14 PM, Daryl Spitzer <daryl.spitzer at gmail.com>wrote:
> Thanks Benjamin. I should have wrote sooner to add that I've
> confirmed that curl -u works. So the problem is definitely with my
> Python basic HTTP authentication code.
>
> I also should have explained that I want to use urllib2 because (as
> Michael Foord wrote in
> http://www.voidspace.org.uk/python/articles/urllib2.shtml#proxies),
> urllib2 will auto-detect proxy settings and use them; curl--as far as
> I know--doesn't auto-detect proxy settings. (So my code using query
> string parameters works fine at work through a proxy server and at
> home.)
>
> Also, I'd just like to learn why my code doesn't work!
>
> I guess I should take jj's advice and look at what is being sent over
> the wire. (And compare that to what's going over the wire with curl.)
> I've never done that before. Can anyone recommend a tool for that
> (preferably that works on Mac OS X)?
>
> --
> Daryl
>
>
> On Wed, Mar 11, 2009 at 2:07 PM, Benjamin Sergeant <bsergean at gmail.com>
> wrote:
> > Warning: quick and dirty solution with no error checking using curl and a
> > unix pipe.
> >
> > - Benjamin
> >
> > [bsergean at localhost bin]$ cat trim.py
> > #!/usr/bin/env python
> >
> > import os, sys
> > from getpass import getpass
> > import xml.dom.minidom
> >
> > if len(sys.argv) != 2:
> > print 'Usage: trim.py <url to shorten>'
> > sys.exit(0)
> >
> > user = raw_input('username: ')
> > passwd = getpass('password: ')
> >
> > # See http://tr.im/api
> > url = 'http://api.tr.im/api/trim_url.xml?url=%s&newtrim=yes' %
> sys.argv[1]
> >
> > # Warning: --silent don't show error messages
> > cmd = 'curl --silent --basic -u %s:%s ' % (user, passwd)
> > cmd += url
> >
> > fd = os.popen(cmd, 'r')
> > response = fd.read()
> > print response
> > fd.close()
> >
> > Doc = xml.dom.minidom.parseString(response)
> > N = Doc.getElementsByTagName("url")
> > shortened_url = str((N[0].childNodes)[0].data)
> >
> > print 'shortened url:', shortened_url
> >
> >
> > [bsergean at localhost bin]$ python trim.py
> > Usage: trim.py <url to shorten>
> > [bsergean at localhost bin]$ python trim.py
> > http://www.reddit.com/r/programming/
> > username: bsergean
> > password:
> > <?xml version="1.0" encoding="UTF-8"?>
> > <trim>
> > <status result="OK" code="200" message="tr.im URL Added."/>
> > <url>http://tr.im/hglC</url>
> > <reference>ZfCAq7qcyouYXwfR2WsuBR6v3V4q7L</reference>
> > <trimpath>hglC</trimpath>
> > </trim>
> >
> > shortened url: http://tr.im/hglC
> >
> >
> >
> >
> >
> >
> > On Tue, Mar 10, 2009 at 11:51 PM, Shannon -jj Behrens <jjinux at gmail.com>
> > wrote:
> >>
> >> It may help to look at what is being sent over the wire.
> >>
> >> -jj
> >>
> >> On Tue, Mar 10, 2009 at 8:26 AM, Daryl Spitzer <daryl.spitzer at gmail.com
> >
> >> wrote:
> >> > I'm playing around, trying to write some code to use the http://tr.im
> >> > APIs (http://tr.im/api/) to shorten a URL.
> >> >
> >> > After reading http://docs.python.org/library/urllib2.html, I tried:
> >> >
> >> > TRIM_API_URL = 'http://api.tr.im/api'
> >> > auth_handler = urllib2.HTTPBasicAuthHandler()
> >> > auth_handler.add_password(realm='tr.im',
> >> > uri=TRIM_API_URL,
> >> > user=USERNAME,
> >> > passwd=PASSWORD)
> >> > opener = urllib2.build_opener(auth_handler)
> >> > urllib2.install_opener(opener)
> >> > response = urllib2.urlopen('%s/trim_simple?url=%s'
> >> > % (TRIM_API_URL, url_to_trim))
> >> > url = response.read().strip()
> >> >
> >> > response.code is 200 (I think it should be 202). url is valid, but
> >> > the basic HTTP authentication doesn't seem to have worked, because the
> >> > shortened URL isn't in my list of URLs (at http://tr.im/?page=1).
> >> >
> >> > After reading
> >> >
> http://www.voidspace.org.uk/python/articles/authentication.shtml#doing-it-properly
> >> > I also tried:
> >> >
> >> > TRIM_API_URL = 'api.tr.im/api'
> >> > password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
> >> > password_mgr.add_password(None, TRIM_API_URL, USERNAME, PASSWORD)
> >> > auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
> >> > opener = urllib2.build_opener(auth_handler)
> >> > urllib2.install_opener(opener)
> >> > response = urllib2.urlopen('http://%s/trim_simple?url=%s'
> >> > % (TRIM_API_URL, url_to_trim))
> >> > url = response.read().strip()
> >> >
> >> > But I get the same results. (response.code is 200 and url is valid,
> >> > but not recorded in my account at http://tr.im/.)
> >> >
> >> > If I use query string parameters instead of basic HTTP authentication,
> >> > like this:
> >> >
> >> > TRIM_API_URL = 'http://api.tr.im/api'
> >> > response =
> >> > urllib2.urlopen('%s/trim_simple?url=%s&username=%s&password=%s'
> >> > % (TRIM_API_URL,
> >> > url_to_trim,
> >> > USERNAME,
> >> > PASSWORD))
> >> > url = response.read().strip()
> >> >
> >> > ...then not only is url valid but it's recorded in my tr.im account.
> >> > (Though response.code is still 200.)
> >> >
> >> > Either there must be something wrong in my authentication code, or in
> >> > tr.im's. (I've written api at tr.im to see if it's a known problem and
> >> > haven't received a response.)
> >> >
> >> > Can anyone spot any problems?
> >> >
> >> > --
> >> > Daryl
> >> > _______________________________________________
> >> > Baypiggies mailing list
> >> > Baypiggies at python.org
> >> > To change your subscription options or unsubscribe:
> >> > http://mail.python.org/mailman/listinfo/baypiggies
> >> >
> >>
> >>
> >>
> >> --
> >> In this life we cannot do great things. We can only do small things
> >> with great love. -- Mother Teresa
> >> http://jjinux.blogspot.com/
> >> _______________________________________________
> >> Baypiggies mailing list
> >> Baypiggies at python.org
> >> To change your subscription options or unsubscribe:
> >> http://mail.python.org/mailman/listinfo/baypiggies
> >
> >
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20090311/7dbae134/attachment-0001.htm>
More information about the Baypiggies
mailing list