Python - scraping - javascript

[Mike Paul]

I'm trying to scrap a dynamic page with lot of javascript in it. Inorder to
get all the data from the page i need to access the javascript. But i've no
idea how to do it.

Say I'm scraping some site htttp://www.xyz.com/xyz

request=urllib2.Request("htttp://www.xyz.com/xyz")
response=urllib2.urlopen(request)
data=response.read()


So i get all the data on the initial page. Now i need to access the
javascript on this page to get additional details. I've heard someone
telling me to use spidermonkey. But no idea on how to send javscript as
request and get the response. How hsuld i be sending the javascript request
as ? how can it be sent?

This is the script I need to access.

<a href=# onclick="groups.render_box('data', {"page": 2});return
false;">Click this to view more items</a>

Can anyone tell me how can i do it very clearly. I've been breaking my head
into this for the past few days with no progress.


Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090815/aa2e7662/attachment.html>