Python- javascript

Mike Paul paul.mike25 at gmail.com
Sat Aug 15 20:02:36 EDT 2009


I'm trying to scrap a dynamic page with lot of javascript in it.
Inorder to get all the data from the page i need to access the
javascript. But i've no idea how to do it.

Say I'm scraping some site htttp://www.xyz.com/xyz

request=urllib2.Request("htttp://www.xyz.com/xyz")
response=urllib2.urlopen(request)
data=response.read()


So i get all the data on the initial page. Now i need to access the
javascript on this page to get additional details. I've heard someone
telling me to use spidermonkey. But no idea  on how to send javscript
as request and get the response. How hsuld i be sending the javascript
request as ? how can it be sent?

This is the script I need to access.

<a href=# onclick="groups.render_box('data', {"page":
2});return false;">Click this to view more items</a>

Can anyone tell me how can i do it very clearly. I've been breaking my
head into this for the past few days with no progress.


Thanks



More information about the Python-list mailing list