read web page that requires javascript on client

[lkcl]

On Mar 20, 1:09 am, Greg <gregsaundersem... at gmail.com> wrote:
> On Mar 18, 7:25 pm, Carl <tg2.u... at gmail.com> wrote:
>
>
>
> > On Mar 18, 1:56 pm, a... at pythoncraft.com (Aahz) wrote:
>
> > > In article <mailman.2143.1237407931.11746.python-l... at python.org>,
> > > R. David Murray <rdmur... at bitdance.com> wrote:
>
> > > >That said, I've heard mention here of something that can apparently be
> > > >used for this.  I think it was some incarnation of Webkit.  I remember
> > > >someone saying you wanted to use the one with, I think it was GTK
> > > >bindings, even though you were dealing with just network IO.  But I don't
> > > >remember clearly and did not record the reference.  Perhaps the person
> > > >who posted that info will answer you, or you will be able to figure out
> > > >from these clues.  Unfortunately I'm not 100% sure it was Webkit.
>
> > > By the power of Gooja!
>
> > >http://groups.google.com/group/comp.lang.python/msg/aed53725885a9250
> > > --
> > > Aahz (a... at pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> > > "Programming language design is not a rational science. Most reasoning
> > > about it is at best rationalization of gut feelings, and at worst plain
> > > wrong."  --GvR, python-ideas, 2009-3-1
>
> > Probably the easiest thing is to actually use a browser. There are
> > many examples of automating a browser via Python. So, you can
> > programmatically launch the browser, point it to theJavaScript
> > afflicted page, let the JS run and grab the page source. As an added
> > bonus you can later interact with the page by programatically, filling
> > form fields, selecting options from lists and clicking buttons.
>
> > HTH, Carl
>
> Selenium. It's not pretty for what I want to do but it works ... then
> again, what I need to do is not pretty either.
> Ciao,
> Greg

http://seleniumhq.org/projects/remote-control/languages/python.html


intriguing.  five solutions.  although, using selenium forces you to
actually have the full complete firefox web browser, including running
the GUI itself, and then having a plugin actually in the browser
itself.

which is rather borked - but works.

l.