[Web-SIG] Python version of WWW::Mechanize

[John J Lee]

On Wed, 26 Nov 2003, Simon Willison wrote:

> Perl's WWW::Mechanize module is awesome:
>
> http://www.perl.com/lpt/a/2003/01/22/mechanize.html
>
>    my $agent = WWW::Mechanize->new();
>    $agent->get("http://www.radiotimes.beeb.com/");
>    $agent->follow("My Diary");
>    $agent->form(2);
>    $agent->field("email", $email);
>    $agent->click();

Something like this would certainly be useful.  I've not done it because I
think it would be nice to have it know about standard browser objects like
frames, windows &c.  OTOH, it would be quick & easy to write something
like the above, and maybe provide a lot of the benefit -- especially the
follow_link and forward / back methods, plus handling of the Referer
header.  What interface did you have in mind for forms?

Something like this would be useful as a base class (which uses the code
in ClientCookie -- ie. HTTPCookieProcessor &c):

http://wwwsearch.sf.net/bits/ua.py


BTW, there's another Perl module like this, WWW::Automate.


> Would something like this be a worthwhile consideration for the Python
> web modules in the standard library, or is it specialised to the point
> that it works better as a separately maintained module?

I think, like any new stuff added to the library, it should be written and
in use before this question really arises.


John