Mechanoid Web Browser - Recording Capability

John J. Lee jjl at pobox.com
Tue Sep 19 14:35:08 EDT 2006


"Seymour" <seymour.morris at gmail.com> writes:
[...]
> one program, MaxQ, that records web surfing.  It seems to work great.
[...]

There are lots of such programs about (ISTR twill used to use MaxQ for
its recording feature, but I think Titus got rid of it in favour of
his own code, for some reason).  How useful they are depends on the
details of what you're doing: the information that goes across HTTP is
on a fairly low level, so e.g., most obviously, you may need to be
sending a session ID that varies per-request.  Those programs usually
have some way of dealing with that specific problem, but you may run
into other problems that have the same origin.  Don't let me put you
off it gets your job done, but it's good to be a bit wary: All current
web-scraping approaches using free software suck in one way or
another.


John




More information about the Python-list mailing list