Python Screen Scraper

[Cameron Laird]

In article <mailman.6889.1177417037.32031.python-list at python.org>,
Michael Bentley  <michael at jedimindworks.com> wrote:
>
>On Apr 24, 2007, at 11:50 AM, James Stroud wrote:
>
>> Hello,
>>
>> Does anyone know of an example, however modest, of a screenscraper
>> authored in python? I am using Firefox.
>>
>> Basically, I am answering problems via my browser and being scored for
>> each problem. I have a tendency to go past my peak for training
>> efficiency, so I would like to scrape the result page for each  
>> problem I
>> answer, compile statistics, and have a program alert me when I should
>> stop (based on score and accuracy--assuming training value is  
>> related to
>> changes in these metrics).
>>
>> I have no idea how to go about writing such a beast and I am hoping  
>> that
>> I could get some pointers or an example that could get me going in the
>> right direction.
>>
>> Parsing, etc, is not a problem, but I'm not exactly sure how I might
>> interface python with Firefox, forwarding scraped pages to my browser
>> (or forwarding from the browser to the scraper).
>>
>> Thanks in advance for any help or advice.
>
>Possibly the easiest thing will be to read from firefox' cache.   
>Otherwise I think your only real options are to either build a proxy  
>or sniff the wire...
>
>

Oh, there are *lots* of ways to go about this.  How crucial is the
role of Firefox in all this?  <URL: http://www.greasespot.net/ >
might be the right answer.  Are you open to use of languages other
than Python?