Internet crawler

[Oleg Broytmann]

Всем мои извинения за почти полный off-topic.

   Сергей, ты тут жаловался на тормозного робота. Вот на эти исходники не
хочешь посмотреть:

  subject: webbase 5.9.0
 added by: Loic Dachary on Feb 23rd 2000, 13:07
  license: GPL
 category: Web/Development

 homepage: http://apps.freshmeat.net/homepage/931874164/
 download: http://apps.freshmeat.net/download/931874164/
changelog: http://apps.freshmeat.net/changelog/931874164/

description:
webbase is an Internet crawler. It is able to crawl and maintain
millions of URLs and store information about them in a MySQL database.
The interface is either a command line program or a C library. It
contains hooks to plug a full text indexing database.

changes:
This release features an upgrade to mifluz-0.13.0 and checks the syntax
of URLs given as arguments.

Oleg.
---- 
    Oleg Broytmann  National Research Surgery Centre  http://sun.med.ru/~phd/
           Programmers don't die, they just GOSUB without RETURN.