OSDN -- 开发和下载开源软件
简体中文翻译状态 translation status for zh

RSS
Download List

项目描述

Ebot is a scalable and distribuited Web crawler. The URLs are saved to a NOSQL database (which supports map/reduce queries) that you can query via RESTful HTTP requests or using your preferred programming languages. The URLs that need to be analyzed are sent to AMQP queues. In this way, it is possible to run several crawlers in parallel and stop and start them without losing URLs.

系统要求

System requirement is not defined
Information regarding Project Releases and Project Resources. Note that the information here is a quote from Freecode.com page, and the downloads themselves may not be hosted on OSDN.

2010-06-08 16:40
0.4

自定义/外部机构分析仪和网址正规化现在支持。配置文件被标准化。
Custom/external body analyzers and URL normalizers are now supported. The configuration files were standardised.

2010-05-30 04:26
0.3

初步支持增加了对Riak作为后端数据库，但仍然是默认的Apache couchdb。用于连接到远程数据库服务器和AMQP成功增加了新的选择。
Initial support was added for Riak as a backend database, though the default is still Apache couchdb. New options for connecting to remote DB and AMQP servers were added.

2010-05-20 05:46
2010-05-19

据统计，增加了为分贝（网址数，分贝大小），AMQP成功（队列计数）和网络（爬行与RRDtool的）。
Statistics have been added for db (number of urls, db size), amqp (queues counts), and web (crawlers) with RRDTOOL.

Project Resources

首页 Project Page