polipus 0.3.6
An easy to use distributed web-crawler framework based on Redis
Gemfile:
=
安裝:
=
Runtime 相依性套件 (9):
aws-s3
>= 0.6.3, ~> 0.6
bson_ext
>= 1.9.2, ~> 1.9
hiredis
>= 0.4.5, ~> 0.4
http-cookie
>= 1.0.1, ~> 1.0
mongo
>= 1.9.2, ~> 1.9.0
nokogiri
>= 1.6.0, ~> 1.6
redis
>= 3.0.4, ~> 3.0
redis-bloomfilter
>= 0.0.1, ~> 0.0
redis-queue
>= 0.0.3, ~> 0.0