爬虫框架，通用爬虫框架，设计参考八爪鱼、火车头等爬虫软件，采用“页面+接口”可以混用的方式进行采集数据。 任务因中断导致没采集完数据，后续执行不重复采集相同数据。

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to a Individualized crawler easily or can only use the default crawl components.

A lightweight web crawler.