gentoo.mahdi.cz
dev-go
:fetchbot
Simple, flexible web crawler that follows robots.txt policies and crawl delays
https://github.com/PuerkitoBio/fetchbot
fetchbot-1.3
BSD
download
~amd64 ~arm ~arm64 ~x86
pentoo