Project Group: com.widowcrawler

Terminator

com.widowcrawler : terminator

Terminator is a parser for robots.txt files.

Last Version: 0.1.1

Release Date:

Exo

com.widowcrawler : exo

Exo is a parser for sitemap.xml files.

Last Version: 0.1

Release Date:

  • 1