Crawler

Let's build something to extract all the content from the web. Who said that we should not use all the information available. Let's crawl, crawl and crawl again. We'll stop when there is nothing left.

2 posts

A Twitter Crawler and News Indexer

public – 5 min read

TrollHunter is a Twitter Crawler & News Website Indexer. It aims at finding Troll Farmers & Fake News on Twitter.

Apr 02, 2020

Find all URLs of a website in a few seconds - Python

public – 8 min read

Have you ever wanted to extract all the URLs of a website quickly? We'll tell you how! It is hundreds of times faster than crawling all the pages of a website to find all of its URLs.

Mar 02, 2020