/ Web Crawling & Web Scraping

A tiny web crawler in Python

A tiny web crawler in Python

A Tiny Web Crawler

A web crawler written in python.

Install Requirements

pip install validators beautifulsoup4 lxml

Python version: Python 3.6.3 :: Anaconda, Inc.

Run

python crawler.py

Starts crawling from root_url until max_link urls are crawled. Saves the output to out.json.

GitHub