A Tiny Web Crawler
A web crawler written in python.
Install Requirements
pip install validators beautifulsoup4 lxml
Python version: Python 3.6.3 :: Anaconda, Inc.
Run
python crawler.py
Starts crawling from root_url until max_link urls are crawled. Saves the output to out.json.
