Devloped By Ashutosh Raval

                                            
                                            
  .--.--.       ___                         
 /  /    '.   ,--.'|_                       
|  :  /`. /   |  | :,'              __  ,-. 
;  |  |--`    :  : ' :            ,' ,'/ /| 
|  :  ;_    .;__,'  /    ,--.--.  '  | |' | 
 \  \    `. |  |   |    /       \ |  |   ,' 
  `----.   \:__,'| :   .--.  .-. |'  :  /   
  __ \  \  |  '  : |__  \__\/: . .|  | '    
 /  /`--'  /  |  | '.'| ," .--.; |;  : |    
'--'.     /   ;  :    ;/  /  ,.  ||  , ;    
  `--'---'    |  ,   /;  :   .'   \---'     
               ---`-' |  ,     .-./         
                       `--`---'

A web crawler script that crawls the target website and lists its links || A web crawler script that lists links by scanning the target website.

Module to be installed

  • Installation for Linux

sudo pip install -r requirements.txt
  • Installation for Windows

python -m pip install -r .\requirements.txt

Use

*Example Use 1

python spider.py --url http://example.com/etc
  • *Example Use 1

python spider.py -u http://example.com/etc

GitHub

https://github.com/Ashutosh1Raval/Star