StatusParser

Retrieve the status codes from a list of URLs.

Installation:

git clone https://github.com/DFC302/statusparser.git
python3 setup.py install (may need sudo)

OR

git clone https://github.com/DFC302/statusparser.git
cd statusparser/
sudo chmod 755 statusparser

Requirements:

Python3

Modules
colorama
argparse
concurrent.futures
sys
socket
requests.packages.urllib3.exceptions - InsecureRequestWarning

Usage:

usage: statusparser.py [-h] [-f FILE] [-o OUT] [-t THREADS]
                       [--timeout TIMEOUT] [--errorfile] [--nocolors]
                       [--noerrors] [-s STATUSCODES [STATUSCODES ...]]

optional arguments:
  -h, --help            show this help message and exit
  -f FILE, --file FILE  Specify input file containing list of URLs
  -o OUT, --out OUT     Specify file to write output too.
  -t THREADS, --threads THREADS
                        Specify number of threads.
  --timeout TIMEOUT     Specify number in seconds for URL timeout. Default 3
  --errorfile           Erros will be written to Error_report.txt
  --nocolors            Print with no color output.
  --noerrors            Silence errors.
  -s STATUSCODE [STATUSCODE ...], --statuscode STATUSCODE [STATUSCODE ...]
                        Print only URL's that return a certain status code.

Example Usage:

File containing URLs are better off having proper scheme -- http:// or https://
However, if they do not, statusparser will add http:// to the domain in question. If the domain uses https, a redirect will catch it.

Basic usage
Usage: python3 statusparser -f [List of URLs file]

Write to file
Usage: python3 statusparser -f [List of URLs file] -o [Write to file]
Note: Basic redirection can also be used instead to keep colors in file ">>". WARNING: Doing this, will cause no output to print to screen while statusparser runs.

Using threads
Usage: python3 statusparser -f [List of URLs file] -t [number of threads #default 20]

Specify timeout
Note: timeout is applied to read and connect timeouts
Note: Lowering timeout may result in connection errors (false status return)
Usage: python3 statusparser -f [List of URLs file] --timeout [number #converted to a floating integer]

Send errors to a error file
Note: Errors are written to Error_report.txt
Note: Errors will continue to be appended to Error_report.txt. Delete or clear if you run multiple tests and do not want mixed or duplicate results.
Usage: python3 statusparser -f [List of URLs file] --errorfile

No color mode
Note: Run statusparser with no colors
Usage: python3 statusparser -f [List of URLs file] --nocolors

No error mode
Note: Suppress all errors
Note: If you want errors written to a file still, you will need to use --errorfile.
Usage: python3 statusparser -f [List of URLs file] --noerrors

StatusCode
Note: Only return certain status codes.
Note: Errors will still print, unless suppressed with --noerrors
Usage: python3 statusparser -f [List of URLs file] -s 200 404 403 (..etc, NO commas)

Common Status Codes You May Come Across:

Status Code Description
200 OK
301 Moved
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
410 Gone
500 Internal Server Error
503 Service Unavailable

GitHub