spiderfoot
SpiderFoot is an open source intelligence (OSINT) automation tool. It integrates with just about every data source available and utilises a range of methods for data analysis, making that data easy to navigate.
SpiderFoot has an embedded web-server for providing a clean and intuitive web-based interface but can also be used completely via the command-line. It's written in Python 3 and GPL-licensed.
FEATURES
- Web based UI or CLI
- Over 170 modules (see below)
- Python 3
- CSV/JSON/GEXF export
- API key export/import
- SQLite back-end for custom querying
- Highly configurable
- Fully documented
- Visualisations
- TOR integration for dark web searching
- Dockerfile for Docker-based deployments
- Can call other tools like DNSTwist, Whatweb and CMSeeK
- Actively developed since 2012!
USES
SpiderFoot's 170+ modules feed each other in a pub/sub model to ensure maximum data extraction to do things like:
- Host/sub-domain/TLD enumeration/extraction
- E-mail address enumeration/extraction
- Phone number extraction
- Bitcoin and Ethereum address extraction
- DNS zone transfers
- Threat intelligence and Blacklist queries
- API integraiton with SHODAN, HaveIBeenPwned, Censys, AlienVault, SecurityTrails, etc.
- Social media account enumeration
- S3/Azure/Digitalocean bucket enumeration/scraping
- IP geo-location
- Web scraping, web content analysis
- Image and binary file meta data analysis
- Office document meta data analysis
- Dark web searches
- So much more...
See it in action here, performing some DNS recon:
PURPOSE
SpiderFoot can be used offensively (e.g. in a red team exercise or penetration test) for reconnaissance of your target or defensively to gather information about what you or your organisation might have exposed over the Internet.
You can target the following entities in a SpiderFoot scan:
- IP address
- Domain/sub-domain name
- Hostname
- Network subnet (CIDR)
- ASN
- E-mail address
- Phone number
- Username
- Person's name
