Generates Lists Of Dwell Hosts And URLs For Focusing on, Automating The Utilization Of MassDNS, Masscan And Nmap To Filter Out Unreachable Hosts And Collect Service Data

Generates lists of reside hosts and URLs for concentrating on, automating the utilization of Massdns, Masscan and nmap to filter out unreachable hosts

Given an enter file of domains, this script will automate the utilization of MassDNS to filter out unresolvable hosts, after which cross the outcomes on to Masscan to substantiate that the hosts are reachable and on which ports. The script will then generate a listing of full URLs for use for additional concentrating on (passing into instruments like gobuster or dirsearch, or making HTTP requests), a listing of reachable domains, and a listing of reachable IP addresses. As an elective final step, you possibly can run an nmap model scan on this decreased host listing, verifying that the sooner reachable hosts are up, and gathering service info from their open ports.

This script is very helpful for big area units, reminiscent of subdomain enumerations gathered from an apex area with hundreds of subdomains. With these giant lists, an nmap scan would merely take too lengthy. The objective right here is to first use the much less correct, however a lot quicker, MassDNS to shortly scale back the dimensions of your enter listing by eradicating unresolvable domains. Then, Masscan will be capable to take the output from MassDNS, and additional affirm that the hosts are reachable, and on which ports. The script will then parse these outcomes and generate lists of the reside hosts found.

Now, the listing of hosts ought to be decreased sufficient to be appropriate for additional scanning/testing. If you wish to go a step additional, you possibly can inform the script to run an nmap scan on the listing of reachable hosts, which ought to take extra affordable period of time with the shorter listing of hosts. After operating nmap, any false positives given from Masscan shall be filtered out. Uncooked nmap output shall be saved within the common nmap XML format, and extra info from the model detection shall be added to a SQLite database.

Set up

If utilizing the nmap scan choice, this device assumes that you have already got nmap put in

Word: Operating the set up script is just wanted if you don’t have already got MassDNS and Masscan put in, or if you want to reinstall them inside this repo. If you don’t run the script, you possibly can present the paths to the respective executables as arguments. The script moreover expects that the resolvers listing included with MassDNS be positioned at {massDNS_directory}/lists/resolvers.txt.

git clone https://github.com/allyomalley/LiveTargetsFinder.git
cd LiveTargetsFinder
sudo pip3 set up -r necessities.txt

(OPTIONAL)

chmod +x install_deps.sh
./install_deps.sh

If you don’t have already got MassDNS and Masscan put in, and would favor to put in them your self, see the documentation for directions:

MassDNS

Masscan

I’ve solely examined this script on macOS and Linux – the python script itself ought to work on a Home windows machine, although I consider the set up for MassDNS and Masscan will differ.

Utilization

python3 liveTargetsFinder.py [domainList] [options]

• Word that the Masscan and MassDNS settings are hardcoded inside liveTargetsFinder.py. Be at liberty to edit them (traces 87 + 97).
• Since this device was designed with very giant lists in thoughts, I tweaked most of the settings to attempt to steadiness velocity, accuracy, and community constraints – these can all be adjusted to fit your wants and bandwith.
• Default settings for Masscan solely scans ports 80 and 443.
  • -s, (--hashmap-size) specifically was chosen for efficiency causes – you’ll seemingly be capable to enhance this.
  • Full MassDNS arguments:
    • -c 25 -o J -r ./massdns/lists/resolvers.txt -s 100 -w massdnsOutput -t A targetHosts
    • Documentation
• One other setting of word is the --max-rate argument for Masscan – you’ll seemingly wish to modify this.
  • Full Masscan arguments:
    • -iL ipFile -oD masscanOutput --open-only --max-rate 5000 -p80,443 --max-retries 10
    • Documentation
• Default nmap settings solely scans ports 80 and 443, with timing -T4 and some NSE scripts.
  • Full nmap arguments:
    • --script http-server-header.nse,http-devframework.nse,http-headers -sV -T4 -p80,443 -oX {output.xml}

Instance

Did run set up script:

python3 liveTargetsFinder.py --target-list victim_domains.txt

Did NOT run the set up script:

python3 liveTargetsFinder.py --target-list victim_domains.txt --massdns-path ../massdns/bin/massdns --masscan-path ../masscan/bin/masscan 

Carry out an nmap scan and write to/append to the default DB path (liveTargetsFinder.sqlite3)

python3 liveTargetsFinder.py --target-list victim_domains.txt --nmap

Carry out an nmap scan and write to/append to the required database

python3 liveTargetsFinder.py --target-list victim_domains.txt --nmap --db-path serviceinfo_victim.sqlite3

Output

Enter: victimDomains.txt