I am taking a cyber security class. This week’s assignment had us work on Honeypots. Honeypot is a server that pretends to have a vulnerability of sorts (open ports, old software etc.) and instead collects data on people who are trying to hack it.
At the end of the experiment I ended up with some data for four honeypots. The data and my write up can be found here. In this post I wanted to focus on how I used Pandas and Python to help me gather some insight into data that I’ve collected.
Pandas is a great library for Python that makes it really easy to explore various kinds of data (JSON, CSV etc). It’s available via pip install pandas. I recommend using ipython (available via pip install ipython) for all simple Python tasks. If you are not used to the command line I recommend using Jupyter Notebook instead, which is like ipython in a browser.
Honeypots Used
- Dionaea with HTTP – Network Scanners – https://github.com/DinoTools/dionaea
- p0f – Network Scanners – https://github.com/threatstream/mhn/wiki/p0f-Sensor
- Suricata Sensor – IDS, IPS and NSM engine – https://github.com/threatstream/mhn/wiki/Suricata-Sensor
- ElasticHoney Sensor – Remote Code Execution in ES before 1.3.8 – https://github.com/threatstream/mhn/wiki/ElasticHoney-Sensor
Data Analysis
Load Data
|
1 2 3 4 5 6 7 8 9 10 |
import pandas as pd import json # Load data, can be found at https://github.com/akras14/codepath9/blob/master/session.json with open("session.json") as f: data = f.read() data = data.split("\n") data.pop() # Drop last empty element data = [json.loads(d) for d in data] |
Sample output
|
1 2 3 4 5 6 7 8 9 10 11 |
data [{'_id': {'$oid': '5ac00385616a1e781bfa54b3'}, 'destination_port': 80, 'honeypot': 'dionaea', 'hpfeed_id': {'$oid': '5ac00383616a1e781bfa54b2'}, 'identifier': 'e8351d14-352d-11e8-a320-42010a800002', 'protocol': 'httpd', 'source_ip': '199.201.64.145', 'source_port': 38877, 'timestamp': {'$date': '2018-03-31T21:54:11.887+0000'}}, # etc ... |
Load data into Panda’s DataFrame
|
1 2 3 |
df = pd.DataFrame.from_dict(data) df.iloc[0] # Show first item in dataframe |
Sample Output
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
_id {'$oid': '5ac00385616a1e781bfa54b3'} destination_ip NaN destination_port 80 honeypot dionaea hpfeed_id {'$oid': '5ac00383616a1e781bfa54b2'} identifier e8351d14-352d-11e8-a320-42010a800002 protocol httpd sensor NaN source_ip 199.201.64.145 source_port 38877 suricata NaN timestamp {'$date': '2018-03-31T21:54:11.887+0000'} Name: 0, dtype: object |
OK, that’s cool. Let see which IP hit me the most.
Show top 10 IPs and attack count
|
1 2 3 4 |
most_common = df['source_ip'].value_counts()[:10] # and most_common.to_dict() |
Sample output
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
10.128.0.8 4457 199.201.64.145 1965 5.188.11.145 1295 199.201.64.139 992 191.101.167.7 764 5.62.39.237 658 5.62.43.21 657 77.72.85.25 512 5.188.9.25 441 5.188.11.63 410 Name: source_ip, dtype: int64 # As dictionary {'10.128.0.8': 4457, '191.101.167.7': 764, '199.201.64.139': 992, '199.201.64.145': 1965, '5.188.11.145': 1295, '5.188.11.63': 410, '5.188.9.25': 441, '5.62.39.237': 658, '5.62.43.21': 657, '77.72.85.25': 512} |
Note: Dictionary is out of order. You can also do a for loop on most_common variable itself, but I wanted to demo a to_dict() conversion.
I wonder what attacks those IPs run on my honeypots?
Show attacks for most common IPs
|
1 2 3 |
for ip in most_common.to_dict(): print (ip, df[df['source_ip'] == ip]['honeypot'].unique()) |
Sample Output
|
1 2 3 4 5 6 7 8 9 10 11 |
10.128.0.8 ['suricata'] 199.201.64.145 ['dionaea'] 5.188.11.145 ['dionaea'] 199.201.64.139 ['dionaea'] 191.101.167.7 ['dionaea'] 5.62.39.237 ['dionaea'] 5.62.43.21 ['dionaea'] 77.72.85.25 ['dionaea'] 5.188.9.25 ['dionaea' 'p0f' 'suricata'] 5.188.11.63 ['dionaea' 'p0f' 'suricata'] |
Cool looks like two IPs were able to hit 3 out of the 4 honeypots. BTW, let’s check all of the attacks that took place.
Show All attacks that took place
|
1 2 |
df['honeypot'].value_counts() |
Sample Output
|
1 2 3 4 5 6 |
dionaea 21657 suricata 5454 p0f 2403 elastichoney 6 Name: honeypot, dtype: int64 |
Interesting. There were only 6 elastichoney attacks. It’s something that the most common IPs check would have overlooked. Let’s see which IPs they came from.
Show all IPs for elastichoney attack
|
1 2 |
df[df['honeypot'] == 'elastichoney']['source_ip'].unique() |
Sample Output
|
1 2 3 |
array(['125.212.217.215', '221.229.204.122', '216.218.206.68', '211.23.154.138'], dtype=object) |
And what kind of attacks did those IPs perform?
Types of attacks for each elastichoney IP found
|
1 2 3 4 5 |
for ip in df[df['honeypot'] == 'elastichoney']['source_ip'].unique(): print(ip) print(df[df['source_ip'] == ip]['honeypot'].value_counts()) print("\n") |
Sample Output
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
125.212.217.215 dionaea 9 p0f 5 elastichoney 3 Name: honeypot, dtype: int64 221.229.204.122 dionaea 32 p0f 1 elastichoney 1 Name: honeypot, dtype: int64 216.218.206.68 dionaea 2 elastichoney 1 Name: honeypot, dtype: int64 211.23.154.138 dionaea 9 p0f 4 elastichoney 1 Name: honeypot, dtype: int64 |
So looks like every IP that hit elastichoney pot, also hit other honeypots, but they did it only a few times. Probably to avoid getting detected by the sort of most common IP check that I ran first.
Please Share: