Logstash (Elastic Stack) Example For Geeting Information About SSH Brutefore Attack
This is an example of a Logstash configuration for the purpose of obtaining SSH bruteforce information from the Filebeat index. This information is from the syslog obtained using the Filebeat module system. You can refer
https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-modules.html
GIST Github for this note.
https://gist.github.com/linuxmalaysia/8edba3f62a7ef5b4ce2351a0775e9de8
### Terima Kasih Kepada Amir Haris Ahmad, Localhost Sdn Bhd
### kerana izinkan saya gunakan servers ujian mereka di Digital Ocean
### dan team beliau dengan berkongsi pengalaman dan pandangan mereka.
###
### Untuk saya menguji bruteforce attack log kepada syslog dengan fail2ban
###
### Server telah dipasang dengan fail2ban dan SSH dibuka dengan port 22.
### SSH tidak membenarkan module password dan hanya digital cert.
###
### Filebeat telah digunakan untuk mengumpulkan log.
### Harisfazillah Jamel - 13102019 (13 Oct 2019)
### ---- Logstash ---- #####
## Start Of LOGSTASH Input
input {
elasticsearch {
hosts => "localhost"
schedule => "1 */4 * * *"
## user => xxxxxx
## password => xxxxxx
tags => "siem"
index => "filebeat-*"
query => '{
"query" : {
"query_string": { "query": "event.type:authentication_failure" }},
"sort": [
{ "@timestamp": {"order":"asc"}}
]
}'
}
}
## End Of LOGSTASH Input
### Start Of LOGSTASH Filter
filter {
### process_time Put this at the beginning of filter
### To record process time. Make sure you copy the end script.
ruby {
code => "event.set('[@metadata][task_start]', Time.now.to_f)"
}
## For all index to have a field called IP.
mutate {
add_field => { "ip" => "%{[source][ip]}" }
}
##
if [ip] and "siem" in [tags] {
fingerprint {
source => "ip"
target => "iocipfingerprint"
method => "MURMUR3"
add_tag => [ "siemip" ]
}
}
## start for blueliv check
## More info https://github.com/Blueliv/elk-config-examples
if [ip] {
elasticsearch {
hosts => ["localhost:9200"]
user => xxxxxxx
password => xxxxxx
index => "crimeservers*"
query => "ip:%{ip}"
fields => {
"type" => "blueliv_type"
"firstSeenAt" => "blueliv_firstSeenAt"
"lastSeenAt" => "blueliv_lastSeenAt"
"url" => "blueliv_url"
}
result_size => 1
enable_sort => false
add_tag => [ "blueliv" ]
}
}
## end of blueliv check
### start for lisbot check
### https://gist.github.com/linuxmalaysia/5910941698f851947ed4aa2d9e44cf49
### change the dictionary_path with your own path
if [ip] and ![ip_rep] {
translate {
refresh_interval => 86400
field => "ip"
destination => "ip_rep"
dictionary_path => "/etc/logstash/tools/listbot/iprep.yaml"
add_tag => [ "listbot" ]
}
}
### end of lisbot check
### process_time this will be at the end of whole ioc filter
###
ruby {
code => "event.set('[@metadata][task_end]', Time.now.to_f)"
}
ruby {
code => "event.set('process_time', (event.get('[@metadata][task_end]') - event.get('[@metadata][task_start]')))"
}
### End of process_time
}
### end of LOGSTASH filter
### Start of LOGSTASH output
output {
if "siem" in [tags] {
elasticsearch {
id => "server1-output"
hosts => ["localhost:9200"]
document_id => "%{iocipfingerprint}"
sniffing => true
index => "authentications-v1"
user => xxxxxx
password => xxxxxx
}
##end if
}
}
### end of LOGSTASH output
### Mula Template
### https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
##### Template to be loaded
##### Namakan fail autentications-template.json dan gunakan arahan curl ini
# curl -uxxxxxx:XXXXXX -sS -i --insecure -XPUT "localhost:9200/_template/authentications" -H 'Content-Type: application/json' -d @autentications-template.json
###
{
"index_patterns": "authentications-*",
"order" : 0,
"version" : 1,
"settings" : {
"index" : {
"number_of_shards" : "1",
"refresh_interval" : "5s"
}
},
"mappings" : {
"dynamic_templates" : [
{
"message_field" : {
"path_match" : "message",
"mapping" : {
"norms" : false,
"type" : "text"
},
"match_mapping_type" : "string"
}
},
{
"string_fields" : {
"mapping" : {
"norms" : false,
"type" : "text",
"fields" : {
"keyword" : {
"ignore_above" : 256,
"type" : "keyword"
}
}
},
"match_mapping_type" : "string",
"match" : "*"
}
}
],
"properties" : {
"@timestamp" : {
"type" : "date"
},
"suricata.eve.timestamp" : {
"type" : "date"
},
"geoip" : {
"dynamic" : true,
"properties" : {
"ip" : {
"type" : "ip"
},
"latitude" : {
"type" : "half_float"
},
"location" : {
"type" : "geo_point"
},
"longitude" : {
"type" : "half_float"
}
}
},
"location": {
"type": "geo_point"
},
"source.ip": {
"type": "ip"
},
"ip": {
"type": "ip"
},
"@version" : {
"type" : "keyword"
}
}
},
"aliases" : { }
}
#### Tamat template
This is an example of a Logstash configuration for the purpose of obtaining SSH bruteforce information from the Filebeat index. This information is from the syslog obtained using the Filebeat module system. You can refer
https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-modules.html
GIST Github for this note.
https://gist.github.com/linuxmalaysia/8edba3f62a7ef5b4ce2351a0775e9de8
### Terima Kasih Kepada Amir Haris Ahmad, Localhost Sdn Bhd
### kerana izinkan saya gunakan servers ujian mereka di Digital Ocean
### dan team beliau dengan berkongsi pengalaman dan pandangan mereka.
###
### Untuk saya menguji bruteforce attack log kepada syslog dengan fail2ban
###
### Server telah dipasang dengan fail2ban dan SSH dibuka dengan port 22.
### SSH tidak membenarkan module password dan hanya digital cert.
###
### Filebeat telah digunakan untuk mengumpulkan log.
### Harisfazillah Jamel - 13102019 (13 Oct 2019)
### ---- Logstash ---- #####
## Start Of LOGSTASH Input
input {
elasticsearch {
hosts => "localhost"
schedule => "1 */4 * * *"
## user => xxxxxx
## password => xxxxxx
tags => "siem"
index => "filebeat-*"
query => '{
"query" : {
"query_string": { "query": "event.type:authentication_failure" }},
"sort": [
{ "@timestamp": {"order":"asc"}}
]
}'
}
}
## End Of LOGSTASH Input
### Start Of LOGSTASH Filter
filter {
### process_time Put this at the beginning of filter
### To record process time. Make sure you copy the end script.
ruby {
code => "event.set('[@metadata][task_start]', Time.now.to_f)"
}
## For all index to have a field called IP.
mutate {
add_field => { "ip" => "%{[source][ip]}" }
}
##
if [ip] and "siem" in [tags] {
fingerprint {
source => "ip"
target => "iocipfingerprint"
method => "MURMUR3"
add_tag => [ "siemip" ]
}
}
## start for blueliv check
## More info https://github.com/Blueliv/elk-config-examples
if [ip] {
elasticsearch {
hosts => ["localhost:9200"]
user => xxxxxxx
password => xxxxxx
index => "crimeservers*"
query => "ip:%{ip}"
fields => {
"type" => "blueliv_type"
"firstSeenAt" => "blueliv_firstSeenAt"
"lastSeenAt" => "blueliv_lastSeenAt"
"url" => "blueliv_url"
}
result_size => 1
enable_sort => false
add_tag => [ "blueliv" ]
}
}
## end of blueliv check
### start for lisbot check
### https://gist.github.com/linuxmalaysia/5910941698f851947ed4aa2d9e44cf49
### change the dictionary_path with your own path
if [ip] and ![ip_rep] {
translate {
refresh_interval => 86400
field => "ip"
destination => "ip_rep"
dictionary_path => "/etc/logstash/tools/listbot/iprep.yaml"
add_tag => [ "listbot" ]
}
}
### end of lisbot check
### process_time this will be at the end of whole ioc filter
###
ruby {
code => "event.set('[@metadata][task_end]', Time.now.to_f)"
}
ruby {
code => "event.set('process_time', (event.get('[@metadata][task_end]') - event.get('[@metadata][task_start]')))"
}
### End of process_time
}
### end of LOGSTASH filter
### Start of LOGSTASH output
output {
if "siem" in [tags] {
elasticsearch {
id => "server1-output"
hosts => ["localhost:9200"]
document_id => "%{iocipfingerprint}"
sniffing => true
index => "authentications-v1"
user => xxxxxx
password => xxxxxx
}
##end if
}
}
### end of LOGSTASH output
### Mula Template
### https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
##### Template to be loaded
##### Namakan fail autentications-template.json dan gunakan arahan curl ini
# curl -uxxxxxx:XXXXXX -sS -i --insecure -XPUT "localhost:9200/_template/authentications" -H 'Content-Type: application/json' -d @autentications-template.json
###
{
"index_patterns": "authentications-*",
"order" : 0,
"version" : 1,
"settings" : {
"index" : {
"number_of_shards" : "1",
"refresh_interval" : "5s"
}
},
"mappings" : {
"dynamic_templates" : [
{
"message_field" : {
"path_match" : "message",
"mapping" : {
"norms" : false,
"type" : "text"
},
"match_mapping_type" : "string"
}
},
{
"string_fields" : {
"mapping" : {
"norms" : false,
"type" : "text",
"fields" : {
"keyword" : {
"ignore_above" : 256,
"type" : "keyword"
}
}
},
"match_mapping_type" : "string",
"match" : "*"
}
}
],
"properties" : {
"@timestamp" : {
"type" : "date"
},
"suricata.eve.timestamp" : {
"type" : "date"
},
"geoip" : {
"dynamic" : true,
"properties" : {
"ip" : {
"type" : "ip"
},
"latitude" : {
"type" : "half_float"
},
"location" : {
"type" : "geo_point"
},
"longitude" : {
"type" : "half_float"
}
}
},
"location": {
"type": "geo_point"
},
"source.ip": {
"type": "ip"
},
"ip": {
"type": "ip"
},
"@version" : {
"type" : "keyword"
}
}
},
"aliases" : { }
}
#### Tamat template
