Logstash Example For Getting Information About SSH Brutefore Attack


 Logstash (Elastic Stack) Example For Geeting Information About SSH Brutefore Attack


 This is an example of a Logstash configuration for the purpose of obtaining SSH bruteforce information from the Filebeat index. This information is from the syslog obtained using the Filebeat module system. You can refer

https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-modules.html

GIST Github for this note.

https://gist.github.com/linuxmalaysia/8edba3f62a7ef5b4ce2351a0775e9de8

### Terima Kasih Kepada Amir Haris Ahmad, Localhost Sdn Bhd
### kerana izinkan saya gunakan servers ujian mereka di Digital Ocean
### dan team beliau dengan berkongsi pengalaman dan pandangan mereka.
###
### Untuk saya menguji bruteforce attack log kepada syslog dengan fail2ban
###
### Server telah dipasang dengan fail2ban dan SSH dibuka dengan port 22.
### SSH tidak membenarkan module password dan hanya digital cert.
###
### Filebeat telah digunakan untuk mengumpulkan log.
### Harisfazillah Jamel - 13102019 (13 Oct 2019)


### ---- Logstash ---- #####
## Start Of LOGSTASH Input

input {
  elasticsearch {
    hosts => "localhost"
    schedule => "1 */4 * * *"
##    user => xxxxxx
##    password => xxxxxx
    tags => "siem"
    index => "filebeat-*"
query => '{
       "query" : {
        "query_string": { "query": "event.type:authentication_failure" }},
    "sort": [
        { "@timestamp": {"order":"asc"}}
    ]
}'


  }
}

## End Of LOGSTASH Input

### Start Of LOGSTASH Filter

filter {

### process_time Put this at the beginning of filter
### To record process time. Make sure you copy the end script.

ruby {
    code => "event.set('[@metadata][task_start]', Time.now.to_f)"
 }


## For all index to have a field called IP.

mutate {
    add_field => { "ip" => "%{[source][ip]}" }
    }

##


if [ip] and "siem" in [tags] {
   fingerprint {
   source => "ip"
       target => "iocipfingerprint"
       method => "MURMUR3"
       add_tag => [ "siemip" ]
     }
}


## start for blueliv check
## More info https://github.com/Blueliv/elk-config-examples

if [ip] {

    elasticsearch {
    hosts => ["localhost:9200"]
    user => xxxxxxx
    password => xxxxxx
    index => "crimeservers*"
    query => "ip:%{ip}"
    fields => {
     "type" => "blueliv_type"
     "firstSeenAt" => "blueliv_firstSeenAt"
     "lastSeenAt" => "blueliv_lastSeenAt"
     "url" => "blueliv_url"
    }
    result_size => 1
    enable_sort => false
    add_tag => [ "blueliv" ]
  }

}

## end of blueliv check

### start for lisbot check
### https://gist.github.com/linuxmalaysia/5910941698f851947ed4aa2d9e44cf49
### change the dictionary_path with your own path

if [ip] and ![ip_rep] {

    translate {
      refresh_interval => 86400
      field => "ip"
      destination => "ip_rep"
      dictionary_path => "/etc/logstash/tools/listbot/iprep.yaml"
      add_tag => [ "listbot" ]
    }
}

### end of lisbot check

### process_time this will be at the end of whole ioc filter
###

    ruby {
    code => "event.set('[@metadata][task_end]', Time.now.to_f)"
    }

    ruby {
    code => "event.set('process_time', (event.get('[@metadata][task_end]') - event.get('[@metadata][task_start]')))"
    }

### End of process_time

}

### end of LOGSTASH filter

### Start of LOGSTASH output

output {

  if "siem" in [tags] {

elasticsearch {

    id => "server1-output"
    hosts => ["localhost:9200"]
    document_id => "%{iocipfingerprint}"
    sniffing => true
    index => "authentications-v1"
    user => xxxxxx
    password => xxxxxx
}

##end if
  }

}

### end of LOGSTASH output

### Mula Template
### https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
##### Template to be loaded
##### Namakan fail autentications-template.json dan gunakan arahan curl ini
# curl -uxxxxxx:XXXXXX -sS -i --insecure -XPUT "localhost:9200/_template/authentications" -H 'Content-Type: application/json' -d @autentications-template.json
###

{
    "index_patterns": "authentications-*",
        "order" : 0,
    "version" : 1,
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "dynamic_templates" : [
        {
          "message_field" : {
            "path_match" : "message",
            "mapping" : {
              "norms" : false,
              "type" : "text"
            },
            "match_mapping_type" : "string"
          }
        },
        {
          "string_fields" : {
            "mapping" : {
              "norms" : false,
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              }
            },
            "match_mapping_type" : "string",
            "match" : "*"
          }
        }
      ],
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "suricata.eve.timestamp" : {
          "type" : "date"
        },
        "geoip" : {
          "dynamic" : true,
          "properties" : {
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            }
          }
        },
        "location": {
             "type": "geo_point"
        },
        "source.ip": {
             "type": "ip"
        },
        "ip": {
             "type": "ip"
        },
       "@version" : {
          "type" : "keyword"
        }
      }
    },
    "aliases" : { }
}

#### Tamat template


This is an example of a Logstash configuration for the purpose of obtaining SSH bruteforce information from the Filebeat index. This information is from the syslog obtained using the Filebeat module system. You can refer

https://www.elastic.co/guide/en/beats/filebeat/master/configuration-filebeat-modules.html

GIST Github for this note.

https://gist.github.com/linuxmalaysia/8edba3f62a7ef5b4ce2351a0775e9de8

### Terima Kasih Kepada Amir Haris Ahmad, Localhost Sdn Bhd
### kerana izinkan saya gunakan servers ujian mereka di Digital Ocean
### dan team beliau dengan berkongsi pengalaman dan pandangan mereka.
###
### Untuk saya menguji bruteforce attack log kepada syslog dengan fail2ban
###
### Server telah dipasang dengan fail2ban dan SSH dibuka dengan port 22.
### SSH tidak membenarkan module password dan hanya digital cert.
###
### Filebeat telah digunakan untuk mengumpulkan log.
### Harisfazillah Jamel - 13102019 (13 Oct 2019)


### ---- Logstash ---- #####
## Start Of LOGSTASH Input

input {
  elasticsearch {
    hosts => "localhost"
    schedule => "1 */4 * * *"
##    user => xxxxxx
##    password => xxxxxx
    tags => "siem"
    index => "filebeat-*"
query => '{
       "query" : {
        "query_string": { "query": "event.type:authentication_failure" }},
    "sort": [
        { "@timestamp": {"order":"asc"}}
    ]
}'


  }
}

## End Of LOGSTASH Input

### Start Of LOGSTASH Filter

filter {

### process_time Put this at the beginning of filter
### To record process time. Make sure you copy the end script.

ruby {
    code => "event.set('[@metadata][task_start]', Time.now.to_f)"
 }


## For all index to have a field called IP.

mutate {
    add_field => { "ip" => "%{[source][ip]}" }
    }

##


if [ip] and "siem" in [tags] {
   fingerprint {
   source => "ip"
       target => "iocipfingerprint"
       method => "MURMUR3"
       add_tag => [ "siemip" ]
     }
}


## start for blueliv check
## More info https://github.com/Blueliv/elk-config-examples

if [ip] {

    elasticsearch {
    hosts => ["localhost:9200"]
    user => xxxxxxx
    password => xxxxxx
    index => "crimeservers*"
    query => "ip:%{ip}"
    fields => {
     "type" => "blueliv_type"
     "firstSeenAt" => "blueliv_firstSeenAt"
     "lastSeenAt" => "blueliv_lastSeenAt"
     "url" => "blueliv_url"
    }
    result_size => 1
    enable_sort => false
    add_tag => [ "blueliv" ]
  }

}

## end of blueliv check

### start for lisbot check
### https://gist.github.com/linuxmalaysia/5910941698f851947ed4aa2d9e44cf49
### change the dictionary_path with your own path

if [ip] and ![ip_rep] {

    translate {
      refresh_interval => 86400
      field => "ip"
      destination => "ip_rep"
      dictionary_path => "/etc/logstash/tools/listbot/iprep.yaml"
      add_tag => [ "listbot" ]
    }
}

### end of lisbot check

### process_time this will be at the end of whole ioc filter
###

    ruby {
    code => "event.set('[@metadata][task_end]', Time.now.to_f)"
    }

    ruby {
    code => "event.set('process_time', (event.get('[@metadata][task_end]') - event.get('[@metadata][task_start]')))"
    }

### End of process_time

}

### end of LOGSTASH filter

### Start of LOGSTASH output

output {

  if "siem" in [tags] {

elasticsearch {

    id => "server1-output"
    hosts => ["localhost:9200"]
    document_id => "%{iocipfingerprint}"
    sniffing => true
    index => "authentications-v1"
    user => xxxxxx
    password => xxxxxx
}

##end if
  }

}

### end of LOGSTASH output

### Mula Template
### https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
##### Template to be loaded
##### Namakan fail autentications-template.json dan gunakan arahan curl ini
# curl -uxxxxxx:XXXXXX -sS -i --insecure -XPUT "localhost:9200/_template/authentications" -H 'Content-Type: application/json' -d @autentications-template.json
###

{
    "index_patterns": "authentications-*",
        "order" : 0,
    "version" : 1,
    "settings" : {
      "index" : {
        "number_of_shards" : "1",
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
      "dynamic_templates" : [
        {
          "message_field" : {
            "path_match" : "message",
            "mapping" : {
              "norms" : false,
              "type" : "text"
            },
            "match_mapping_type" : "string"
          }
        },
        {
          "string_fields" : {
            "mapping" : {
              "norms" : false,
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "ignore_above" : 256,
                  "type" : "keyword"
                }
              }
            },
            "match_mapping_type" : "string",
            "match" : "*"
          }
        }
      ],
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        },
        "suricata.eve.timestamp" : {
          "type" : "date"
        },
        "geoip" : {
          "dynamic" : true,
          "properties" : {
            "ip" : {
              "type" : "ip"
            },
            "latitude" : {
              "type" : "half_float"
            },
            "location" : {
              "type" : "geo_point"
            },
            "longitude" : {
              "type" : "half_float"
            }
          }
        },
        "location": {
             "type": "geo_point"
        },
        "source.ip": {
             "type": "ip"
        },
        "ip": {
             "type": "ip"
        },
       "@version" : {
          "type" : "keyword"
        }
      }
    },
    "aliases" : { }
}

#### Tamat template

Popular Posts

Labels

64bit Activity Adempire advocate Akta Apache ASAS Azam backup backuppc Bash Beowulf Big Data Broadband Budget Centos Cinta Cluster CMS cmsfornerd Company Complain computer Computer Operation Conference Contest Data Centre Operation DBmail Digg Digital Certification Discussion Group Django DNS Docker Domain Duit Online Economy Elastic Stack Elasticsearch ELK email email server English Evangelist Events Family Tree Fedora File System Firefox Foss FOSS.my FreeBSD FTX Gesaan Gluster Gmail Godaddy.com Google Google App GTUG Hacking Hadoop hafnie Harisfazillah Jamel horde HP-UX hwclock IBM ICT Service Delivery and Operation Indonesia Internet Internet Tools Itanium Jabatan IT Negara Jaring Java Javascript Jepun Jiwang Joke Joomla Kesihatan Kibana KOSTEM ldap Linux Linux Counter linuxmalaysia Logstash Love Mailman MailScanner Mailwatch Malay Malaysia MAMPU MDeC meetup Melaka Melayu Merdeka Microsoft Migration mirror sites Money Online Monitor MOSC 2010 MOSC2010 mosc2011 MOSC2013 MOSCMY MOSCMY2014 MOSCMY2015 Mozilla MPI MSC Malaysia MSC Malaysia OSCONF MSCOSCONF My Love MyGOSSCON MyMeeting Mypenguin99 mysql Nagios NagiosQL Negaraku nss_ldap ntp OBW2014 Open Office Open Source openldap Openoffice.org OpenStack Opera OS2 OS400 OSCC OSCC MAMPU osdc.my OSS OSS Policy OWASP Parallel Computing People Power Personal Petition PGP PHP Pligg Politik Postfix Postgresql Programming Proxmox Python q1moscmy2015 Questionnaires Research Research tools RPM SASSIAN Sassian 85-89 Sassians 85-89 SCO Security Sekolah Sekolah Alam Shah Shell script Software License Solaris SongketMail SongketMailFilter sourceforge spam spamassassin Spoof Survey SVR4 System Tools Technorati Terjemahan Terminal TMnet Training translation Treasury Malaysia Trend Micro Twitter Ubuntu Unix Virtualization VMS VOIP Wang Web Server Windows Zimbra