Setting up an ELK server

peter hartley > Cybersecurity > Setting up an ELK server

Automated ELK Stack Deployment

The files in this repository were used to configure the network depicted below.

These files have been tested and used to generate a live ELK deployment on Azure. They can be used to either recreate the entire deployment pictured above. Alternatively, select portions of the file can be used to intall portions of the ELK server such as filebeat.

This document contains the following details:

  • Description of the Topology
  • Access Policies
  • ELK Configuration
    • Beats in Use
    • Machines Being Monitored
  • How to Use the Ansible Build

Description of the Topology

The main purpose of this network is to expose a load-balanced and monitored instance of DVWA, the D*mn Vulnerable Web Application.

Load balancing ensures that the application will be highly available, in addition to restricting inbound access to the network. Load balancers distribute the traffic among the three webservers ensuring that only authorized users will be able to access the virtual machines.

Integrating an ELK server allows users to easily monitor the vulnerable VMs for changes to the file system and system metrics such as CPU usage, logins, sudo escalation attempts and more.

The configuration details of each machine may be found below.

NameFunctionIP AddressOperating System
Jump BoxGateway10.0.0.5Linux
Web1Web Server10.0.0.7Linux
web2Web Server10.0.0.8Linux
Web3Web Server10.0.0.9Linux
Elk-ServerElk Server10.1.0.5Linux

ELK Server Configuration

The ELK VM exposes an Elastic Stack instance. Docker is used to download and manage an ELK container.

Rather than configure ELK manually, we opted to develop a reusable Ansible Playbook to accomplish the task. This playbook is duplicated below.

To use this playbook, one must log into the Jump Box, then issue: ansible-playbook elk.yml. This runs the elk.yml playbook on the elk host.

- name: Configure Elk VM with Docker
  hosts: elk
  remote_user: sysadmin
  become: true
    # Use apt module
    - name: Install
        update_cache: yes
        force_apt_get: yes
        state: present

      # Use apt module
    - name: Install pip3
        force_apt_get: yes
        name: python3-pip
        state: present

      # Use pip module
    - name: Install Docker python module
        name: docker
        state: present

      # Use command module
    - name: Increase virtual memory
      command: sysctl -w vm.max_map_count=262144

      # Use sysctl module
    - name: Use more memory
        name: vm.max_map_count
        value: "262144"
        state: present
        reload: yes

      # Use docker_container module
    - name: download and launch a docker elk container
        name: elk
        image: sebp/elk:761
        state: started
        restart_policy: always
          - 5601:5601
          - 9200:9200
          - 5044:5044

      # Enable docker service
    - name: Enable docker service
        name: docker
        enabled: yes

Access Policies

The machines on the internal network are not exposed to the public Internet.

Only the jump box machine can accept connections from the Internet. Access to this machine is only allowed from the following IP addresses:


Machines within the network can only be accessed by each other.

  • Web1 Web2, and Web3 send traffic to the ELK Server

A summary of the access policies in place can be found in the table below.

NamePublicly AccessibleAllowed IP Addresses
Jump BoxYes72.229.144.217

Elk Configuration

Ansible was used to automate configuration of the ELK machine. No configuration was performed manually, which is advantageous because using the configuration can be repeated automatically as new machines are added. Also if updates need to be made the changes can take place in one file and then run to updated the individual machines.

The playbook implements the following tasks:

  • Install docker
  • Install python
  • Install Docker python module
  • Increase virtual memory
  • Download and launch a docker ELK container
  • Enable docker service

The following screenshot displays the result of running docker ps after successfully configuring the ELK instance.

Target Machines & Beats

This ELK server is configured to monitor the following machines:


We have installed the following Beats on these machines:

  • Filebeat – Detects changes to the filesystem. Collects Apache logs
  • Metricbeat – Detects changes in filesystem metrics such as CPU usage SSH login attempts, failed sudo escalations and CPU/RAM statistics
  • Auditbeat – Audits the activities of users and processes from the Linux Audit Framework. Detect changes to critical files, like binaries and configuration files.
  • Packetbeat – Collects packets that pass througgh the NIC, similar to Wireshark. This traces all activity that takes place on the network.

Using the Playbook

In order to use the playbooks, you will need to have an Ansible control node already configured. Assuming you have such a control node provisioned:

SSH into the control node and follow the steps below:

  • update hosts file within /etc/ansible and include the IPs of the Webservers
[webservers] ansible_python_interpreter=/usr/bin/python3 ansible_python_interpreter=/usr/bin/python3 ansible_python_interpreter=/usr/bin/python3

[elk] ansible_python_interpreter=/usr/bin/python3
  • Copy the playbook files to the appriopriate directories in /etc/ansible (ex.: /etc/ansible/filebeat/filebeat-playbook.yml).
  • Update the configuration files for each beat to include the ELK server host information
  • Copy the configuration files to the appropriuate directory within /etc/ansible (ex: /etc/ansible/filebeat/filebeat.yml)
  • Copt to the /etc/ansible directory. The file looks like this:

#Install and configure filebeat
ansible-playbook filebeat/filebeat-playbook.yml

#Install and configure metricbeat
ansible-playbook metricbeat/metricbeat-playbook.yml

#Install and configure auditbeat
ansible-playbook auditbeat/auditbeat-playbook.yml

#Install and configure packetbeat
ansible-playbook packetbeat/packetbeat-playbook.yml

This will install all 4 beats on the webservers. If you do not want to install an specific beat just comment out the line containing it. Ex:

#Install and configure packetbeat
#ansible-playbook packetbeat/packetbeat-playbook.yml

These files can be copied via git and run via the commands below:

$ cd /etc/ansible
# Clone Repository + IaC Files
$ git clone
# Move Playbooks and hosts file Into `/etc/ansible`
$ mv scripts/ansible/* .
$ sh
Theme: Overlay by Kaira