ELK Stack

The ELK Stack (Elasticsearch, Logstash, Kibana) provides centralised log ingestion, storage, and visualisation. This guide installs version 7.x on Ubuntu.

Step 1 – Install Java and prerequisites

sudo apt-get update
sudo apt-get install -y openjdk-11-jdk apt-transport-https curl nginx

Step 2 – Add the Elastic repository

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" |   sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update

Step 3 – Install Elasticsearch

sudo apt-get install -y elasticsearch
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch
sudo systemctl start elasticsearch

Verify: curl -s http://localhost:9200 | python3 -m json.tool

Step 4 – Install Kibana

sudo apt-get install -y kibana
sudo systemctl enable kibana
sudo systemctl start kibana

Kibana listens on port 5601 by default (localhost only).

Step 5 – Install Logstash

sudo apt-get install -y logstash
sudo systemctl enable logstash
sudo systemctl start logstash

Step 6 – Proxy Kibana behind Nginx with basic auth

sudo apt-get install -y apache2-utils
sudo htpasswd -c /etc/nginx/kibana.htpasswd admin

Create /etc/nginx/sites-available/kibana:

server {
    listen 80;
    server_name kibana.yourdomain.com;
    auth_basic "Kibana";
    auth_basic_user_file /etc/nginx/kibana.htpasswd;
    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
    }
}

sudo ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/
sudo systemctl restart nginx