Month: June 2018

Elastic beats tutorial with docker

Elastic beats tutorial: A quick look at using elastic beats with docker containers

Creating a POC of elastic beats

I gave myself a single goal today: Get a working POC of elastic beats using an nginx container and a beats container. I wanted specifically to prove that I could create a loosely coupled container logging system using beats containers.

Here’s a quick overview of the POC:

Just a quick note, the files below are very particular about spaces/indentation used. If you get strange errors just check the spaces are correct. WordPress keeps removing my spaces, I think I’ve added them in now permanently, just make sure the right spacing and indentation is in your files.

Step 1: Beats

Setup the configuration of the metricbeat.

Create a new directory for you to save your configuration file:

1
$ mkdir practice-beats

Then create a file called “metricbeat.yml” and add this:

1
2
3
4
5
6
7
8
9
10
11
12
13
metricbeat.modules:
- module: nginx
  metricsets: ["stubstatus"]
  period: 10s

  # Nginx hosts
  hosts: ["my-awesome-nginx"]

  # Path to server status. Default server-status
  #server_status_path: "server-status"

output.console:
  pretty: true

The important parts are that we are using the nginx elastic beats module. And we are going to be watching for the my-awesome-nginx host (which we will create in step 2).

The other important part to note is we are mapping the output to the console:

1
2
output.console:
pretty: true

I’m mapping to the console/terminal for simplicity but you can map the output to something more useful like logstash or elasticsearch (I’ll do follow up posts on that process).

Run the following docker command to create the beats container:

1
2
3
docker run --name my-awesome-beats \
-v $(pwd)/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml \
docker.elastic.co/beats/metricbeat:6.2.4

Beats will trigger the metricbeat request to nginx every 10 seconds.

You will see data like this appearing every 10 seconds:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
{
"@timestamp": "2018-06-08T14:51:20.367Z",
"@metadata": {
"beat": "metricbeat",
"type": "doc",
"version": "6.2.4"
},
"metricset": {
"host": "my-awesome-nginx",
"rtt": 11910,
"name": "stubstatus",
"module": "nginx"
},
"error": {
"message": "error making http request: Get http://my-awesome-nginx/server-status: lookup my-awesome-nginx on 192.168.65.1:53: no such host"
},
"beat": {
"name": "7e29608de529",
"hostname": "7e29608de529",
"version": "6.2.4"
}
}

For now you will see this error:

1
WARN transport/tcp.go:36 DNS lookup failure "my-awesome-nginx": lookup my-awesome-nginx on 192.168.65.1:53: no such host

That’s ok, we just need to setup a custom network, which we’ll do in Step 3.

What you are seeing now is a working beat 🙂 It’s alive!

Step 2: Nginx

Lets create a nginx container with some configurations as well.

Leave the elastic beats terminal running, open a new terminal window and follow the following steps:

We need to edit the default nginx configuration and add an nginx module called “stubstatus”.

Useful links:

https://www.tecmint.com/enable-nginx-status-page/

https://nginx.org/en/docs/http/ngx_http_stub_status_module.html

(Just a note, in the above examples, the location for metricbeats needs to be “location /server-status”)

To do this we add this to the default.conf file.

Go out of your current directory and then create a new directory:

1
$ mkdir practice-nginx

Then in there create 2 files:
– default.conf
– index.html

The metricbeats module will be triggering on this endpoint: “/server-status” so we need to make sure that nginx puts the right information at the point by adding this:

1
2
3
4
5
...
location /server-status {
stub_status;
}
...

Copy and paste this configuration script to your default.conf file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
server {
    listen       80;
    server_name  localhost;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    location /server-status {
        stub_status;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}

Let’s create an index.html page with only the text “Hello world!”.

So if you are using a terminal you can just do this:

1
2
$ vi index.html
hello world

And save 🙂

Then run the following docker command to spin up an nginx container:

1
2
3
4
docker run --name my-awesome-nginx \
-v $(pwd)/index.html:/usr/share/nginx/html/index.html \
-v $(pwd)/default.conf:/etc/nginx/conf.d/default.conf \
-p 8010:80 -d nginx

What are we doing here?

–name lets us give the container we are about to spin up a name
-v here we are voluming a file, we use it twice to volume two files, index.html and default.conf
index.html is going in a location where nginx is expecting to find an index.html file
Inside the container it will be looking in the location /usr/share/nginx/
default.conf is the configuration file that nginx will use by default. it is located in the path /etc/nginx/conf.d/
-p here we are telling docker to map the port 8010 on our computer to the port 80 on the nginx container

If the docker command ran correctly you should be able to go to this url and see our hello world:

1
http://localhost:8010

You should see your Hello World!

We should also be able to check that our configuration change works by going here:

1
http://localhost:8010/server-status

You should see something like:

1
2
3
4
Active connections: 1
server accepts handled requests
133 133 191
Reading: 0 Writing: 1 Waiting: 0

Step 3: Network Beats container and the Nginx container

Next we need to create a custom user network. Let’s call it “catsandboots”.

There’s a little easter egg joke in that 😉 Hint: Ask Siri to beatbox.

To create our docker network we can run this command:

1
$ docker network create catsandboots

To add our nginx and beats containers to that network we can do this:

We named the two containers like this:
– my-awesome-beats
– my-awesome-nginx

We then need to run this docker command:

1
2
$ docker network connect catsandboots my-awesome-beats
$ docker network connect catsandboots my-awesome-nginx

To check they’re on the right network you can run:

1
$ docker inspect my-awesome-beats

and

1
$ docker inspect my-awesome-nginx

Alternatively you can inspect the network and go:

1
$ docker inspect catsandboots

If you go back to the terminal with the running elastic beats container (aka my-awesome-beats) then you should now start seeing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
{
"@timestamp": "2018-06-08T15:07:20.385Z",
"@metadata": {
"beat": "metricbeat",
"type": "doc",
"version": "6.2.4"
},
"metricset": {
"host": "my-awesome-nginx",
"rtt": 4574,
"name": "stubstatus",
"module": "nginx"
},
"nginx": {
"stubstatus": {
"current": 3,
"dropped": 0,
"writing": 1,
"waiting": 0,
"accepts": 2,
"requests": 3,
"hostname": "my-awesome-nginx",
"reading": 0,
"handled": 2,
"active": 1
}
},
"beat": {
"name": "7e29608de529",
"hostname": "7e29608de529",
"version": "6.2.4"
}
}

Your beats container is now receiving actual data from nginx via the nginx substatus module. Just in case you didn’t see this useful link above here it is again: https://nginx.org/en/docs/http/ngx_http_stub_status_module.html

You should now be seeing useful information like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
"nginx": {
"stubstatus": {
"current": 3,
"dropped": 0,
"writing": 1,
"waiting": 0,
"accepts": 2,
"requests": 3,
"hostname": "my-awesome-nginx",
"reading": 0,
"handled": 2,
"active": 1
}

Done!

So that’s a quick tutorial to get an instance of elastic beats metricbeat running as a POC. There are a bunch of beats you can use, check out the documentation. I’m gonna try get a couple more tutorials out for some of the other beats. And also get a proper working elastic stack setup from container to beats to elasticsearch to kibana 🙂 Soon.