---

Datadog

---

II - Datadog Account and Agent creation

a - Creating a Datadog account

Now that we know what Datadog is, we can create an account so that we can discover its many features. We can go to the creation page at https://us5.datadoghq.com/signup.

We can fill in our account creation information and click on the sign up button.

Once done, we can choose the services we can monitor later on our account. We can add more later, so don't panic. So we'll choose the following basic services: Docker, Kubernetes, Nginx, Mysql, Postgresql, Python, Github.

We can then fill in the rest of the form. We specify that we have a server to monitor by slightly dragging the bar below the label how many servers does your organisation manage?. We also specify that we're not an infrastructure or service host by choosing no on the are you a Managed Service Provider? / Hosting Provider? field. Finally we specify the first reason for using Datadog on the What's your first goad with Datadog field. For the purposes of this course, we'll simply fill in Monitoring servers and services, and we can click the next button.

b - Datadog agent

Monitoring is crucial for microservices and multi-level applications in a data center or cloud infrastructure. One of the most widely used log monitoring tools is therefore Datadog, which retrieves logs, events and metrics from hosts via an agent installed on the host we want to monitor.

The Datadog Agent is a lightweight, open source software installed on application hosts that pushes every log, event, trace and metric produced by our applications and infrastructure using the Datadog API. This Datadog agent also aggregates metrics from other processes or containers on the host.

The Datadog agent also emits Application Performance Monitoring data, which it sends to Datadog.

On the system or systems we wish to monitor, we will therefore need to install the Datadog agent and configure it to send data to Datadog. It can parse, filter and enrich this data, then send it to any supported destination, including a Datadog destination. By default, Datadog Agent sends data only to Datadog.

c - Datadog and Containers

c.1 - Overview (Reminder)

Containers solve many operational problems, but they also have their problems. They come and go very quickly, therefore Docker monitoring is extremely crucial. We can't simply create containers without monitoring their performance and results. It's like driving on a crowded road without following the rules. Docker monitoring is essential to make sure we find and fix problems before they turn into nightmares. In addition, infrastructure monitoring is essential to know the impact of processes on CPU, memory and other hardware usage.

.

Although Docker containers have eased the process of developing and running applications, Docker users and organizations need a platform from which they can perform monitoring of their Docker-based applications and their distribution. These requirements are exceeded by Datadog, the leading cloud-based monitoring platform for cloud applications.

c.2 - Datadog and Docker

Datadog is the world's leading provider of SaaS-based monitoring services for cloud applications. It has been used by many large IT organizations to streamline Docker monitoring. Now, Docker and Datadog are working together to improve the monitoring of cloud-based application development. Datadog can be integrated with Docker using the Docker API to enable engineering teams to monitor their Docker applications with minimal configuration.

In addition, Datadog has an excellent Docker integration plugin that provides an enhanced monitoring dashboard for container-based monitoring.

We'll now move on to the next part of our installation and install our Datadog agent through Docker.

We therefore select Docker in the rest of our account configuration.

Once we click on Docker, Datadog provides us with the commands to execute in order to install our Datadog agent and this thanks to Docker.

We'll now execute the first instruction provided for installing the Datadog agent:

docker run -d --name dd-agent -v /var/run/docker.sock:/var/run/docker. sock:ro -v /proc/:/host/proc/:ro -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro -e DD_API_KEY=0720dfddfd929a528d558ecbcee9eac3 -e DD_SITE="us5.datadoghq.com" gcr.io/datadoghq/agent:7

Output display:

Unable to find image 'gcr.io/datadoghq/agent:7' locally
7: Pulling from datadoghq/agent
1be8fe28248f: Pull complete
Digest: sha256:e00e839b1195c3cf1da389b7d9bacd419b62ae4712b5dd287ca7ad410c0057a7
Status: Downloaded newer image for gcr.io/datadoghq/agent:7
67e22b0505bdf61e414be7e2b292e0e8260bb1d7c9bf67c31d4edce352ec9ef9

Some explanations

• DD_API_KEY: This is the variable defining the api key for talking to our Datadog instance. We pass it the value 0720dfddfd929a528d558ecbcee9eac3 relative to the account used in the course. API keys are unique to our organization. An API key is required by the Datadog Agent to send metrics and events to Datadog. We'll talk about this in detail later in the course.

• DD_SITE: This is the Datadog site we're using for our configuration, and we're passing it the value us5.datadoghq.com to this Docker environment variable thanks to the -e flag. Datadog offers different sites available worldwide. The sites are all independent. It is not possible to share data between one site and another. We'll talk more about this later in the course

.

A few minutes later we can reload our pages to check that the agent is communicating with our account.

As we can see, we've got our first agent up and running. Now all we have to do is drag it to the bottom of the page and click on the finish button.

We are now on the home page of our Datadog account.

We can check the list of hosts on our infrastructure by clicking on the infrastructure menu icon, then on the infrastructure list sub-menu.

We find our host with the active state as well as some additional information we'll talk about later.

As we can see, we only have one machine present on within an array that represents the machines in our Datadog infrastructure. We have 06 columns present in this table:

• Hostname: which is the hostname of the machine to be monitored.
• Status: which is the status of the Datadog agent on the machine.
• CPU: which is the processor consumption of the machine to be monitored.
• IOWAIT: which is the percentage of time the processor (or processors) was idle and during which the system had pending disk I/O requests.
• Load: which is the system load over the last 15 minutes.
• APPS: which represents the list of services or tools we can monitor using our Datadog agent.

We can also see that we have 04 applications that we can monitor on our current instance:

• Container: the list of containers present on our Docker instance.
• Docker: our entire Docker instance and associated items.
• Ntp: for Network Time Protocol, this is an Internet protocol used to synchronize with computer clock sources in a network.
• System: the list of system elements on our server that we can monitor.

We can click on the agent's hostname field to view some metrics for our applications.

We arrive on our agent's details page. We can then click on the Metrics tab to view our host's metrics.

We can retrieve metrics such as RAM consumption, CPU network traffic etc.

.

We can also click on the containers tab to check the list of containers.

As we can see, we only have one container running at the moment on our host. We can also click on the agent name to get a container monitoring dashboard. We'll talk about dashboards later in the course.

We're going to run some new containers so we can monitor them too. We'll run a docker-compose stack within our Datascientest machine.

We'll start by creating make a clone of our Wordpress project:

git clone https://github.com/DataScientest/wordpress-docker-compose-stack.git

We will then execute this microservices stack with the following command:

docker-compose up -d

Output display:

Creating network "wordpress-docker-compose_default" with the default driver
Creating volume "wordpress-docker-compose_db_data" with default driver
Pulling db (mysql:latest)...
latest: Pulling from library/mysql
e2c03c89dcad: Pull complete
68eb43837bf8: Pull complete
Digest: sha256:232936eb036d444045da2b87a90d48241c60b68b376caf509051cb6cffea6fdc
Status: Downloaded newer image for mysql:latest
Pulling pma (phpmyadmin/phpmyadmin:)...
latest: Pulling from phpmyadmin/phpmyadmin
f1f26f570256: Pull complete
ee0a4e40ccac: Pull complete
Digest: sha256:ed87921184b59f7d8fc85c6a5f041c22758a4d4419c0ee3bac38eb7e133eaed3
Status: Downloaded newer image for phpmyadmin/phpmyadmin:latest
Pulling wp (wordpress:latest)...
latest: Pulling from library/wordpress
9d21b12d5fab: Pulling fs layer
1d2f438b3058: Pulling fs layer
f04ed8a9ea49: Pulling fs layer
10478cdfe5da: Pulling complete
Digest: sha256:9bdcaf49c07ab8433b1de25761524948e57e89c039915d0372e9ab86bc64354c
Status: Downloaded newer image for wordpress:latest
Pulling wpcli (wordpress:cli)...
cli: Pulling from library/wordpress
c1d6d1b2d5a3: Pull complete
3f18dab2210b: Pulling complete
c509fa41f328: Pull complete
Digest: sha256:c4301e71542f038362312eb25350eab26f49d95b6e3266377e6787d9feb57b11
Status: Downloaded newer image for wordpress:cli
Creating wordpress-docker-compose_db_1 ... done
Creating wordpress-docker-compose_pma_1 ... done
Creating wordpress-docker-compose_wp_1 ... done
Creating wordpress-docker-compose_wpcli_1 ... done

We can wait a few minutes. Let's reload our page and check our dashboard again within Datadog.

We see that we now have 04 containers detected. We can click on the metrics tab again to check the new metrics.

We can now click on the dashboard menu and the dashboard lists sub-menu.

Let's click on the dashboard Docker - Overview.

and we can view a set of graphs that make up the metrics returned by our Datadog agent.

Below, we can see the consumption of the processor load, memory (RAM) as well as the machine's network.

If we go even further down, we can see resources consumed by the various containers.

We can also enlarge the graphics on this dashboard. For example, we'll choose the CPU system by image graph located on the previous capture and click on an icon that will allow us to enlarge this graph and see the different graphs. We can place our cursor on the graph

We can therefore see the CPU consumption of our image gcr.io/datadoghq/agent:7 which is used for the agent Datadog.

We can click on the image mysql to visualize CPU consumption.

Once our configurations are complete, we can now stop execution of our containers.

docker-compose down

Output display:

Stopping wordpress-docker-compose_wp_1 ... done
Stopping wordpress-docker-compose_pma_1 ... done
Stopping wordpress-docker-compose_db_1 ... done
Removing wordpress-docker-compose_wpcli_1 ... done
Removing wordpress-docker-compose_wp_1 ... done
Removing wordpress-docker-compose_pma_1 ... done
Removing wordpress-docker-compose_db_1 ... done