Envoy statistics with StatsD

For an overview of other options for gathering statistics on Emissary-ingress, see the Statistics and Monitoring overview.

At the core of Emissary-ingress is Envoy Proxy, which has built-in support for exporting a multitude of statistics about its own operations to StatsD (or to the modified DogStatsD used by Datadog).

If enabled, then Emissary-ingress has Envoy expose this information via the StatsD protocol. To enable this, you will simply need to set the environment variable STATSD_ENABLED=true in Emissary-ingress's deployment YAML:

When this variable is set, Emissary-ingress by default sends statistics to a Kubernetes service named statsd-sink on UDP port 8125 (the usual port of the StatsD protocol). You may instead tell Emissary-ingress to send the statistics to a different StatsD server by setting the STATSD_HOST environment variable. This can be useful if you have an existing StatsD sink available in your cluster.

We have included a few example configurations in the statsd-sink/ directory to help you get started. Clone or download the repository to get local, editable copies and open a terminal window in the emissary/deployments/ folder.

Using Graphite as the StatsD sink

Graphite is a web-based real-time graphing system. Spin up an example Graphite setup:

This sets up the statsd-sink service and a deployment that contains Graphite and its related infrastructure. Graphite's web interface is available at http://statsd-sink/ from within the cluster. Use port forwarding to access the interface from your local machine:

This sets up Graphite access at http://localhost:8080/.

Using Datadog DogStatsD as the StatsD sink

If you are a user of the Datadog monitoring system, pulling in the Envoy statistics from Emissary-ingress is very easy.

Because the DogStatsD protocol is slightly different than the normal StatsD protocol, in addition to setting Emissary-ingress's STATSD_ENABLED=true environment variable, you also need to set the DOGSTATSD=true environment variable:

Then, you will need to deploy the DogStatsD agent in to your cluster to act as the StatsD sink. To do this, replace the sample API key in our sample YAML file with your own, then apply that YAML:

This sets up the statsd-sink service and a deployment of the DogStatsD agent that forwards the Emissary-ingress statistics to your Datadog account.

Additionally, Emissary-ingress supports setting the dd.internal.entity_id statitics tag using the DD_ENTITY_ID environment variable. If this value is set, statistics will be tagged with the value of the environment variable. Otherwise, this statistics tag will be omitted (the default).