Custom Exporter with Prometheus

Prometheus is an excellent tool for collecting the metrics. In the previous article, I have explained the different data type of Prometheus. If you missed it please view this article here.

Sometimes we need to implement the custom exporter for Prometheus. For example, if I am having metrics already present in another system then I just need to export it. In such cases you might to SET the value for the counter. However, the counter is not designed for setting up the value. Gauge type will do that for you. There are numerous exporters are already available in Prometheus.

Prometheus provides custom collectors to solve these problems. You can write the exporters in supported client libraries.

I have written the custom exporter using the python client. You can deploy it using the Kubernetes as well.

In the exporter, we will have gauge and counter metrics exposed. For this, I ‘ll use the prometheus_client to import the libraries. I have the repository on Github you can check it here.

There are three steps:

1. Writing the custom exporter in python for Prometheus.
2. Dockerize the exporter
3. Deploy on Kubernetes

You can add your custom logic with the exporter.

Lets check the first step:
/code/collector.py:

import time
from prometheus_client.core import GaugeMetricFamily, REGISTRY, CounterMetricFamily
from prometheus_client import start_http_server


class CustomCollector(object):
    def __init__(self):
        pass

    def collect(self):
        g = GaugeMetricFamily("MemoryUsage", 'Help text', labels=['instance'])
        g.add_metric(["instance01.us.west.local"], 20)
        yield g

        c = CounterMetricFamily("HttpRequests", 'Help text', labels=['app'])
        c.add_metric(["example"], 2000)
        yield c


if __name__ == '__main__':
    start_http_server(8000)
    REGISTRY.register(CustomCollector())
    while True:
        time.sleep(1)

I am exposing the static gauge and counter metrics. But you can just modify it with your logic from another system.

In second step we will dockerize it:

Make sure you have pip-requirements:
/code/pip-requirements.txt

prometheus_client

Dockerfile:

FROM python:3.6

ADD code /code
RUN pip install -r /code/pip-requirements.txt

WORKDIR /code
ENV PYTHONPATH '/code/'

CMD ["python" , "/code/collector.py"]

In the last third step you can deploy the collector on Kubernetes.
/yaml/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-custom-collector-deployment
  labels:
    app: prometheus-custom-collector
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus-custom-collector
  template:
    metadata:
      labels:
        app: prometheus-custom-collector
    spec:
      containers:
      - name: prometheus-custom-collector
        image: kubejack/prometheus-custom-collector:0.2
        ports:
        - containerPort: 80

/yaml/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: prometheus-custom-collector-service
spec:
  selector:
    app: prometheus-custom-collector
  type: NodePort
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8000

Final step will be deploying the collector on Kubernetes:

$kubectl create -f yaml/

Now you can easily add your endpoint to the prometheus and start collecting the metrics.

Example:

custom-collector-prometheus

Do check my other posts on https://kuberneteslab.com and also here on medium.

Please check following useful links for references:

Reference :

Github: https://github.com/jakirpatel/prometheus-custom-collector

Github: https://github.com/prometheus/client_python

Web: https://www.robustperception.io/setting-a-prometheus-counter