bcc2telegraf

bcc2telegraf is an integration that sends ebpf-based bcc histogram metrics (https://github.com/iovisor/bcc) to telegraf daemon (https://github.com/influxdata/telegraf)

Original bcc/table.py v.0.10 just printed histograms to stdout.
This extension instead creates a memory buffer full of text lines formatted to the requirements of telegraf line protocol and sends it by http to telegraf url .

Important difference is bcc helpers create log2 histogram with low-high ends of bucket like 0-1,1-2,3-4,5-8,9-16,17-32 etc.
Tools like prometheus or grafana require those histograms be cumulative, with low always starting with 0, like 0-1,0-2,0-4,0-8,0-16,0-32 etc.
So bcc2telegraf transforms log2 histograms into cumulative ones.

Sample telegraf lines generated by this module:

ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=512 value=100 1645026564
ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=1024 value=777 1645026564
ebpf,metric=biolatency,host=abc.example.com,dc=east-1,tag1=dm-14,cloud=Nine,hi=2048 value=888 1645026564

You may create some function which extends tag1 with some other tags, based on the histogram bucket struct key described in ebpf C program
e.g. your_function returns 8.8.8.8,cloud=GCP then pass this function to send_log2_hist (... bucket_fn=your_function...)
and telegraf line will contain ... tag1=8.8.8.8,cloud=GCP...

Usage:

  • set TELEGRAF_URL environment variable to your telegraf service
  • copy a python-based bcc program, e.g. biolatency or tcprtt
  • change it by adding an
    import bcc2telegraf
    and replace instances of
    dist.print_log2_hist(label...)
    with
    dist.send_log2_hist(label...)
    or with
    dist.send_log2_hist(label,bucket_fn=your_function...)
  • think about creating a systemctl-managed service out of that program

GitHub

View Github