Selecting an OpenTelemetry backend – Issues to bear in mind

OpenTelemetry is a Cloud Native Computing Basis(CNCF) incubating mission geared toward standardizing the best way we instrument functions for producing telemetry information(logs, metrics, and traces). Nonetheless, OpenTelemetry doesn’t present storage and visualization for the collected telemetry information. And that’s the place an OpenTelemetry backend is required.

Cloud computing and containerization made deploying and scaling functions simpler. Fashionable functions make use of modular code and architectures like microservices and serverless. Engineering groups can ship options quicker, and any surge in person demand might be met by simply spinning up extra containers.

However each coin has two sides. Whereas having advantages like smaller engineering groups and on-demand scaling of functions, cloud computing and containerization have additionally elevated operational complexity manifolds. Troubleshooting an utility based mostly on a distributed system for efficiency points is like discovering a needle in a haystack.

Gathering information from functions that may act as alerts for troubleshooting efficiency points is a follow as outdated as writing software program. The information that helps to investigate efficiency points is called telemetry information. For cloud-native functions, it grew to become a problem to have a constant framework for producing telemetry information.

OpenTelemetry solves this drawback by creating an open customary for producing telemetry information. Let’s be taught a bit about OpenTelemetry.

What’s OpenTelemetry?

OpenTelemetry is an open-source assortment of instruments, APIs, and SDKs that goals to standardize the best way we generate and acquire telemetry information. It follows a specification-driven growth. The OpenTelemetry specification has design and implementation pointers for the way the instrumentation libraries must be applied. As well as, it supplies consumer libraries in all the most important programming languages which observe the specification.

OpenTelemetry was fashioned after the merger of two open-source initiatives – OpenCensus and OpenTracing in 2019. Since then, it has been the go-to open supply customary for instrumenting cloud-native functions.

The specification is designed into distinct forms of telemetry referred to as alerts. Presently, OpenTelemetry has specs for these three alerts:

• Logs,
• Metrics, and
• Traces

Collectively these three alerts kind the three pillars of observability. OpenTelemetry is the bedrock for establishing an observability framework. The appliance code is instrumented utilizing OpenTelemetry consumer libraries, which permits the era of telemetry information. As soon as the telemetry information is generated and picked up, OpenTelemetry wants a backend evaluation instrument to which it may ship the information.

Why does OpenTelemetry want a backend?

The founders of OpenTelemetry wished to standardize two issues:

• The way in which we instrument utility code
• The information format of generated telemetry information

As soon as we’ve the telemetry information in a constant format, it may be despatched to any observability backend. This supplies customers the liberty to pick any OpenTelemetry backend based mostly on their observability wants.

The goal of OpenTelemetry was all the time to have a pluggable structure utilizing which customers can construct strong observability stacks utilizing extra expertise protocols and codecs.

Right here’s a snapshot exhibiting how OpenTelemetry matches inside a microservice-based utility and an observability backend.

The OpenTelemetry collector acts because the routing layer. As soon as the telemetry information is collected with the assistance of OpenTelemetry libraries, it’s despatched to the OpenTelemetry collector. The OpenTelemetry collector can export the telemetry information in a number of codecs to a number of observability backends.

Issues to look out for when selecting an OpenTelemetry backend

OpenTelemetry supplies you the liberty to decide on a backend evaluation instrument of your alternative. Most observability distributors presently have launched help for OpenTelemetry. So how do you guarantee which OpenTelemetry backend to go for? There are three principal parts that an OpenTelemetry backend is liable for:

• Knowledge Storage
  Observability information might be big. The information storage of an OpenTelemetry backend must be extremely scalable, and question execution efficiency must be top-notch.

• Question Service
  OpenTelemetry has three distinct alerts. Every sign has completely different information codecs with completely different use-cases for customers. Furthermore, the alerts are correlated. The question service of an OpenTelemetry backend must be constructed with these issues in thoughts.

• Visualization
  Making sense out of information is what impacts the end-user of observability dashboards. OpenTelemetry backends ought to present intuitive dashboards that allow end-users to take fast actions on efficiency points.

Under is the listing of things that must be considered earlier than choosing an OpenTelemetry backend:

• Help for all distinct alerts of OpenTelemetry
  At present, OpenTelemetry collects telemetry information in three distinct alerts, specifically, logs, metrics, and traces. Organising a sturdy observability framework requires using all three alerts.
  An OpenTelemetry backend ought to have the ability to ingest and visualize all three alerts. Furthermore, the frontend of the OpenTelemetry backend also needs to present options to simply correlate the alerts.

• Native help for OpenTelemetry semantic conventions
  In OpenTelemetry, each element of a distributed system is outlined as an attribute. The attribute is nothing however a key-value pair. These attributes are outlined by the OpenTelemetry specification as OpenTelemetry semantic conventions. For instance, here’s a glimpse of how HTTP conventions seem like:

An OpenTelemetry backend ought to have native help to retailer information with OpenTelemetry semantic conventions. Current observability distributors normally rework the information collected utilizing OpenTelemetry semantic conventions into their propriety codecs. However OpenTelemetry has a big listing of semantic conventions which could not be absolutely utilized in such situations.

• Ought to permit aggregates on hint information
  Working aggregates on hint information lets you create service-centric views. OpenTelemetry additionally supplies you the flexibility to create customized tags. Mixed with customized tags and aggregated hint information provides you a strong magnifying glass to floor efficiency points in your companies. For instance, you will get the error charge and 99th percentile latency of customer_type: gold or deployment_version: v2 or external_call: paypal

• Open Supply
  OpenTelemetry is an open supply customary with an enormous group backing. It’s testimonial to the truth that community-driven initiatives can clear up massive advanced engineering issues. It’s not essential for the OpenTelemetry backend to be open supply.

However having an open supply OpenTelemetry backend can allow you to have a full-stack open supply answer. Open supply options have extra flexibility, and for those who self-host, you don’t want to fret about issues like information privateness.

Practically all observability distributors now declare to be 100% appropriate with OpenTelemetry. But it surely’s tough to maneuver away from legacy techniques. An answer constructed natively for OpenTelemetry is usually a good selection for an OpenTelemetry backend. And that’s the place SigNoz comes into the image.

Attempting out an OpenTelemetry Backend

If you wish to mess around with an OpenTelemetry backend, it’s straightforward to get began with SigNoz. SigNoz is an open supply APM constructed natively on OpenTelemetry.

SigNoz might be put in on macOS or Linux computer systems in simply three steps through the use of a easy set up script.

The set up script routinely installs Docker Engine on Linux. Nonetheless, you have to manually set up Docker Engine on macOS earlier than operating the set up script.

git clone -b principal https://github.com/SigNoz/signoz.git
cd signoz/deploy/
./set up.sh

You possibly can go to our documentation for directions on tips on how to set up SigNoz utilizing Docker Swarm and Helm Charts.

As soon as your utility is instrumented with OpenTelemetry consumer libraries, the information might be despatched to the SigNoz backend by specifying a selected port on the machine the place SigNoz is put in.

You possibly can then use Signoz to watch utility metrics with out-of-box charts and visualization.

The tracing sign from OpenTelemetry instrumentation helps you correlate occasions throughout companies. With SigNoz, you possibly can visualize your tracing information utilizing Flamegraphs and Gantt charts. It exhibits you a whole breakdown of the request together with each bit of information collected with OpenTelemetry semantic conventions.

SigNoz additionally enables you to run aggregates in your tracing information. Working aggregates on tracing information lets you create service-centric views, offering insights to debug functions on the service stage. It additionally is smart for engineering groups as they personal particular microservices.

You possibly can try the SigNoz GitHub repo right here:

Additional Studying

SigNoz – an open supply different to DataDog

OpenTelemetry Collector – a whole information