Microsoft is excited to embrace OpenTelemetry as the future of telemetry instrumentation. You, our customers, have asked for vendor-neutral instrumentation, and we're delighted to partner with the OpenTelemetry community to create consistent APIs/SDKs across languages.
Microsoft played a key role in brokering an agreement between OpenCensus and OpenTracing, two previously popular open-source telemetry projects to create OpenTelemetry. OpenTelemetry includes contributions from all major cloud and Application Performance Management (APM) vendors and is housed by the Cloud Native Computing Foundation (CNCF) of which Microsoft is a Platinum Member. Microsoft is among the largest contributors to OpenTelemetry.
This blog focuses on OpenTelemetry as it relates to Azure Monitor Application Insights. In the future, we hope to provide more context on how OpenTelemetry is impacting other parts of Azure!
Concepts and Progress
Telemetry, the data collected to observe your application, can be broken into three types or "pillars":
- Distributed Traces
- Metrics
- Logs
Initially the OpenTelemetry community took on Distributed Traces. Metrics and Logs are still in progress. A complete observability story includes all three pillars. Azure Monitor is working on an OpenTelemetry-based supported offering that includes Distributed Traces only with plans to add the other pillars as they mature in the OpenTelemetry community.
In the following sections, we'll cover some telemetry collection basics.
Instrumenting your application
At a basic level, "instrumenting” is simply enabling an application to capture telemetry.
There are two methods to instrument your application:
- Manual Instrumentation
- Automatic Instrumentation (Auto-Instrumentation)
Manual instrumentation is coding against the OpenTelemetry API. In the context of an end-user, it typically refers to installing a language-specific SDK in an application. Azure Monitor’s OpenTelemetry-based manual instrumentation offerings on our roadmap consist of Azure Monitor Exporters in C#, JavaScript (Node.js), and Python.
“Manual” does not mean you’ll be required to write complex code to define spans for distributed traces (though it remains an option). A rich and growing set of instrumentation libraries maintained by OpenTelemetry contributors will enable you to effortlessly capture telemetry signals across common frameworks and libraries. A subset of OpenTelemetry Instrumentation Libraries will be supported by Azure Monitor, informed by customer feedback. Additionally, we are working to instrument the most popular Azure Service SDKs to emit OpenTelemetry Protocol (OTLP) and optimizing them for Azure Monitor.
Auto-instrumentation is enabling telemetry collection through configuration without touching the application's code. While more convenient, it tends to be less configurable and it’s not available in all languages. Azure Monitor’s OpenTelemetry-based auto-instrumentation offering consists of the Java 3.X GA offering, and we continue to invest in it informed by customer feedback. The OpenTelemetry community is also experimenting with C# and Python auto-instrumentation, but Azure Monitor is focused on creating a simple and effective manual instrumentation story in the near-term.
*We are evaluating the suitability of OpenTelemetry JavaScript for browser scenarios. We continue to recommend our Application Insights Client-side JavaScript SDK for the near future.
Sending your telemetry
There are two ways to send your data to Azure Monitor (or any vendor):
- Direct Exporter
- Via an Agent
A direct exporter sends telemetry in-process (from the application’s code) directly to Azure Monitor’s ingestion endpoint. The main advantage of this approach is simplicity.
Azure Monitor’s immediate focus is on direct exporters for our supported languages. This is due to our roots in the developer productivity space and due to the complexities integrating OTLP into our existing Azure Monitor Agent.
We plan to support an agent-based approach in the future, though the details and timeline aren't available yet. Our objective is to provide a path for any OpenTelemetry supported langauge to send to Azure Monitor via OTLP. This will enable customers to observe applications written in languages beyond our supported languages.
*Some customers have begun to use the OpenTelemetry-Collector as an agent alternative even though Microsoft doesn’t officially support “Via an Agent” approach for application monitoring yet. In the meantime, the open source community has contributed an OpenTelemetry-Collector Azure Monitor Exporter that some customers are using to send data to Azure Monitor Application Insights.
How does this affect me?
If you are an existing Azure Monitor Application Insights customer, we don’t recommend that you migrate to OpenTelemetry yet (except for Java applications where we recommend our OpenTelemetry-based Java 3.X GA offering for everyone which GA’d in November 2020).
Here’s why. The existing Application Insights SDKs are feature-rich, and it’ll be quite a while until OpenTelemetry matures to feature parity. As the OpenTelemetry-based offering gets closer to feature parity, we’ll provide clear guidance to existing customers using Application Insights SDKs.
If you are developing a new application and want to onboard to Azure Monitor Application Insights with the most feature-rich experience, we recommend using the existing Application Insights SDKs for C# (ASP.NET or ASP.NET Core) and Javascript (Node.js) or Python-OpenCensus. The features only in existing Application Insights SDKs as of the end of 2021, include:
- Ability to create custom metrics and events
- Ability to pass metrics unaffected by sampling
- Enhanced data transport security using AAD Authentication
- Enhanced data transport reliability using offline disk storage with retry logic
- Additional diagnostic tools (.NET only): Profiler and Snapshot Debugger
If you are developing a new application and are willing to sacrifice the features noted above to use OpenTelemetry, stay tuned for our Release Announcement expected before year’s end. You can see our progress for C#, JavaScript (Node.js), and Python in open-source repos.
Scenarios that may sway you toward OpenTelemetry sooner-than-later include sending telemetry to Azure Monitor + another vendor simultaneously, collecting and converging existing instrumentation protocols, or leveraging features available in the OpenTelemetry-Collector. For example, some developers may want to use a local tool such as Jaeger or Zipkin to test on a dev box and configure their instrumentation to use Azure Monitor when running in a production environment.
Looking Forward
We are committed to OpenTelemetry as the future of telemetry instrumentation, and over time we expect Azure services and platform to adopt it more broadly. This journey will take time, and we are committed to bringing you, our customers, along on the journey with us.
Learn More
- The Azure Monitor OpenTelemetry FAQ provides additional context and links out to more supporting content.
- The CNCF provides lots of analytics on OpenTelemetry’s project metrics including Contributions by Organization.
- Sources that explain the three pillars of observability in detail include the OpenTelemetry community website and Distributed Systems Observability by Cindy Sridharan.
Posted at https://sl.advdat.com/3k47sum
