In the previous post I gave a brief introduction to the DiagnosticSource infrastructure and how you can use it to log events emitted by the .NET framework libraries. If you're new to DiagnosticSource I suggest reading that post first. In this post I look at how to consume objects that are passed as anonymous types.
Named types - when diagnostic source is easy
In the previous post I showed that you can access the objects passed in a DiagnosticSource event by casting the provided object to the right type. For example, for the "HostBuilding" event emitted by the DiagnosticListener called "Microsoft.Hosting" you can cast the event object to HostBuilder:
public void OnNext(KeyValuePair<string, object?> value)
{
if(value.Key == "HostBuilding")
{
var hostBuilder = (HostBuilder)value.Value;
hostBuilder.UseEnvironment("Production");
}
}
This works because the framework library is passing a named type, HostBuilder in this case, as part of the event. But the event type might not be a named type it could be an anonymous type!
Anonymous types - painful to consume
As a library author, when you are emitting an event to a DiagnosticListener, you can pass any object as the "data" part of the event. In the case of the "HostBuilding" event you're already seen, that's simple, they pass in the HostBuilder instance.
But the API accepts an object, so you can pass in an anonymous type. In fact, in .NET 6, that's exactly what the framework does in various places. For example, the "Microsoft.AspNetCore.Hosting.BeginRequest" event is created like this:
// These values are set elsewhere...
DiagnosticListener _diagnosticListener = // ...
HttpContext httpContext = // ...
long startTimestamp = // ...
_diagnosticListener.Write(
"Microsoft.AspNetCore.Hosting.BeginRequest",
new
{
httpContext = httpContext,
timestamp = startTimestamp
});
In .NET 7 the anonymous type in this event will be replaced with a named type instead.
To prove that we really are getting an anonymous type in .NET 6, we can whip up a TestKeyValueObserver as in the previous post:
using System.Diagnostics;
public class TestKeyValueObserver : IObserver<KeyValuePair<string, object?>>
{
public void OnNext(KeyValuePair<string, object?> value)
{
if(value.Key == "Microsoft.AspNetCore.Hosting.BeginRequest")
{
Console.WriteLine($"Event type: {value.Value?.GetType()}");
}
}
public void OnCompleted() {}
public void OnError(Exception error) {}
}
and subscribe to the DiagnosticListener called "Microsoft.AspNetCore". When we run our application and start an HTTP request, we'll see the following in the Console:
Event type: <>f__AnonymousType0`2[Microsoft.AspNetCore.Http.HttpContext,System.Int64]
Ok, fine, we have an anonymous type. Is that a big deal?
Well, kind of.
The problem is that there's no easy way to get the property values from an anonymous type without using reflection.
Yes, you read that right. If you want to (for example) grab the long startTimestamp value from the Observer, you need to do something like this:
using System.Diagnostics;
public class TestKeyValueObserver : IObserver<KeyValuePair<string, object?>>
{
public void OnNext(KeyValuePair<string, object?> value)
{
if(value.Key == "Microsoft.AspNetCore.Hosting.BeginRequest")
{
// Get a reference to the
Type type = value.Value.GetType();
PropertyInfo property = type.GetProperty("timestamp");
var timestamp = (long)property.GetValue(value.Value);
Console.WriteLine($"Request started at {timestamp}");
Console.WriteLine($"Event type: {value.Value?.GetType()}");
}
}
public void OnCompleted() {}
public void OnError(Exception error) {}
}
Remember, this is all happing synchronously, so you're doing that reflection in the hot path of a request (in this case). That's really not ideal. Plus the consumer experience here sucks. We now not only have the boilerplate of creating and subscribing both an IObserver<DiagnosticListener> and an IObserver<KeyValuePair<string, object?>>, we also have to do all this reflection faffing.
On the face of it, this seems like an obviously bad idea compared to using a public named Type that you can cast to, so why did Microsoft choose to do it like that?
Anonymous types - good for backwards compatibility
Using anonymous types wasn't an oversight. There's a document in the .NET runtime repository called "DiagnosticSource User's Guide", which has this advice under the heading "Best Practices: Payloads":
DO use the anonymous type syntax 'new { property1 = value1 ...}' as the default way to pass a payload even if there is only one data element. This makes adding more data later easy and compatible.
So the user guide explicitly suggests using the anonymous types, even though they're a pain to consume. The reasoning is right there at the end of the section: This makes adding more data later easy and compatible.
This makes total sense. The reflection code I wrote in the previous section works with any anonymous or named type that contains a long property called timestamp. If the framework later emits additional values (example shown below), the IObserver<> code will still work:
_diagnosticListener.Write(
"Microsoft.AspNetCore.Hosting.BeginRequest",
new
{
httpContext = httpContext,
timestamp = startTimestamp,
userId = someId, // adding some extra values
somethingElse = string.Empty,
});
So anonymous types are great for backwards compatibility, and bad for developer experience. So why did the framework authors choose to use named types in some cases, and anonymous types in others. Well, advice around this is right there in the user's guide too (emphasis mine):
CONSIDER creating an explicit type for the payload. The main value for doing this is that the receiver can cast the received object to that type and immediately fetch fields (with anonymous types reflection must be used to fetch fields). This is both easier to program and more efficient. Thus in scenarios where there is likely high-volume filtering to be done by the logging listener, having this type available to do the cast is valuable.
In general, there appears to be a trend towards using named types in all cases now, though that's just anecdotal. Most of the newer DiagnosticSource events that I've seen use named types instead of anonymous types.
Nevertheless, for the anonymous types, are we really stuck manually doing reflection to access the objects? Thankfully not…
Handling anonymous types with DiagnosticAdapter
As I already mentioned, DiagnonsticListeners run synchronously as part of your infrastructure code, so you typically want them to be fast. Any time you spend in your listener is actively blocking your request, and slowing your application down.
Asking people to use reflection to access the event properties is clearly at-odds with the requirement for speed. You could write your own methods to reflect over the provided anonymous object and extract the original property values like timestamp and httpContext (like I showed in the previous example). But luckily Microsoft provided a helper package to do this for you, Microsoft.Extensions.DiagnosticAdapter.
This package does a lot of work (I'm talking IL emitting, proxy methods, dynamic assemblies, the full shebang) to significantly simplify the API that's required and to be as fast as possible. This makes writing a DiagnosticListener more declarative, and removes a big chunk of boilerplate. But the really important thing is that it dramatically simplifies working with anonymous type event data.
As an example of how to use the package, I'll walk through subscribing to the "Microsoft.AspNetCore.Hosting.BeginRequest" event from the DiagnosticListener called "Microsoft.AspNetCore":
1. Add the Microsoft.Extensions.DiagnosticAdapter package
You can add the NuGet package to your project using:
dotnet add package Microsoft.Extensions.DiagnosticAdapter
2. Create an adapter implementation
Next, create an adapter implementation for your event using the [DiagnosticName] attribute to control which event to listen to. The adapter below only subscribes to a single event, but you can add more methods to subscribe to more events if you wish.
using Microsoft.Extensions.DiagnosticAdapter;
public class ValueListenerAdapter
{
// 👇 The [DiagnosticName] attribute describes which events to listen to
[DiagnosticName("Microsoft.AspNetCore.Hosting.BeginRequest")]
public virtual void OnBeginRequest(HttpContext httpContext, long timestamp)
{
// 👆 the signature of the method is used to extract information from
// the anonymous type. You still need to know the names of the properties
// and their types, but no reflection in user-code is required
Console.WriteLine($"Request started at {timestamp} to {httpContext.Request.Path}");
}
}
The method name is arbitrary, but the method parameters should match the properties of the event data. You still need to know the names of the properties and their types, but this is a much nicer API to work with than having to do "manual" reflection. This adapter replaces the IObserver<KeyValuePair<string, object?>> implementation I showed previously.
3. Create an IObserver to subscribe
You need to register your adapter to the DiagnosticListener called "Microsoft.AspNetCore". Create an IObserver<DiagnosticListener> implementation, using the SubscribeWithAdapter() extension method to register your adapter to the appropriate DiagnosticListener:
using System.Diagnostics;
public class TestDiagnosticObserver : IObserver<DiagnosticListener>
{
public void OnNext(DiagnosticListener value)
{
if (value.Name == "Microsoft.AspNetCore")
{
value.SubscribeWithAdapter(new ValueListenerAdapter());
}
}
public void OnCompleted() {}
public void OnError(Exception error) {}
}
4. Register the observer
Finally register your IObserver<DiagnosticListener> in Program.cs:
using System.Diagnostics;
// 👇 Register your IObserver<DiagnosticListener> to the global registry
DiagnosticListener.AllListeners.Subscribe(new TestDiagnosticObserver());
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.Run();
Now when you run your app and make a request, your adapter will write to the console:
Request started at 13861397642614 to /
Without the Microsoft.Extensions.DiagnosticAdapter package, reading the anonymous events requires a lot of work. So you may understand my frustration when I discovered Microsoft stopped shipping this package after .NET Core 3.1!
Why isn't Microsoft.Extensions.DiagnosticAdapter available any more?
The decision to abandon Microsoft.Extensions.DiagnosticAdapter was part of the "great consolidation" event in the Microsoft GitHub repositories. This was when many packages in the dotnet/extensions repository were moved into the dotnet/runtime or dotnet/aspnetcore repositories.
But there was a single casualty. Nestled at the bottom of that announcement post:
There was no indication of why it wasn't shipping, or what the alternative should be. Just "no longer shipping".
If you were following along in the previous section and ran the
dotnet add package ...command, you may have noticed that it installed version3.1.25, even though it's a .NET 6 app!
Luckily, the Microsoft.Extensions.DiagnosticAdapter package has no other "framework" dependencies other than System.Diagnostics.DiagnosticSource, so as long as there are no incompatible changes in newer versions of that library, we are hopefully ok to continue using it where necessary.
If I had to guess why it was abandoned, I suspect it was a combination of
- Moving away from anonymous types for
DiagnosticSourceevents - Lack of support on some platforms (e.g. IL emit isn't always available)
- It's "done". There isn't really much more development required on it
The NuGet package directly supports .NET Core 2.0, .NET Framework 4.6.1, and .NET Standard 2.0, so you should be able to use it pretty much everywhere you need to.
At least one benefit of Microsoft abandoning Microsoft.Extensions.DiagnosticAdapter is that they can't sabotage it like they have other packages, so it should always be useable in the future!🤞
Summary
In this post I described the problem of consuming DiagnosticListener events that use anonymous types. To access the data in these types you're forced to use reflection, which can be slow and clumsy to work with. As an alternative, I showed how to use the Microsoft.Extensions.DiagnosticAdapter NuGet package, which provides a declarative API for working with the anonymous type data. Unfortunately, this package is no longer shipping with .NET, but you can still use the .NET Core 3.1.x version, as that will work with later versions of .NET without any issues.
