Observability

Observability is on by default in fullstackhero. Every HTTP request is traced; every Mediator command is traced; every EF Core query is traced; every Hangfire job is traced. Structured logs flow through Serilog with HTTP-context enrichers, and metrics + logs cross the OTLP exporter to whatever collector you point it at (Aspire dashboard locally, Honeycomb / Datadog / Grafana Cloud / Tempo / Loki / etc. in prod).

What gets observed without you doing anything

All under one ActivitySource graph, exported to OTLP.

Logging — Serilog with enrichers

AddHeroLogging configures Serilog with these enrichers on every log entry:

• RequestId — ASP.NET Core’s per-request id
• TraceId — the OTel trace id (links logs to traces)
• SpanId — the current span id
• UserId — current authenticated user (when available)
• TenantId — resolved tenant for the current request
• CorrelationId — for cross-request workflows

_logger.LogInformation("Created product {Sku} for tenant {TenantId}", product.Sku, tenantId);

Outputs to console + file by default. Add OTLP sink for Loki / Grafana Cloud / Honeycomb log ingestion:

{
  "Serilog": {
    "WriteTo": [
      { "Name": "Console" },
      { "Name": "File", "Args": { "path": "logs/.log", "rollingInterval": "Day" } },
      { "Name": "OpenTelemetry", "Args": { "endpoint": "https://api.honeycomb.io/v1/logs" } }
    ]
  }
}

Tracing — what shows up in traces

A typical request trace looks like this:

HTTP POST /api/v1/catalog/products                      [12ms]
├─ Mediator.Send<CreateProductCommand>                  [11ms]
│  ├─ ValidationBehavior                                [<1ms]
│  ├─ CreateProductCommandHandler                       [9ms]
│  │  ├─ EF: SELECT * FROM Brands WHERE Id = @brandId   [2ms]
│  │  ├─ EF: SELECT * FROM Categories WHERE Id = @catId [1ms]
│  │  └─ EF: INSERT INTO Products...                    [4ms]
│  └─ MediatorTracingBehavior (parent span)
└─ HTTP response 201 Created

Every step is a span with start time, duration, and structured attributes. Trace context propagates to outbound HTTP calls (Webhooks deliveries, third-party APIs) so a single trace id follows the request across processes.

Metrics — what’s measured

• Runtime metrics — GC counts, threadpool size, exception count, allocation rate (OpenTelemetry.Instrumentation.Runtime).
• HTTP server metrics — request duration (histogram), in-flight requests (counter), responses by status code.
• Cache metrics — cache.hits, cache.misses, cache.duration (histogram, ms), cache.payload.bytes (histogram) — emitted by ObservableHybridCache.
• Hangfire metrics — jobs enqueued, jobs succeeded, jobs failed (per queue).
• Custom metrics — register Meter and Counter<T> instances anywhere; the kit’s OTel registration picks them up automatically.

Configuration

{
  "OpenTelemetry": {
    "ServiceName": "fullstackhero",
    "Endpoint": "http://localhost:4317",       // OTLP gRPC endpoint
    "Protocol": "Grpc"                          // or "HttpProtobuf"
  }
}

For Aspire local dev, the AppHost wires the dashboard’s OTLP endpoint automatically and traces show up live. For prod, point Endpoint at your observability backend’s OTLP receiver.

Disable specific instrumentations by removing them from AddHeroOpenTelemetry — but the defaults are tuned for cost / signal balance.

Querying observability data

Three queries every team learns:

1. Trace by user — filter by user.id = "..." to see every request a single user made.
2. Trace by tenant — filter by tenant.id = "..." to see one tenant’s load profile.
3. Slowest queries — filter by db.statement EXISTS and order by duration desc.

The kit’s enrichers attach user.id, tenant.id, and correlation.id to every span and log line, so all three are one query away.

What to add yourself

The kit ships infrastructure tracing. You add business tracing — spans for the long-running steps your handlers do that aren’t already covered:

public sealed class GenerateInvoicesCommandHandler(/* ... */)
    : ICommandHandler<GenerateInvoicesCommand, Unit>
{
    private static readonly ActivitySource Source = new("Modules.Billing");

    public async ValueTask<Unit> Handle(GenerateInvoicesCommand cmd, CancellationToken ct)
    {
        using var activity = Source.StartActivity("GenerateInvoicesForPeriod");
        activity?.SetTag("period.year", cmd.PeriodYear);
        activity?.SetTag("period.month", cmd.PeriodMonth);
        // ...
    }
}

Custom counters for domain-specific metrics (“invoices issued per period”, “webhooks delivered per tenant”) give you product dashboards out of the box.

Related

• Auditing module — different concern (compliance/forensic), but uses the same trace/correlation ids.
• Background jobs — Hangfire telemetry filter.
• Web building block — AddHeroLogging + AddHeroOpenTelemetry.