Site search

Find architecture, research, and terms

Start typing to search the editorial index.

Production practiceAdvanced

Observability and Evidence for AI Runtimes

Distinguish operational telemetry from reviewer-facing evidence, define correlation identities and event requirements, redact sensitive data, and map to OpenTelemetry.

Observability versus evidence

Concern Observability Evidence
Primary user Operator Reviewer, user, auditor
Typical data Traces, metrics, logs Typed decisions, approvals, artifacts, uncertainty
Retention Operational policy Product, legal, and evidence policy
Privacy Infrastructure controls Explicit user-facing minimization and access
Meaning System behavior Why a run produced an outcome

Correlation model

trace IDrun IDevent sequenceaction IDartifact IDcheckpoint IDapproval ID

Event requirements

Every evidence event has schema version, event type, monotonic sequence, UTC timestamp, actor, subject, status, typed data, integrity metadata, and privacy classification.

Redaction

  • No raw credentials or unnecessary prompt content.
  • No prohibited personal data.
  • Use hashes when content need not be copied.
  • Store sensitive artifacts behind access controls and retain redaction decisions as evidence.

C# evidence ledger

using System.ComponentModel.DataAnnotations;

/// <summary>Represents an ordered evidence event.</summary>
public sealed record EvidenceEvent
{
    /// <summary>Gets the stable event type.</summary>
    [Display(Name = "Event Type")]
    public required string EventType { get; init; }

    /// <summary>Gets the sequence within the run.</summary>
    [Display(Name = "Sequence")]
    public required long Sequence { get; init; }

    /// <summary>Gets the event instant.</summary>
    [Display(Name = "Occurred At Utc")]
    public required DateTimeOffset OccurredAtUtc { get; init; }

    /// <summary>Gets the privacy classification.</summary>
    [Display(Name = "Privacy Classification")]
    public required string PrivacyClassification { get; init; }
}

/// <summary>Appends immutable evidence events.</summary>
public interface IEvidenceLedger
{
    /// <summary>Appends an event using optimistic sequence validation.</summary>
    /// <param name="runId">The stable run identifier.</param>
    /// <param name="evidenceEvent">The event to append.</param>
    /// <param name="cancellationToken">Signals cancellation of storage I/O.</param>
    Task AppendAsync(
        string runId,
        EvidenceEvent evidenceEvent,
        CancellationToken cancellationToken);
}

OpenTelemetry mapping

Map model and agent operations to current official semantic conventions where applicable, but preserve MIR-specific lifecycle, policy, approval, checkpoint, artifact, and uncertainty semantics in the evidence layer. OpenTelemetry alone does not define a complete reviewer-facing evidence package. Source: OpenTelemetry GenAI conventions

Source record

References

Suggest a correction
  1. OpenTelemetry project. Cloud Native Computing Foundation. Published Current specification repository; last reviewed 2026-06-24 UTC. Official specification.

  2. National Institute of Standards and Technology. NIST. Published 2023-01-26; last reviewed 2026-06-20 UTC. Government framework.

  3. National Institute of Standards and Technology. NIST. Published 2024-07-26; last reviewed 2026-06-20 UTC. Government profile.