Observe

Why Observability?

Defense without observability is flying blind. You need to know what is happening with your protected agents, not just that Guardrails are running, but what they are catching, how often, and what patterns are emerging. Dome includes built-in telemetry that captures the full picture of agent behavior in production. This data serves three purposes:

Operational visibility: Are your Guardrails working? What is getting blocked? What is the latency impact?
Security monitoring: Are you seeing attack patterns? Anomalous behavior? Escalating threats?
Continuous improvement: What is happening in production that evaluation did not anticipate?

Telemetry

Dome automatically captures telemetry from every protected interaction. When instrumented by the developer, this includes:

Execution Traces

Full traces of LLM and tool execution:

{
  "trace_id": "abc123",
  "timestamp": "2024-01-15T10:30:00Z",
  "agent_id": "support-bot-v2",
  "spans": [
    {
      "name": "input_guardrail",
      "duration_ms": 23,
      "guards_executed": ["prompt-injection", "pii-detection"],
      "result": "pass"
    },
    {
      "name": "llm_call",
      "model": "gpt-4",
      "duration_ms": 1250,
      "tokens": {"input": 450, "output": 230}
    },
    {
      "name": "tool_call",
      "tool": "search_knowledge_base",
      "duration_ms": 89,
      "result": "success"
    },
    {
      "name": "output_guardrail",
      "duration_ms": 18,
      "guards_executed": ["toxicity", "pii-redaction"],
      "result": "pass"
    }
  ]
}

Traces let you see exactly what happened in each interaction—which Guards ran, what the LLM did, which tools were called, and how long each step took.

Instrumentation

To capture full traces, instrument your agent with Dome’s SDK:

from vijil.dome import DomeClient, trace

dome = DomeClient()

@trace("my-agent")
async def handle_request(user_input: str):
    # Input guardrail (automatic)
    filtered_input = await dome.scan_input(user_input)

    # LLM call (traced)
    with trace.span("llm_call"):
        response = await llm.generate(filtered_input)

    # Tool calls (traced)
    with trace.span("tool_call", tool="search"):
        results = await search_tool(response.query)

    # Output guardrail (automatic)
    filtered_output = await dome.scan_output(response.text)

    return filtered_output

Guardrail spans are captured automatically. LLM and tool spans require explicit instrumentation but provide complete visibility into agent behavior.

Metrics

Dome aggregates telemetry into operational metrics:

Protection Metrics

Metric	Description
`inputs_scanned`	Total inputs processed by input Guardrails
`inputs_blocked`	Inputs blocked by Guards
`inputs_redacted`	Inputs modified by redaction Guards
`outputs_scanned`	Total outputs processed by output Guardrails
`outputs_blocked`	Outputs blocked by Guards
`outputs_redacted`	Outputs modified by redaction Guards
`block_rate`	Percentage of interactions blocked

Latency Metrics

Metric	Description
`input_guardrail_p50`	Median input Guardrail latency
`input_guardrail_p99`	99th percentile input Guardrail latency
`output_guardrail_p50`	Median output Guardrail latency
`output_guardrail_p99`	99th percentile output Guardrail latency
`total_overhead_ms`	Total latency added by Dome

Guard-Level Metrics

Metric	Description
`guard.{name}.triggers`	How often this Guard triggers
`guard.{name}.confidence_avg`	Average confidence when triggered
`guard.{name}.latency_p50`	Median Guard execution time

Metrics are available via API, dashboard, and can be exported to your observability stack (Datadog, Prometheus, etc.).

Logs

Dome generates structured logs for every significant event:

Detection Logs

When a Guard detects a threat:

{
  "level": "warn",
  "event": "guard_triggered",
  "timestamp": "2024-01-15T10:30:00Z",
  "agent_id": "support-bot-v2",
  "guard": "prompt-injection",
  "action": "block",
  "confidence": 0.94,
  "input_hash": "sha256:abc...",
  "evidence": "Detected instruction override pattern"
}

Audit Logs

For compliance and security review:

{
  "level": "info",
  "event": "interaction_complete",
  "timestamp": "2024-01-15T10:30:02Z",
  "agent_id": "support-bot-v2",
  "session_id": "sess_xyz",
  "input_guardrail": {"result": "pass", "duration_ms": 23},
  "output_guardrail": {"result": "redact", "duration_ms": 18, "guard": "pii"},
  "total_duration_ms": 1380
}

Logs can be shipped to your SIEM, log aggregator, or stored in Vijil for analysis.

Dashboard

The Dome dashboard provides real-time visibility:

Overview: Block rates, latency, active agents
Guard performance: Which Guards are triggering, at what rates
Threat timeline: Detection events over time
Drill-down: From metric to trace to individual interaction

Next Steps

Observe Traces

Monitor Guardrail performance

How Defense Works

The defense architecture

Overview

Trust Score

Evaluation

Runtime Defense

Reference

Why Observability?

Telemetry

Execution Traces

Instrumentation

Metrics

Protection Metrics

Latency Metrics

Guard-Level Metrics

Logs

Detection Logs

Audit Logs

Dashboard

Next Steps

Observe Traces

How Defense Works

​Why Observability?

​Telemetry

​Execution Traces

​Instrumentation

​Metrics

​Protection Metrics

​Latency Metrics

​Guard-Level Metrics

​Logs

​Detection Logs

​Audit Logs

​Dashboard

​Next Steps

Observe Traces

How Defense Works

Why Observability?

Telemetry

Execution Traces

Instrumentation

Metrics

Protection Metrics

Latency Metrics

Guard-Level Metrics

Logs

Detection Logs

Audit Logs

Dashboard

Next Steps