> ## Documentation Index
> Fetch the complete documentation index at: https://noradocs.solomontsao.com/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenTelemetry export

> Export Nora's GenAI telemetry — chat spans, token/cost metrics, and per-agent resource gauges — over OTLP and Prometheus into the observability stack you already run.

# OpenTelemetry export

Nora can export its agent telemetry under the OpenTelemetry [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`), so per-agent token usage, estimated cost, and runtime resource metrics flow into the Datadog / Grafana / Langfuse / Honeycomb stack you already operate — no Nora-specific agent or scraper required.

It is **disabled by default** and **fail-open**: a collector outage or misconfiguration never affects chat or any other request path.

## What gets exported

**Spans (OTLP only):** one `chat` span per recorded LLM exchange, with `gen_ai.operation.name=chat`, `gen_ai.request.model`, `gen_ai.system` (provider), `gen_ai.usage.input_tokens` / `output_tokens`, `gen_ai.conversation.id` (session), and `nora.agent.id` / `nora.runtime.family` / `nora.sandbox.profile` / `nora.source`. The span is bracketed around the runtime call, so its duration reflects real latency.

**Metrics (OTLP and/or Prometheus):**

| Instrument                  | Type      | Notes                                                                                           |
| --------------------------- | --------- | ----------------------------------------------------------------------------------------------- |
| `gen_ai.client.token.usage` | histogram | Tokens per exchange, split by `gen_ai.token.type` (`input` / `output`)                          |
| `nora.agent.cost.usd`       | counter   | Estimated spend, attributed by agent + model, using the same model-aware pricing as `/app/cost` |
| `nora.agent.cpu.percent`    | gauge     | Latest sampled container CPU per running agent                                                  |
| `nora.agent.memory.percent` | gauge     | Latest sampled container memory percent per running agent                                       |
| `nora.agent.memory.usage`   | gauge     | Latest sampled container memory (MB) per running agent                                          |

<Note>
  Nora observes telemetry at the **exchange** granularity it actually receives from the runtimes.
  Per-LLM-call and per-tool-call sub-spans depend on event streams OpenClaw and Hermes do not
  currently expose to the control plane, and are tracked as a future enhancement. A2A Agent Cards
  are likewise on the roadmap.
</Note>

## Enable it

Set the master switch and at least one exporter, then restart the backend:

```bash theme={null}
NORA_OTEL_ENABLED=true

# OTLP/HTTP push — metrics AND spans (an OpenTelemetry Collector, Grafana Alloy,
# or a vendor OTLP endpoint). Auth headers are read natively by the exporters.
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
# OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer%20<token>
OTEL_SERVICE_NAME=nora-control-plane

# Prometheus pull — metrics only (0 or empty disables). Defaults to loopback;
# the scrape surface is unauthenticated and exposes agent/model/provider labels,
# so widen the bind only behind an internal network or firewall.
NORA_OTEL_PROMETHEUS_PORT=9464
NORA_OTEL_PROMETHEUS_HOST=127.0.0.1
```

You can run either exporter or both. Spans are OTLP-only — Prometheus carries metrics, not traces. With the Prometheus exporter on, scrape `http://<backend-host>:9464/metrics`.

## Cost accuracy

The `nora.agent.cost.usd` counter is emitted **unrounded** per exchange (sub-cent values accumulate accurately), using `COST_PER_1K_TOKENS` and the optional per-model `COST_MODEL_RATES_JSON` map — see [environment variables](/configuration/environment-variables). Configure those rates to match your providers for meaningful cost dashboards.

## Verify

With Prometheus enabled:

```bash theme={null}
curl -s http://localhost:9464/metrics | grep -E 'gen_ai_client_token_usage|nora_agent_cost_usd'
```

Drive one agent chat, then re-scrape — the token histogram and cost counter will carry your agent, model, runtime family, and sandbox profile labels. For a NemoClaw smoke, deploy an OpenClaw agent with `sandbox_profile=nemoclaw`; the export should show `nora.runtime.family=openclaw` and `nora.sandbox.profile=nemoclaw` without any API-key labels.