MCP-1st-Birthday/MCP-telemetry

RAG Telemetry XR — Model-Driven Telemetry Configs Powered by MCP + Qdrant

RAG Telemetry XR turns natural language requests like “stream BGP and interface KPIs to 192.0.2.10:57500 over gRPC, 10s interval” into production‑ready model‑driven telemetry configuration. Under the hood, it uses a Retrieval‑Augmented Generation (RAG) pipeline over a curated YANG sensor catalog in Qdrant, then generates clean, validated XR config(extendable to the other vendors) via an LLM. The app runs as a Gradio Space and doubles as a functioning MCP server, making it easy to plug into MCP‑enabled agentic clients like Claude Desktop or Cursor.

• —Live demo: HuggingFace Space
• —Demo video: Youtube
• —Social post X: Linkedin

Why it matters

• —Manual telemetry configuration is time‑consuming and brittle. Sensor paths are tricky, syntax is rigid, and teams waste hours assembling “known good” snippets.
• —RAG Telemetry XR codifies best‑practice YANG sensor paths and emits strict IOS XR telemetry syntax, cutting config time from hours to seconds — with transparency into exactly which sources were used.

What You Can Do

• —Generate IOS XR telemetry config from a plain‑English prompt.
• —Retrieve relevant YANG sensor paths for BGP, OSPF, interfaces, and more.
• —Preview which chunks informed the result for auditability and trust.
• —Use it as a Space UI or call it as an MCP tool inside your favorite agentic client.

How It Works

RAG Telemetry XR uses a lightweight, production‑focused pipeline:

1) Query understanding

• —Your natural language prompt is embedded with text-embedding-3-small.

2) Vector search over curated sensors

• —A Qdrant collection(with different embeddings approaches) (e.g., fixed_window_embeddings) stores YANG sensor paths and context.
• —We query top_k chunks with optional filtering and thresholding.

3) Deterministic config synthesis

• —A strict system prompt enforces valid IOS XR model‑driven telemetry syntax only.
• —The LLM (gpt-4.1-mini by default) renders the final config with the chosen sensor paths.

4) Transparency by design

• —The UI returns a debug panel listing which chunks, files, and scores informed the config.

Features

• —Deterministic XR syntax: emits only telemetry model-driven blocks with sensor-group, destination-group, and subscription — no invalid commands.
• —Vendor‑specific context: YANG sensor catalogs tuned for IOS XR (7.x).
• —Pluggable retrieval: points to cloud Qdrant or a local Dockerized instance.
• —Gradio UI: fast iteration, friendly defaults, and copy‑pasteable output.
• —MCP‑ready: callable as a standalone MCP server or directly from the Space (mcp_server=True).
• —Auditable RAG: shows source chunks and scores for every generation.

New: Validation, Confidence, Presets, Diff, and Audit Logs

• —Validate configs: CLI, UI, and MCP tool catch structural issues and banned tokens.
• —Confidence controls: score threshold slider and min/avg/max score stats.
• —Sensor pack presets: quick chips to bias retrieval (e.g., BGP, Interfaces, OSPF).
• —Diff tool: compare two configs in the UI or via MCP tool.
• —Audit logging: JSONL entries with prompt, settings, chunk IDs/scores, and config hash.

Screenshots

[image] [image]

Live Space Usage

1) Open the Space and enter a prompt

• —Example: “Generate telemetry configuration for Cisco IOS XR about BGP. Use gRPC with no TLS, telemetry server 192.0.2.0 port 57500. Choose relevant BGP sensor paths.”

2) Tune retrieval

• —Adjust top_k (e.g., 5–15), set score_threshold, and choose the Qdrant collection (default: fixed_window_embeddings).
• —Optionally enable sensor pack presets (BGP, Interfaces, OSPF) to bias retrieval.

3) Generate

• —Click “Generate Telemetry Config” to see the XR config and the retrieved chunk list.

4) Apply or iterate

• —Copy the final config into your router or adjust the prompt/sensors and re‑generate.

Local Development

Prereqs

• —Python 3.10+
• —Docker (for local Qdrant)

Install

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# If you plan to use MCP features locally, ensure Gradio 6:
pip install "gradio>=6.0.0"

Environment

cp .env.example .env   # if you keep an example; otherwise create .env
# required
OPENAI_API_KEY=sk-...

# choose ONE Qdrant option
# 1) Cloud
QDRANT_URL=https://YOUR-INSTANCE.qdrant.tech
QDRANT_API_KEY=your-qdrant-api-key

# 2) Local (default host/port)
QDRANT_HOST=localhost
QDRANT_PORT=6333

# optional
EMBEDDING_MODEL=text-embedding-3-small
CHAT_MODEL=gpt-4.1-mini
QDRANT_COLLECTION=fixed_window_embeddings
# provider stub (OpenAI default). Gemini is stubbed for future use.
LLM_PROVIDER=openai

Run the Gradio app

python app.py

Start local Qdrant with included collections

docker run --rm -p 6333:6333 \
  -v $(pwd)/qdrant_storage:/qdrant/storage \
  qdrant/qdrant:v1.8.3

CLI test

python telemetry_rag.py --top-k 10 --collection fixed_window_embeddings \
  "Generate telemetry configuration for Cisco IOS XR about BGP. Use gRPC with no TLS, telemetry server 192.0.2.0 port 57500. Choose relevant BGP sensor paths."

CLI Utilities

Validate a config file or stdin:

python3 validate_config.py path/to/config.txt
# JSON report
python3 validate_config.py path/to/config.txt --json

Generate via CLI and pretty-print JSON:

python3 telemetry_rag.py --json --top-k 10 --collection fixed_window_embeddings \
  "Generate telemetry configuration for Cisco IOS XR about BGP..."

Audit logs are appended to runs/events.jsonl with prompt, models, collection, retrieved chunk ids/scores, stats, and a SHA‑256 of the output config.

Using as an MCP Server

This project ships with two MCP options:

1) Standalone MCP server

• —Start: python telemetry_mcp_server.py
• —Exposes tools:
• —generate_telemetry_config(user_query, top_k, collection_name) → JSON with config, retrieved_chunks, and stats.
• —validate_config(config_text) → JSON with is_valid, errors, warnings, and a parsed summary.
• —diff_config(old_config, new_config) → unified diff string.

Claude Desktop (example claude_desktop_config.json)

{
  "mcpServers": {
    "telemetry-rag": {
      "command": "python",
      "args": ["telemetry_mcp_server.py"],
      "env": {
        "OPENAI_API_KEY": "${OPENAI_API_KEY}",
        "QDRANT_URL": "${QDRANT_URL}",
        "QDRANT_API_KEY": "${QDRANT_API_KEY}"
      }
    }
  }
}

2) Space as MCP server

• —The Gradio app launches with mcp_server=True, allowing MCP‑aware clients to call the Space directly where supported.

MCP Client Demo Script (for your video)

• —Start the server: python telemetry_mcp_server.py
• —In Claude Desktop, enable the telemetry-rag server via the provided config.
• —Ask: “Generate IOS XR telemetry config for OSPF and interface KPIs to 192.0.2.10:57500, 10s.”
• —Show tool call response: copy the generated XR config.
• —Show the Space UI: paste the same prompt, compare retrieved chunks and output.
• —Highlight transparency: which files/chunks from the catalog were used.
• —Paste config into a lab/router (optional) to validate syntax/apply.

Configuration Contract (IOS XR 7.x)

The LLM is constrained to output only the following structure:

telemetry model-driven
 sensor-group <SENSOR_GROUP_NAME>
  sensor-path <PATH_1>
  sensor-path <PATH_2>
 !
 destination-group DG-GRPC
  address-family ipv4
   destination <DEST_IP>
    port <DEST_PORT>
    encoding self-describing-gpb
    protocol grpc no-tls
 !
 subscription <SUBSCRIPTION_NAME>
  sensor-group-id <SENSOR_GROUP_NAME> sample-interval <INTERVAL_MS>
  destination-group-id DG-GRPC
 !

Guardrails

• —No legacy/invalid commands (stream, transport, destination-ip, no tls, etc.).
• —All sensor-path entries live within a sensor-group.
• —All destination settings live within a destination-group.
• —Context is used only to pick relevant sensor paths; not to change syntax.

Qdrant Schema Expectations

Collections are intentionally generic so the retriever works across different payload layouts. We look for these keys when present:

• —Text body: text OR text_preview OR (description + path)
• —Metadata: file_path (or module), chunk_index, and any domain tags

You can bring your own collections — just ensure sensor path text is embedded and discoverable.

Architecture Overview

• —UI: Gradio Blocks (app.py)
• —RAG: retrieve_chunks() → build_context_text() → generate_telemetry_config()
• —Vectors: Qdrant query_points with optional Filter and score thresholds
• —LLM: OpenAI chat completions constrained by a strict XR system prompt
• —MCP: FastMCP server exposing generate_telemetry_config, validate_config, and diff_config
• —Validation: validate_config.py with structural checks and banned token detection
• —Audit: JSONL logs written to runs/events.jsonl

Roadmap

• —Multi‑vendor support (IOS XE, JunOS, NX‑OS) with vendor‑aware prompts
• —Transport profiles (TLS/mTLS, Kafka, gNMI) with validated syntax templates
• —Advanced agent workflows (planning + diff/merge on running configs)
• —Confidence scoring and fallback heuristics for low‑recall queries
• —Sensor pack management and Space‑editable retrieval filters

Team

• —museltabares

License

Original work created during the hackathon (Nov 14–30).

Acknowledgments

• —Qdrant for vector search
• —Gradio for rapid UI
• —MCP (Model Context Protocol) for a clean tool interface

Development reference: https://huggingface.co/docs/hub/spaces-config-reference