CloudDash AI

Overview

CloudDash AI is a production-grade multi-agent customer support system designed to autonomously handle support inquiries for a cloud infrastructure monitoring SaaS. Built with Python and FastAPI, the system intelligently classifies customer intents, routes queries to specialized AI agents, and grounds its responses in a structured knowledge base using an advanced hybrid RAG (Retrieval-Augmented Generation) pipeline.

Key Capabilities

• Intelligent Routing & Triage: Classifies intent and extracts entities to route customer queries accurately to domain-specific agents.
• Domain-Specific Agents: Dedicated Technical Support, Billing, and Escalation agents.
• Advanced RAG: Dense vector search paired with BM25 keyword matching, Reciprocal Rank Fusion (RRF), and cross-encoder re-ranking.
• Aggregated Handovers: Seamless context preservation across agent domains to prevent the jarring "ping-pong" effect for end users.
• Safety Guardrails: Prompt injection detection, PII redaction, and hallucination checks.
• Comprehensive Observability: Structured JSON logs and trace ID propagation.

Agent Architecture

The system utilizes a custom, asynchronous Orchestrator to manage routing logic and state without the overhead of heavy frameworks like LangGraph. The orchestrator delegates tasks to a registry of specialized agents:

1. Triage Agent: Takes the raw customer query, classifies the intent, and extracts key entities (e.g., customer_id, plan, urgency). It does not answer questions directly but routes them to the appropriate specialist.
2. Technical Agent: Utilizes the hybrid RAG pipeline to retrieve relevant knowledge base articles and provide step-by-step troubleshooting. It automatically escalates if no relevant context can be found.
3. Billing Agent: Focuses on pricing policies and account management, simulating an integration with a mock CRM to fetch real-time invoice and plan data.
4. Escalation Agent: When an AI cannot resolve an issue, this agent compresses the conversation history into a concise summary (under 500 tokens), determines priority, and packages the interaction into a structured ticket for a human operator.

The Hybrid RAG Pipeline

Pure vector search often struggles with exact-match queries for technical jargon (e.g., "SSO", "SAML", "webhook"). CloudDash AI implements a sophisticated 6-step retrieval chain to guarantee highly grounded technical support:

1. Query Rewrite: Condenses the last 3 conversation turns into a standalone question.
2. Dense Retrieval: Performs vector similarity search using ChromaDB and text-embedding-3-small.
3. BM25 Keyword Retrieval: Leverages sparse keyword matching to catch exact technical terminology.
4. Reciprocal Rank Fusion (RRF): Merges the dense and sparse retrieval rankings.
5. Cross-Encoder Re-Ranking: Uses ms-marco-MiniLM-L-6 to rescore the top candidates and significantly improve final top-k precision.
6. Citation Packaging: Attaches exact KB snippet citations and scores to the final agent response.

State & Handover Management

Instead of simple "push" handovers, the Orchestrator performs Aggregated Handovers. For example, if a user asks to fix an alert issue and simultaneously asks to upgrade their plan, the Orchestrator invokes the Technical Agent, detects a required handover, invokes the Billing Agent, and aggregates both responses into a single, cohesive message.

Every agent-to-agent transfer preserves full conversational context, extracts entities, and emits a structured HandoverAuditLog event.

Security & Guardrails

The API layer sits behind strict guardrails to enforce safety:

• Input Guardrail: Uses regex pattern matching to detect prompt injections (e.g., DAN mode, jailbreaks) and filters out off-topic requests before they hit the Orchestrator.
• Output Guardrail: Validates the agent's final payload. It performs regex-based PII redaction (email, phone, credit card) and uses a heuristic overlap check against the retrieved context to detect and prevent hallucinations.

Observability

Every interaction is tracked via structured JSON logs. Trace IDs are propagated across the lifecycle, from the POST /conversations/{id}/messages API endpoint through the Orchestrator, Retrieval Chain, and LLM calls. The system is also designed to integrate cleanly with Langfuse, allowing developers to inspect agent prompts, completions, latency, and context on a per-turn basis.