Insights

Redefining Incident Triage and Recovery with Agentic AI

Priyadarshini Jayakumar, Manager
and
UST AI and Engineering Team

Introduction

In today’s always-on digital landscape, operational resilience is no longer optional—it’s mission-critical. Telecom and enterprise platforms face surging customer demand, complex system integrations, and unpredictable incident scenarios that can quickly escalate into outages if not resolved swiftly.

To address this challenge, GeneiOps was co-developed as a joint effort between a leading US telecom giant and UST. GeneiOps is an AI agent built as a comprehensive solution that integrates generative AI (GenAI) with automated operations (AutoOps) to streamline incident triage and recovery. The result: faster response times, automated root cause analysis (RCA), and self-healing capabilities that scale across global enterprises.

 DIVIDER

Why GeneiOps?

Traditional operations models rely heavily on human triage and manual execution of recovery steps. While effective at smaller scales, these methods struggle under the weight of large-scale, real-time operations. GeneiOps was built to shift this paradigm—offloading repetitive, time-sensitive tasks to intelligent AI agents, while empowering operations teams to focus on higher-value decisions.

At its core, GeneiOps connects telemetry tools, knowledge sources, and automation frameworks into a seamless orchestration engine. With LangGraph for intelligent routing, LangChain for reasoning, and Claude Sonnet 3.5 as the primary large language models, GeneiOps delivers scalable intelligence directly into incident workflows.

 DIVIDER

The GeneiOps agentic framework

The GeneiOps agent is designed around five key types of work:

• Issue Triaging–Directly interrogates observability tools like Splunk; identifies anomalies and determines potential causes.
• Alert Monitoring–Acknowledges alerts, assigns Service Level Agreements (SLAs), and runs automation tasks such as sanity checks or environment promotions.
• Health Check Monitoring–Performs proactive monitoring of live stacks every 15 minutes to flag degradations early.
• Alert Trend Analysis–Detects repeating alerts over defined time frames and provides historical insights.
• Automation Support–Executes recovery scripts and operational tasks; continuously enriches the knowledge base.

 DIVIDER

The Architecture behind the agent

The GeneiOps architecture integrates telemetry inputs (Splunk, AppD, Grafana, Pyro anomalies, incident reports) into the GenAI layer, where AWS Bedrock, vector databases, and knowledge sources interpret data. The AutoOps layer then handles RCA, triage, and automated recovery, with outputs delivered directly to collaboration tools like Slack and Microsoft Teams.

Key features include:

• Auto RCA with decision logic to identify root causes.
• Automation via text chat commands, enabling seamless execution.
• Continuous knowledge base integration from standard operating procedures (SOP) and Teams chats.
• Leadership summary reports for visibility into trends and metrics.

DIVIDER

Real-world benefits

• Faster Incident Recovery: Automated triage and RCA to reduce mean time to resolution (MTTR).
• Proactive Monitoring: Continuous health checks to minimize downtime risk.
• Scalable Automation: Supports 20–25+ repeatable automation tasks across environments.
• Knowledge-Infused Operations: Captures expertise from SOPs and chats to ease future incident resolution.
• Cross-Platform Integration: Works seamlessly across AWS CloudWatch, Splunk, Slack, and Teams.

DIVIDER

Looking ahead

As enterprises adopt AI-driven operations, GeneiOps represents a new frontier where agentic AI and automation converge to deliver resilient, self-healing systems. For telecom companies and beyond, this means less time spent firefighting and more time spent innovating.

The collaboration between this leading US telecom giant and UST showcases how industry expertise and AI innovation can align to create globally scalable solutions. With GeneiOps, we’re moving closer to a future where systems manage themselves, and human teams focus on strategy, creativity, and growth.

Interested in learning more about GeneiOps? Connect with UST AI and engineering experts to explore how this framework can be applied to your enterprise operations.