The GenAI Chatbot Transforming Network Operations at Southern California Edison

The Network Operations Center (NOC) at Southern California Edison (SCE) oversees one of the largest private utility telecommunication networks in the country. Operating 24/7, SCE's NOC is responsible for ensuring reliable data network operation, performance monitoring and real-time issue response.

In collaboration with NVIDIA and World Wide Technology (WWT), SCE built a generative AI (GenAI) chatbot that leverages retrieval-augmented generation (RAG) as the first module.

The platform will leverage predictive analytics and machine learning to anticipate and prevent future telecommunication network issues. It will also allow SCE's NOC team to efficiently troubleshoot tickets, access real-time network trends, proactively mitigate incidents, and streamline onboarding and training.

The solution is designed to optimize operational efficiency, enhance reliability and improve resiliency across SCE's telecommunications network.

Challenges and objectives

The modern electric grid is becoming much more dynamic, which is driving utilities to deploy orders of magnitude more monitoring and control devices. These devices must all be connected by the field area telecommunications network. The number of devices on the telecommunication network is projected to grow by 30 percent each year. The data network managed by the NOC is becoming more critical to operations while, at the same time, tolerance for data network outages is decreasing.

Traditional utility network monitoring and issue resolution rely on reactive processes, which can potentially lead to outages, downtime and operational inefficiencies. Real-time data visibility and proactive incident prevention are crucial for network resilience, especially in private networks supporting essential services.

SCE's GenAI chatbot addresses an industrychallenge of shifting from reactive to proactive network management. By integrating GenAI with predictive analytics and machine learning, SCE's chatbot is the first element of a platform that will transform the NOC into a data-driven hub capable of identifying and preventing emerging issues before they disrupt operations.

Solution implementation

To manage one of the largest private utility telecommunication networks in the country, SCE's Project Orca integrates GenAI, machine learning and predictive analytics to drive impactful and scalable outcomes. A critical component of this transformative solution is NVIDIA NIM, a set of optimized inference microservices for the latest AI models that accelerates the deployment and management of GenAI applications, and NVIDIA NeMo Retriever for enhanced RAG.

The system consists of a RAG pipeline using the Llama 3.1 8B NIM microservice for LLM inference; the NeMo Retriever extraction microservice for parsing complex PDF documentation; and the NeMo Retriever embedding microservice for generating embeddings. It is designed to operate on an ultra-secure, air-gapped network, leveraging a combination of NVIDIA H100 and L40S GPUs. Project Orca has successfully advanced through each stage of NVIDIA's AI Foundry motion within WWT's Advanced Technology Center (ATC).

Integrating NIM microservices, part of the NVIDIA AI Enterprise software platform for high-performance AI inference on NVIDIA accelerated infrastructure, will allow SCE to process data from more than 100,000 network assets.

Use case benefits

Key benefits from Project Orca include:

Reduced downtime and enhanced network reliability: AI-driven predictive analytics will identify and mitigate potential network issues before they escalate into outages.
Faster and more efficient ticket resolution: The GenAI chatbot is built to enable NOC operators to troubleshoot tickets more efficiently, providing real-time access to actionable data and insights.
Improved resource allocation and productivity: Automation and machine learning handle routine tasks autonomously, allowing NOC staff to focus on more complex and strategic challenges.
Long-term efficiencies and reduced maintenance costs: A proactive, data-driven approach to network management reduces the need for reactive maintenance.

The importance of trusted AI partners

WWT and SCE worked with NVIDIA to develop, test and implement the Orca GenAI chatbot by focusing on:

Technology integration
Cross-functional collaboration
Demonstrating value and impact early and often
Continuous improvement and feedback loops
Project leadership and oversight

Throughout the implementation process, SCE has focused on quantifying the benefits achieved through Project Orca, reinforcing the importance of AI-driven automation in transforming its network operations.

Conclusion

The implementation of SCE's AI-driven platform for NOC operations represents a transformative step forward in AI-powered network management — one that can profoundly impact SCE and the broader utility and telecommunications industry. By shifting from a reactive to a proactive incident management approach, SCE sets a new benchmark for telecommunication network management that is poised to influence industry standards for reliability and efficiency.

SCE's scalable GenAI-driven solution ensures its private network remains resilient and adaptable to future needs — especially as load demand on the network is only expected to grow in the face of increasing AI adoption and the electrification of other industries.

The key factors and capabilities that led SCE to pursue a GenAI chatbot using NVIDIA NIM include advanced data processing and analysis; proactive incident management; enhanced decision-making capabilities; improved troubleshooting efficiency; scalability and adaptability; and cost-effectiveness.

Leveraging the close collaboration between NVIDIA and WWT, SCE has demonstrated the potential of these capabilities to enhance network resiliency, reduce operational challenges and improve service delivery.

As GenAI models continue to evolve, SCE's new AI-driven platform will continue to drive innovation, enabling future-ready utility telecommunication network for Southern California Edison and its customers.

Hear from Southern California Edison in an industry panel at NVIDIA GTC, "Energy Transition: Impact of Generative AI in the Power Ecosystem of Generation, Transmission, and Distribution."