awesome-AIOps

A curated list of awesome academic researches and industrial materials about Artificial Intelligence for IT Operations (AIOps).

• Researchers
• Industrial Materials
  • Competitions
  • White Papers
  • Blogs & Tutorials & Magazines
  • Benchmarks
  • Tools
  • Companies
• Academic Materials
  • Talks
  • Workshops
• Papers
  • Survey & Empirical Study
  • Benchmarks
  • (Large) Language Models for IT Operations
  • Knowledge Graph for AIOps
  • Microservices and Serverless
  • Dependency and Tracing
  • Anomaly/Failure Detection
  • Root Cause Analysis
  • Incident and Alarm Management
  • Node, Disk, and Storage
  • VM Analysis and Management
  • Deployment
• Datasets
• Others
  • Courses

Researchers

Industrial Materials

Competitions

• [AIOps Challenge] A series of AIOps competitions hosted by Tsinghua University
• [PAKDD2020] Alibaba AIOps Competition

White Papers

• [VMware] Proactive Incident and Problem Management
• [GREATOPS 高效运维社区] 《企业级 AIOps 实施建议》白皮书
• [Awesome Open Source] Aiops Handbook

Blogs & Tutorials & Magazines

• [Moogsoft] What is AIOps?
• [Tsinghua University] 清华裴丹：AIOps落地的15条原则
• [Tsinghua University] 清华裴丹：AIOps效果落地最后一公里
• [Alibaba Cloud] 基于大数据的智能网络分析-齐天
• [Microsoft] Advancing Azure service quality with artificial intelligence: AIOps
• [Grafana] GrafanaCON: Grafana Observability Conference 2022
• [InfoQ] 2023，可观测性需求将迎来“爆发之年”？
• [Alibaba] 阿里云张建锋谈新型计算体系：云正在重构硬件、软件和终端世界

Benchmarks

• [Cornell] DeathStarBench (An open-source benchmark suite for cloud microservices)
• [Google Cloud] Online Boutique (A microservices demo application)
• [Fudan] Train Ticket (A benchmark microservice system)
• [Weaveworks] Sock Shop (A microservices demo application)

Tools

• [Log Analytics] LogPAI
• [AI for Cloud Operation] OpsPAI
• [Outlier Detection] PyOD
• [Anomaly Detection] ADTK
• [Anomaly Detection] PySAD
• [Online Machine Learning] River
• [Online Machine Learning] scikit-multiflow
• [Fault Injection] Chaos Mesh
• [Fault Injection] ChaosBlade
• [Container Monitoring] cAdvisor
• [Performance Monitoring] Netdata
• [Anomaly Detection Labeling Tool] Microsoft TagAnomaly
• [Serverless App Dev. Framework] AWS Serverless Application Model (AWS SAM)
• [Performance Testing Tool] Locust
• [Alibaba Java Diagnostic Tool] Arthas

Companies

• Datadog: A monitoring and security platform for cloud applications
• 必示 bizseer
• 日志易
• 博睿数据
• 听云 TINGYUN: 端到端的全平台应用性能管理系统
• Loom Systems
• Keep: Open-source alert management and AIOps platform

Academic Materials

Talks

• [Michael R. Lyu] Reliability-Driven AIOps for Cloud Resilience (Keynote talk at ICSE '21)

Workshops

• ICSE21 Workshop on Cloud Intelligence
• AAAI-20 Workshop on Cloud Intelligence
• AIOPS 2020 (International Workshop on Artificial Intelligence for IT Operations)

Papers

Survey & Empirical Study

• [arXiv '24] A Survey on Failure Analysis and Fault Injection in AI Systems
• [arXiv '23] AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges
• [CSUR '22] Anomaly Detection and Failure Root Cause Analysis in (Micro) Service-Based Cloud Applications: A Survey
• [ASE '22] Going through the Life Cycle of Faults in Clouds: Guidelines on Fault Handling
• [arXiv '21] Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection
• [CSUR '21] A Survey on Automated Log Analysis for Reliability Engineering
• [ESEC/FSE '20] Towards intelligent incident management: why we need it and how we make it
• [arXiv '20] A Systematic Mapping Study in AIOps
• [ICSE '19] AIOps: Real-World Challenges and Research Innovations
• [HotOS '19] What bugs cause production cloud incidents?
• [ISSRE '16] Experience Report: System Log Analysis for Anomaly Detection
• [ASE '13] Software analytics for incident management of online services: An experience report

Benchmarks

• [arXiv '22] Constructing Large-Scale Real-World Benchmark Datasets for AIOps
• [ASPLOS '19] An Open-Source Benchmark Suite for Microservices and Their Hardware-Software Implications for Cloud and Edge Systems

(Large) Language Models for IT Operations

• [ISSTA '24] LILAC: Log Parsing using LLMs with Adaptive Parsing Cache
• [arXiv '24] Exploring LLM-based Agents for Root Cause Analysis
• [arXiv '24] Nissist: An Incident Mitigation Copilot based on Troubleshooting Guides
• [arXiv '24] Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4
• [arXiv '23] Automatic Root Cause Analysis via Large Language Models for Cloud Incidents
• [arXiv '23] OpsEval: A Comprehensive Task-Oriented AIOps Benchmark for Large Language Models
• [arXiv '23] Xpert: Empowering Incident Management with Query Recommendations via Large Language Models
• [arXiv '23] Exploring the Effectiveness of LLMs in Automated Logging Generation: An Empirical Study
• [arXiv '23] Assess and Summarize: Improve Outage Understanding with Large Language Models
• [arXiv '23] Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answering
• [arXiv '23] Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models
• [SoCC '19] A System-Wide Debugging Assistant Powered by Natural Language Processing

Knowledge Graph for AIOps

• [ICSE-SEIP '22] Mining Root Cause Knowledge from Cloud Service Incident Investigations for AIOps
• [ICSE-SEIP '21] Neural knowledge extraction from cloud service incidents
• [arXiv '21] SoftNER: Mining Knowledge Graphs From Cloud Incidents
• [APPLSCI '20] A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications

Microservices and Serverless

• [ASPLOS '21] Sage: Practical & Scalable ML-Driven Performance Debugging in Microservices
• [ICDCS '21] Defuse: A Dependency-Guided Function Scheduler to Mitigate Cold Starts on FaaS Platforms
• [FSE '20] Graph-based trace analysis for microservice architecture understanding and problem diagnosis
• [OSDI '20] FIRM: An Intelligent Fine-grained Resource Management Framework for SLO-Oriented Microservices
• [ESEC/FSE '19] Latent Error Prediction and Fault Localization for Microservice Applications by Learning from System Trace Logs
• [TSE '18] Fault Analysis and Debugging of Microservice Systems: Industrial Survey, Benchmark System, and Empirical Study

Dependency and Tracing

• [ASE '21] AID: Efficient Prediction of Aggregated Intensity of Dependency in Large-scale Cloud Systems [code]
• [NSDI '07] X-Trace: A Pervasive Network Tracing Framework
• [HotNets '06] Discovering Dependencies for Network Management

Anomaly/Failure Detection

• [POMACS '24] The Tale of Errors in Microservices [data]
• [ICSE '23] CONAN: Diagnosing Batch Failures for Cloud Systems
• [ISSRE '22] Share or Not Share? Towards the Practicability of Deep Models for Unsupervised Anomaly Detection in Modern Online Systems [code]
• [ICSE '22] Adaptive Performance Anomaly Detection for Online Service Systems via Pattern Sketching [code]
• [KDD '19] Time-Series Anomaly Detection Service at Microsoft
• [ESEC/FSE '18] Identifying Impactful Service System Problems via Log Analysis
• [CCS '17] DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning

Root Cause Analysis

• [SIGCOMM '23] Murphy: Performance Diagnosis of Distributed Cloud Applications
• [FSE '23] Nezha: Interpretable Fine-Grained Root Causes Analysis for Microservices on Multi-modal Observability Data
• [OSDI '18] Capturing and Enhancing In Situ System Observability for Failure Detection

Incident and Alarm Management

• [ATC '23] AutoARTS: Taxonomy, Insights and Tools for Root Cause Labelling of Incidents in Microsoft Azure
• [ICSE '23] Incident-aware Duplicate Ticket Aggregation for Cloud Systems
• [SoCC '22] How to Fight Production Incidents? An Empirical Study on a Large-scale Cloud Service
• [DSN '22] Characterizing and Mitigating Anti-patterns of Alerts in Industrial Cloud Systems
• [USENIX ATC '21] Fighting the Fog of War: Automated Incident Detection for Cloud Systems
• [ASE '21] Graph-based Incident Aggregation for Large-Scale Online Service Systems
• [ASE '21] Groot: An Event-graph-based Approach for Root Cause Analysis in Industrial Settings
• [SIGCOMM '20] Scouts: Improving the Diagnosis Process Through Domain-customized Incident Routing
• [ASE '20] How Incidental are the Incidents?: Characterizing and Prioritizing Incidents for Large-Scale Online Service Systems
• [ESEC/FSE '20] Identifying linked incidents in large-scale online service systems
• [ESEC/FSE '20] Efficient incident identification from multi-dimensional issue reports via meta-heuristic search
• [ESEC/FSE '20] Real-time incident prediction for online service systems
• [ESEC/FSE '20] How to mitigate the incident? an effective troubleshooting guide recommendation technique for online service systems
• [ICSE '20] Understanding and Handling Alert Storm for Online Service Systems
• [HotOS '19] What bugs cause production cloud incidents?
• [ASE '19] Continuous Incident Triage for Large-Scale Online Service Systems
• [ICSE '19] An empirical investigation of incident triage for online service systems
• [WWW '19] Outage Prediction and Diagnosis for Cloud Service Systems
• [KDD '14] Correlating Events with Time Series for Incident Diagnosis

Node, Disk, and Storage

• [FAST '23] Perseus: A Fail-Slow Detection Framework for Cloud Storage Systems [data]
• [DSN '21] General Feature Selection for Failure Prediction in Large-scale SSD Deployment
• [TOSEM '20] Predicting Node Failures in an Ultra-Large-Scale Cloud Computing Platform: An AIOps Solution
• [ICDCS '20] Toward Adaptive Disk Failure Prediction via Stream Mining
• [VLDB '20] Diagnosing root causes of intermittent slow queries in cloud databases
• [USENIX ATC '19] IASO: A Fail-Slow Detection and Mitigation Framework for Distributed Storage Services
• [NSDI '18] Deepview: Virtual Disk Failure Diagnosis and Pattern Detection for Azure
• [ESEC/FSE '18] Predicting Node Failure in Cloud Service Systems
• [USENIX ATC '18] Improving Service Availability of Cloud Systems by Predicting Disk Error

VM Analysis and Management

• [NSDI '22] CloudCluster: Unearthing the Functional Structure of a Cloud Service
• [OSDI '20] Predictive and Adaptive Failure Mitigation to Avert Production Cloud VM Interruptions

Deployment

• [SOSP '21] Understanding and Detecting Software Upgrade Failures in Distributed Systems
• [NSDI '20] Gandalf: An Intelligent, End-To-End Analytics Service for Safe Deployment in Large-Scale Cloud Infrastructure

Datasets

• [CUHK] Loghub
• [Microsoft Azure] Azure Public Dataset
• [Tsinghua] AIOps Challenge Dataset
• [Google] Cluster Traces
• [Backblaze] Hard Drive Dataset
• [Baidu] SMART Dataset of PAKDD CUP 2020
• [Red Hat] Ceph Device Telemetry Dataset
• [Alibaba] SSD SMART logs and failure data
• [Alibaba] Alibaba Cluster Trace Program
• [CloudWise] GAIA Dataset
• [Huawei Cloud] Serverless traces

Others

Courses

• [Coursera] Cloud-Based Network Design & Management Techniques
• [Tsinghua] AIOps Course of Tsinghua