How do you evaluate AIOps? (Image Credit: Gerd Altmann from Pixabay )AIOps is the application of AI to IT operations. To understand how we might evaluate an AIOps platform, we need to recognize which elements of AI are relevant to IT Operations. Within AIOps circles there has emerged a notion of AI that is different to many proposed definitions. It shapes AI and makes it relevant to AIOps. I like to call this concept the five dimensions of AIOps.

What are the five dimensions?

In the AIOps world, we look at AI as the reverse engineering of algorithms that underlie the brain’s cognitive processes. These have evolved over time to cope with, and to manipulate, complex and rapidly changing environments. Fundamentally, we are looking at five distinct types of algorithms.

  1. This algorithm takes a stream of data and selects which part of that stream, and data subsets, are of most interest and require further investigation. This is data selection.
  2. Once data has been selected, the second type of algorithm works on discovering patterns in the data. For example, this algorithm helps to make sense out of the data, and to discover how different data points are relevant to one another.
  3. Inferencing algorithms help you draw out all the information that’s implicit in the patterns that describe datasets.
  4. The results from data selection, pattern discovery and automated inferencing are communicated by this algorithm. It may be to human users or other applications, in a way that they can be presented and discussed.
  5. The transition of AI into robotics. You take the first three algorithms and instead of communicating their results to a human being, this algorithm makes it possible to communicate them to a machine, which can then act and respond to that contact.

These five algorithms are critical to evaluating the effectiveness of an AIOps platform. The best way to evaluate is to reduce each algorithm into a single question. This will enable the user to detect whether the platform exhibits all five dimensions and can deliver value.

Does the AIOps platform deliver value

What do I mean by value? From my perspective, it’s the ability to ingest signals coming from multiple sources in your infrastructure, integrate them, understand them, and then ensure delivery to the right recipients who can then take action. An AIOps platform should be able to do this. So, what five questions should we be asking the platform?

1. Is anything going on?

To start the evaluation, ask yourself whether anything is going on in the IT environment. Remember you’re taking in large amounts of data, much of which is redundant or just noise, and it’s difficult to know whether something of interest is going on. The very first thing an AIOps platform ought to be able to do is ingest data and either highlight areas of concern or indicate that everything is normal. From an AIOps dimensional perspective, this question relates to data selection.

2. What is going on?

Once your platform has processed the first question, it must then determine what is going on. This question corresponds to the dimension of pattern discovery. This algorithm looks at the data, tries to structure it and classify it, and attempts to understand what is taking place within the data. This includes instantly recognizing patterns in the data or training the algorithm, via machine learning techniques, to recognize patterns. You can also use neural networks and deep learning technologies to answer this question.

Many commercially available platforms don’t answer the first question but jump immediately to this second question. They don’t try and qualify the data that’s coming in. Instead, they start by looking for patterns. By doing this you could be wasting your time on discovering noise patterns, rather than proper incidents.

3. Why is it going on?

The next natural step is to ask yourself why this is occurring. Why are these patterns present? What is the relationship between these patterns? What does it mean? This question corresponds to the dimension of inferencing, basically going from the correlational aspects of pattern discovery to causal analysis.

4. Who should care about this?

Once you’ve established why something is going on, it then becomes critical to communicate the results of the analysis to the correct agent so it can react to what has surfaced. This relates to the communication algorithms which constitute the previously outlined fourth dimension.

5. What should be done?

Finally, once you have determined who should care about this, the last piece of the puzzle is to establish what actions need to take place to resolve the issue. This is equivalent to the fifth dimension of AIOps which relates to robotic automation.

In summary, the five dimensions can be used to evaluate whether an AIOps platform can answer five simple questions.

The market

If you go to the market and look at all the different types of AIOps offerings, you will see that many solutions will be able to answer one or two of these questions, but very few will be able to answer all five.

Many technologies, such as unified monitoring and event and log management, have been designed to answer the second and third questions, but struggle to establish who should care about an incident and how it should be resolved.

Smart alerting, runbook automation or service desk vendors, will be able to answer questions four and five, but they won’t be able to answer the first three. At the moment, the majority of vendors are taking fledgling steps in terms of answering the fifth question: What should be done?

The five questions provide users with a scorecard to assess the different offerings on the market, and in a quick way provide a process to clarify whether the offering can deliver the full value of AIOps. Typically, many vendors try to build AIOps on top of an existing IT Operations management category. But, as you would expect, this type of solution is limited in terms of the number of questions it can answer. In reality, there are very few vendors that can actually answer all five.

moogsoft logoMoogsoft is a pioneer and leading provider of AIOps solutions that help IT teams work faster and smarter. With patented AI analyzing billions of events daily across the world’s most complex IT environments, the Moogsoft AIOps platform helps the world’s top enterprises avoid outages, automate service assurance, and accelerate digital transformation initiatives. Founded in 2011, Moogsoft has more than 120 customers worldwide including SAP SuccessFactors, American Airlines, Fannie Mae, Yahoo!, and HCL Technologies. It has established strategic partnerships with leading managed service providers and outsourcing organizations including AWS, Cisco, HCL Technologies, TCS and Wipro. For more information about Moogsoft’s AIOps platform and its newest addition of customers, visit


Please enter your comment!
Please enter your name here