3 Steps to Automatic Incident Categorisation

3 Steps to Automatic Incident Categorisation

Are you trying to help your IT support agents with Artificial Intelligence? Great! You and your team don’t have the time to become experts in things like “Cluster Purity,” “Stop Words,” or “Levenshtein Distance algorithm”? Then this article might help you on how to benefit from AI-powered Incident Categorization in few steps.

Step 1 - Optimize Your Category Structure

Before you can even think about what training data to use for your incident categorization, what filters to apply, and how many records to use, you will need to think about whether your current incident categorization does not deserve an overhaul. Based on samples I’ve seen, some ITSM solutions might have categorization trees that work immediately with common AI algorithms. I have also seen categorization structures with over 150 categories on the first level and a very unbalanced distribution of results.
Incident categorization (thumbnail)

Undistributed results mean that some overwhelming number of incidents have been assigned to few categories such as “Miscellaneous” or “Inquiry,” while other categories have been used only 0.01 % of all records. No algorithm will like such training data. The introduction of automatic incident categorization might be, therefore, a great time to take another look at your categorization tree. Would you find a meaningful first-level categorization with some 7 to 15 categories?

Categorization is an art for itself. There is no one-size-fits-all concept because of the different nature of service desks and their support coverage. Furthermore, end users often do not know to which category an issue belongs. But would some of the following common categories for any IT organization be a better starting point than what you have got now: Access Management, Asset Management, ERP, Communication, Connectivity, Customer Relationship Management, End-User Devices, Facility Management, Financial Management, General Office Tools, Human Resource Management, Information Security, Infrastructure Management, IT Service Management, Printing.

You also might want to think right away of the entire category tree. Which categories and which subcategories do you want to offer for two-level categorization in the future? Is the second level related to the services you offer or simply a subcategory? In the latter case, you might want to simplify the first level categories even further into something such: Access, Software, Hardware, Printing, Network, and Database.

If you changed your categorization, you could now consider how to get a decent number of incidents for each category. You can recategorize past incidents, or you can just work for few months with the new categories and accumulate enough training data without extra effort.

Step 2 - Pick Your Training Data

Picking a meaningful set of source data for the AI algorithms is less complicated than you might think. It’s for sure much less labor-intensive than training a chatbot to recognize intents and the corresponding actions. The training set for the incident categorization should consider mainly two factors: quantity and relevance.

Relevance is important regarding the timeliness of your data. There might be little value in using categorization, which has been used 2 years ago with a different support scope or a different IT architecture (cloud, on-premise, etc.).

Secondly, and more importantly, the quantity of data is important. Every ITSM solution with AI-powered incident categorization might be somehow different in how many records are optimal. Still, in Efecte, the rule of thumb is that the training data should have at least 500 records for each category. Furthermore, the total amount of data is somehow relevant. It might be that throwing more than 50000 records at the algorithm actually doesn’t improve the results much anymore. But you do need enough data for each category. Ultimately,” picking” your training data might mean nothing else than archiving outdated and irrelevant records.

Based on our experience, customers do not spend time filtering or curating the data manually when using a powerful algorithm that recognizes out-of-the-box the most common characters and words that don’t add value. Once you have enough relevant training data, your incident categorization should give you good enough suggestions to aid the agents in their daily work.

Step 3 - Scale your Confidence Scores

The last and simple step before going live with your incident categorization is more of aesthetic nature but makes a difference in user acceptance of the machine learning technology: scaling your confidence scores. A confidence score is a percentage figure displayed with each suggested category illustrating how confident the algorithm is with the suggestion fitting to the corresponding incident.

As mentioned earlier, in theory, one needs enough training data for each category. But real life is never perfect. Suppose you haven’t got enough training data, especially after changing your categories. In that case, the confidence rates displayed with the suggested categories might be awfully low, even for the best scoring results. Luckily, pretty much every AI implementation for incident categorizations allows you to adjust the scaling factor of the confidence rates. I’m sure after two or three attempts of changing the scaling factor, you will get meaningful confidence rates that do make sense to the unsuspecting agent using the AI-powered incident categorization for the first time.


Automated or assisted incident categorization does not have to be rocket science requiring a dedicated data scientist to create value. If you get the foundation right, i.e., the actual category structure, then the rest should be a breeze. Yet, it can help a lot in the daily life of an IT support agent.

If you want to know more about how incident categorization works in Efecte or you want to see a live demo, then do not hesitate to check in with us.

Contact us now

AI and ITSM 5 tips


 Peter Schneider

Written By -

I am the Chief Evangelist and Storyteller @ Efecte. I'm the former Chief Product Officer of Efecte and have a passion for sharing industry insights and how customers can benefit from our offering.

Recent blog posts

We implemented AI in our Service Desk: Here is what we found.

April 29, 2024

At Efecte, we often preach the benefits of AI for Service Management and we firmly believe in practicing what we preach. We have already talked about..


Will AI replace the Service Desk Agent?

April 2, 2024

The work of service desk teams is increasingly demanding, partly due to the higher complexity of the topics they address. Without more advanced..


Getting started with AI in IT Service Management: 3 steps to success

February 15, 2024

Digital transformation has revolutionized many areas of modern business, but the IT Service Desk has been left behind. According to Gartner, over 80%..