Machine learning: why computer self-learning provides better business insights

At Automated Intelligence, we analyse data. We take documents, millions or billions of them, and we look for patterns in that information. Why? So that organisations can make sense of what they hold and uncover the valuable information within it.

Machine learning is where we build a model of everything we know about data (structure, words, patterns etc), and then use that model and compare new data to it to make decisions about what it is.

We continue to retrain the base model so that the results become more and more accurate without any human intervention. This process effectively lets the computer become more intelligent over time.

Take a pharmaceutical company for example. It needs to find a specific drug name within the terabytes of its data, but the computer, in analysing that information, might notice that every time that drug is mentioned, another particular word appears within the document as well.

So, the computer is not only going to look for the specific search term, but will look for the other word because it knows, based on hundreds or thousands of hits, that the statistical relevance of those two things is very high.

The technology then returns documents which are every bit as accurate because it has ‘learnt’ about the context. Even if a person could read a million documents, it would be impossible for them to remember them all to find that connection.

Where Machine Learning has really added value to our solutions recently is something called ‘Named Entity Recognition’ for GDPR Subject Access Requests (SAR).

What that means is our technology, AI.DATALIFT, knows how to identify a person even if that person’s name isn’t mentioned in those documents whatsoever. Not once.

It allows us to identify when a piece of content contains details related to a person other than what we are looking for and, in that way, we can flag the data for review and direct the reviewer to the issue instantly. Combined with pattern matching, we can find things that the organisation didn’t even know it had.

How? Well, in analysing the data, we discover that when a name is mentioned, so too is a pattern which looks like a National Insurance Number (NINO). We can start to correlate occurrences and formulate a very accurate relationship of name to NINO.

Indirectly, from the NINO, we can identify other personal information by inference, again with a very high degree of confidence. We can build up a profile of a citizen and report on that data even though we may not actually have realised we had the information.

Ultimately this means more meaningful information- and that means better insights into your customers or users, and a more complete view over your entire data estate.

Using the power of machine learning we can ask questions of the data, the organisation can ask questions of the data, and the data can ultimately ask questions of itself to provide high business-value results.

That’s got us and our customers excited about where we can take machine learning and the problems we can solve for them.