What is accuracy and precision in machine learning?
Machine learning is a branch of artificial intelligence and deals with the improvement of problem-solving by computers without the need for a human to program this behavior. The computer takes in data with the desired connections and learns to establish these connections with new data itself.
We would like to explain the differences to you using an example of the detection of bank robbers on the images of surveillance cameras.
The system receives many pictures of bank visitors and for each picture the information whether it is a bank robber or not. The model for receiving this information can be very different. Whether it is a neural network, a Bayesian network, or another system, you want to be able to measure the performance of the system by its results. You can use metrics to do this, but you should be clear about what exactly they say. One of the measures used is accuracy.
What is Accuracy in Machine Learning?
In machine learning, accuracy is defined as the proportion of correct predictions in all predictions made. This seems to be sufficient as a measure of the performance of a machine learning system, which, however, turns out to be incomplete on closer inspection.
Consider an example of a system for detecting bank robbers on images from a surveillance camera. The percentage of robbers is very small, so the accuracy is very high if all bank customers are simply judged, to be honest. The system is then as trivial as it is useless. Without any performance, it allows every person access and, of course, every bank robber.
What is Precision in Machine Learning?
As a measure, Precision is defined as the proportion of correct positive predictions of all cases classified as positive. In our example, this is the percentage of real bank robbers in relation to all bank visitors rated as robbers.
This measure is suitable for capturing the costs of false-positive assessments. If, for example, every visitor classified as a bank robber were automatically targeted, cases that were judged to be false positively would have very high costs. A low precision means that many people who are considered to be bank robbers are not at all.
How are these metrics related?
Accuracy is a good measure if the cases to be distinguished all belong to groups of approximately the same size. In the example with the bank visitors, this is not the case. The effects of an error should also be taken into account because these are often by no means comparably negative. Rechecking an honest bank customer is much less serious than granting access to a bank robber.
Precision measures the effort to treat cases that are wrong as positive. Depending on the application, this can be a second check or something incomparably more serious.
Conclusion
The Accuracy and Precision metrics are not very meaningful on their own. Which dimension is relevant depends crucially on the respective application.
