A commonly used method to evaluate the accuracy

of a classification model is a graphical representation called Receiving Operator Characteristics

curve, for short ROC curve. The ROC Curve shows the quality and robustness

of the classification model as defined by the false positive rate and true positive

rate – or in other words – sensitivity and 1-specificity. Many scoring metrics measure the performance

of a classification model for only one specific value of the threshold applied to the probability

of the positive class. However, this is just one possible working

point for the classification model. The ROC curve attempts to provide a more comprehensive view of the model performances for different classification thresholds. Here you see the ROC Curve. On the x-axis you see the false positive rate and on the y-axis the true positive rate. This blue line is the ROC curve of the model

we have trained. This black line is the ROC curve of a random classifier. So, how are all those different values of the false positive rate and the true positive rate calculated? Let’s have a look. For each input event, a classification model

produces a class probability that determines the class assignment. By default, the events are assigned to the

positive class if the probability of the positive class is 0.5 or higher, and to the negative

class otherwise. If we increase this classification threshold,

fewer data points will be assigned to the positive class. If we decrease the threshold, more events

will be assigned to the positive class. Here we see a number of events ordered by

their probabilities to belong to the positive class. By default, the value of the classification

threshold is set to 0.5. And here are the corresponding assignments

to the two classes. Next, we decrease the threshold value, for

example to 0.2. Now, we see that the class assignment changes

for these data points with a probability for the positive class falling between 0.2 and 0.5. Finally, let’s increase the classification

threshold to 0.7. Now, only those events will be assigned to

the positive class, because their probability for the positive class is greater than 0.7. As you can see here, the true positive rate

and the false positive rate also change every time we change the value of the threshold. The true positive rate is calculated as the

number of correct predictions to the positive class divided by the total number of events in the positive class. The false positive

rate is calculated as the number of incorrect predictions to the positive class divided by the total number of events in the negative class. For these threshold values, let’s plot the

false positive rates on the x-axis and the true positive rates on the y-axis. This dot represents the performance of the

model for the default threshold, this dot stands for the threshold

of 0.2, and this dot for the threshold of 0.7. If we now connect these dots, we have a sketch

of the ROC curve. In reality, the ROC curve is drawn based on

many more threshold values and therefore many more pairs of these two rates. The ROC curve of a perfect classifier would

move along the axis to reach this point for the perfect threshold, lying in this corner. Here, the true positive rate is 1 and the false positive rate is 0. The closer the curve of our model is to that

point, the better the model is performing. In the case of a random classifier for a two-class

problem, all events have the same probability for the positive class independently of their

actual class values. By varying the value of the classification

threshold the ROC curve becomes this straight line. This is the worst performance your model can

produce. If the ROC curve of your model lies below

the line of the random classifier then you probably made a mistake in building the ROC curve. The ROC curve has a few advantages as a model

evaluation technique: Firstly, it allows you to visually compare

different models in terms of performance. If we have more than one model, the model

with the highest ROC curve will be the best performing one. Secondly, if the eye comparison among model

curves becomes complicated, we can always use the Area Under the Curve, for short AuC,

measure. The AuC measures this area here. It is 1 for the perfect classifier and 0.5 for the random classifier. Thirdly, the ROC curve allows us to choose

the threshold that produces the best performance. One method to select the optimal threshold

is to minimize the Euclidean distance between the curve and the point in the top left corner

indicating perfect model performance. The point in the curve that minimizes the

distance is this tangent point shown here. In this video, we have shown how an ROC curve is drawn, given the actual class values, probabilities

of the positive class, false positive rate, true positive rate, and classification thresholds. We have also shown the robustness of the ROC

curve as a method to quantify and compare model performances.

I know nothing about statistics but I got the concept. Best explanation ever about a topic.