Classification Metric Evaluation

formulas, interpretation, and step-by-step calculation

Overview

Please type code into your code window,
instead of copying and pasting
-this can help you understand the process better

This article Evaluation Metrics for Classification Model, uses: confusion matrix first, then metric formulas, then works with calculations, followed by support, classification report, and ROC-AUC.

The formulas apply to any binary classification.

1. Confusion Matrix Foundation

A confusion matrix compares actual values with predicted values.

	Predicted Positive	Predicted Negative
Actual Positive	TP = True Positive (actual = predited)	FN = False Negative (actual <> predited) (Type II Error)
Actual Negative	FP = False Positive (actual <> predited) (Type I Error)	TN = True Negative (actual = predited)

True Positive (TP): actual positive and predicted positive.

True Negative (TN): actual negative and predicted negative.

False Positive (FP): actual negative but predicted positive. Also called Type I Error.

False Negative (FN): actual positive but predicted negative. Also called Type II Error.

Worked Example:

of the total 1000 records that were predicted, if TP = 560, TN = 330, FP = 60 and FN = 50, then,

Total = TP + TN + FP + FN = 560 + 330 + 60 + 50 = 1000

2. Accuracy

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Percentage of correct predictions:

Accuracy = (560 + 330) / (560 + 330 + 60 + 50) = 890 / 1000 = 0.89 = 89%

3. Misclassification Rate

Misclassification Rate = (FP + FN) / (TP + TN + FP + FN)

Percentage of wrong predictions:

Misclassification Rate = (60 + 50) / 1000 = 110 / 1000 = 0.11 = 11%

4. Positive Recall / Sensitivity / True Positive Rate

Recall = TP / (TP + FN)

Out of all actual positives, how many were detected?

This metric focuses on actual positives in the denominator.

Recall = 560 / (560 + 50) = 560 / 610 = 0.9180 = 91.80%

5. Negative Recall / Specificity / True Negative Rate

Specificity = TN / (TN + FP)

Out of all actual negatives, how many were detected correctly?

Specificity = 330 / (330 + 60) = 330 / 390 = 0.8462 = 84.62%

6. Positive Precision / Positive Predictive Value

Precision = TP / (TP + FP)

Out of all predicted positives, how many were actually correct?

Precision = 560 / (560 + 60) = 560 / 620 = 0.9032 = 90.32%

7. Negative Precision / Negative Predictive Value

Negative Precision = TN / (TN + FN)

Out of all predicted negatives, how many were actually correct?

Negative Precision = 330 / (330 + 50) = 330 / 380 = 0.8684 = 86.84%

8. Precision vs Recall Tradeoff

This explicitly highlights threshold-based tradeoff:

Increasing recall usually decreases precision
Increasing precision usually decreases recall
The balance depends on the classification threshold

Further Metrics and Report Reading

9. F1 Score

F1 Score = 2PR / (P + R)

F1 is the harmonic mean of precision and recall. It is useful when precision and recall both matter, especially on imbalanced data.

Positive F1 Score: use positive precision and positive recall.

P = 0.9032 R = 0.9180 F1 = 2 x 0.9032 x 0.9180 / (0.9032 + 0.9180) = 1.6581 / 1.8212 = 0.9105 = 91.05%

Negative F1 Score: use negative precision and negative recall.

P(negative) = 0.8684 R(negative) = 0.8462 Negative F1 = 2 x 0.8684 x 0.8462 / (0.8684 + 0.8462) = 1.4697 / 1.7146 = 0.8571 = 85.71%

10. FP Rate and FN Rate

Metric	Other Names	Formula	Step Calculation
TPR	Sensitivity, Recall, Hit Rate	TP / (TP + FN)	560 / 610 = 0.9180 = 91.80%
TNR	Specificity, Selectivity	TN / (TN + FP)	330 / 390 = 0.8462 = 84.62%
FPR	Fall-out, Type I Error Rate	FP / (FP + TN)	60 / 390 = 0.1538 = 15.38%
FNR	Miss Rate, Type II Error Rate	FN / (FN + TP)	50 / 610 = 0.0820 = 8.20%

11. Support

This defines support as the number of actual occurrences of each class in the test data. The slide example shows:

Support for class 0 = 123
Support for class 1 = 153
Total support = 123 + 153 = 276

Support gives context to precision, recall, and F1-score. Very small support can make a metric unstable.

12. Classification Report

from sklearn.metrics import classification_report print(classification_report(y_test, y_pred))

Class	Precision	Recall	F1-score	Support
0	0.87	0.83	0.85	123
1	0.87	0.90	0.88	153
Accuracy	0.87			276
Macro Avg	0.87	0.87	0.87	276
Weighted Avg	0.87	0.87	0.87	276

How macro average is calculated:

Macro Precision = (0.87 + 0.87) / 2 = 0.87 Macro Recall = (0.83 + 0.90) / 2 = 0.865 â‰ˆ 0.87 Macro F1 = (0.85 + 0.88) / 2 = 0.865 â‰ˆ 0.87

How weighted average is calculated:

Weighted Precision = (0.87 x 123 + 0.87 x 153) / 276 = 0.87 Weighted Recall = (0.83 x 123 + 0.90 x 153) / 276 = (102.09 + 137.70) / 276 = 239.79 / 276 = 0.8688 â‰ˆ 0.87 Weighted F1 = (0.85 x 123 + 0.88 x 153) / 276 = (104.55 + 134.64) / 276 = 239.19 / 276 = 0.8666 â‰ˆ 0.87

13. Multiclass Confusion Matrix

For N classes, the confusion matrix becomes an N x N matrix. The diagonal cells are correct predictions, and off-diagonal cells are misclassifications.

Actual \ Predicted	Apple (0)	Mango (1)	Banana (2)
Apple (0)	50	2	3
Mango (1)	4	45	6
Banana (2)	1	5	48

Example interpretation:

2 in row 0, column 1 means Apple predicted as Mango
6 in row 1, column 2 means Mango predicted as Banana

14. ROC and AUC

This describes ROC and AUC as strong tools for binary classification evaluation, especially when plain accuracy can be misleading.

ROC curve plots: Y-axis -> TPR = TP / (TP + FN) X-axis -> FPR = FP / (FP + TN)

AUC means Area Under the Curve. Higher AUC means the model separates the two classes better across different thresholds.

Practical reading rule: use confusion matrix values to calculate the core metrics, then use ROC-AUC to judge class separation across thresholds rather than at only one cutoff.

Start from TP, TN, FP, and FN. Once the confusion matrix is clear, every major classification metric becomes a direct calculation and is easier to interpret correctly.