Introduction:
Cumulative Accuracy Profile(CAP) curve is used to evaluate the performance accuracy of Classification Algorithms. It determines the cumulative number for required property(in y-axis) across the corresponding cumulative number for total population(in x-axis).
Characteristics:
- Area between the curve and line is proportional to Accuracy of model.
- CAP model will enhance the response rate.
Accuracy Ratio:
Accuracy Ratio can be determined by taking ratio between Area under perfect model and Area under good model. AR varies from 0 to 1. Closer to 1, better the model.
The cumulative value in y-axis corresponding to 50% in x-axis determines number of positive elements selected from half of the populations.
Random model: In worst case scenario, the model can only predict some true positive after the whole population has been analyzed.
Perfect model: The hypothetical model that predicts the subset in which all selected elements fall under the positive property.
Good model: Any model between the random and perfect model is considered as good model.
Example:
Random Model: Consider a clothing store sending offer advertisement for Christmas to 500 customers randomly through email. On which only 100 customers purchased clothes from Christmas offer sale.
Perfect Model: Therefore for New Year’s sale, by analyzing historical data to better target customers more accurately, they decide to send advertising to that specific 100 customers and find 100% positive response sales.
Machine Intelligence is the last invention that humanity will ever need to make.
– Nick Bostrom
