Understanding And Interpreting Gain And Lift Charts

Lift and Gain Charts are a useful way of visualizing how good a predictive model is. In SPSS, a typical gain chart appears as follows:



In today's post, we will attempt to understand the logic behind generating a gain chart and then discuss how gain and lift charts are interpreted.


To do this, we will use the example of a direct mailing company. Let us assume that based on experience, the company knows that the average response rate on its direct mail campaigns is 10%. Let us further make the following assumptions:


* Cost per ad mailed = $1
* Return per response = $50


Additionally, let us assume that the company mails out ads in lots of 10,000. Based on these assumptions, if the company mails out 100,000 ads, a table summarizing the results it would obtain from this campaign is provided below:



Now let us assume that the company uses SPSS Modeler to develop a predictive model using data from previous campaigns. "Response / No Response" is identified as the "target" field and various demographic, socio-economic and behavioral variables are used as predictors. As a result of the predictive model, the company is able to sort its entire prospect list in decreasing order of expected sales. Consequently, rather than mailing out its ads to a random bunch of 10,000 prospects, the company mails out its ads to the "most likely" 10,000 first, followed by the next 10,000 and so on. Following this method, the company generates the following results table:



As can be seen, results from the second table are significantly better than those indicated in the first table. If the company were interested in maximizing returns, it would obtain a return of $400,000 by merely spending $50,000 using the predictive model whereas previously it would need to have spent $80,000! From a profit standpoint, in order to obtain a minimum profit of $360,000 while previously the company would have had to spend $90,000, using the predictive model, the company would only need to spend $60,000!


By comparing the response rate of the two campaigns, we obtain the following results:



By plotting this table on a line chart, we obtain the following result:



As can be seen from the chart above, the gap between the green line (response after predictive model) and the red line (response before predictive model) indicates the gains that the company sees from using the predictive model to prioritize its mailing list in decreasing order of expected sales rather than merely mailing out ads in a random order.


So how are gain and lift calculated? Let us define both these terms to find out:


* Gain = (Expected Response Using Predictive Model) / (Expected Response From Random Mailing)


* Lift = (Expected Response In A Specific Lot Of 10,000 Prospects Using Predictive Model) / (Expected Response In A Random Lot Of 10,000 Prospects Without Using Predictive Model)


For the data used in this example, Gain and Lift is as follows:




Since we use SPSS to classify the prospect list in decreasing order of expected sales, the Gain and Lift chart is downward sloping as can be seen below:



To summarize, Gain and Lift Charts help answer the following questions:


* How good is the predictive model?


* How does the response rate of a targeted selection compare to a random selection? Gain > 1 means the results from the predictive model are better than random.


As always, comments and feedback are welcome.