Predicting which leads become customers
I modeled 4,612 leads for an online education company, and found that good overall accuracy hid a weak spot in the prediction that mattered most.
The problem
ExtraaLearn, the online education company in this case study, wanted to know which leads were likely to become paying customers, and what drives that decision.
The approach
I explored a dataset of 4,612 leads and 15 attributes, then built decision tree and random forest classifiers and looked at whether pruning was needed.
The result
Overall accuracy looked good, but the model found only about half of the leads who converted in training (recall of 0.50), and 41% in testing. Reporting that plainly mattered more than the headline accuracy, because converted leads are the ones the business cares about.
In this data, the leads who converted were mostly professionals in their late 40s who used the web app and communicated by phone.
What I’d change
I’d resample the data, give more weight to the customer class, and fine-tune the model.