Formula
p(x) = 1 / (1 + e−z)
where z is a linear combination of features and coefficients.
where z is a linear combination of features and coefficients.
Model Choices
- Binary logistic regression
- Regularised logistic regression using L1, L2 or Elastic Net
- Multinomial logistic regression
- Polynomial terms when justified by the data and domain knowledge
Evaluation
- Accuracy may be misleading when classes are imbalanced.
- Precision focuses on the reliability of positive predictions.
- Recall measures the proportion of actual positives detected.
- F1-score balances precision and recall.
- ROC AUC and Precision-Recall curves assess discrimination across thresholds.
- Use stratified cross-validation when class balance matters.
Tuning and Imbalance
- Adjust the decision threshold to reflect the cost of classification errors.
- Consider class weights before applying synthetic sampling methods.
- Keep validation data representative of the real population.
Interpretation
- Inspect coefficients, odds ratios, confusion matrices and SHAP values.
- Avoid drawing causal conclusions from predictive associations.