Constrained regression with ordered and margin-sensitive parameters: Application in improving interpretability for regression models with prior knowledge

Abstract

Despite high accuracies, artificial intelligence models often struggle to gain business trust due to poor interpretability, resulting in lack of business adoption and operationalization. To address this challenge, Constrained Regression with Ordered and Margin-sensitive Parameters (CROMP) is introduced as a method to enhance interpretability in multi-criteria multivariate regression problems by ensuring adherence to domain-specific prior business knowledge. Unlike other techniques that treat constraint satisfaction and explainable models as post hoc concerns, CROMP embeds them directly into the training process. Specifically, CROMP enforces constraints on regression coefficients to ensure adherence to predefined business rules such as coefficient ordering, minimum margins between them, and boundary limits. These constraints are particularly critical in scenarios where domain-driven expectations about relationships between variables must be satisfied.

The effectiveness of CROMP is benchmarked against a set of widely used existing methods, demonstrating its superiority in handling complex business constraints across two different scenarios: wage modelling and marketing mix modelling. Experimental results indicate that CROMP maintains competitive accuracy while satisfying pre-defined business constraints most consistently compared to other models. The results highlight the method's robustness in different data-volume scenarios, making it suitable for practical deployment in business-critical tasks. The software implementation, datasets used, and reproducibility pipeline are made available publicly.