ABSTRACT
While the trend in machine learning has tended towards more complex hypothesis spaces, it is not clear that this extra complexity is always necessary or helpful for many domains. In particular, models and their predictions are often made easier to understand by adding interpretability constraints. These constraints shrink the hypothesis space; that is, they make the model simpler. Statistical learning theory suggests that generalization may be improved as a result as well. However, adding extra constraints can make optimization (exponentially) harder. For instance it is much easier in practice to create an accurate neural network than an accurate and sparse decision tree. We address the following question: Can we show that a simple-but-accurate machine learning model might exist for our problem, before actually finding it? If the answer is promising, it would then be worthwhile to solve the harder constrained optimization problem to find such a model. In this talk, I present an easy calculation to check for the possibility of a simpler model. This calculation indicates that simpler-but-accurate models do exist in practice more often than you might think. I then briefly overview several new methods for interpretable machine learning. These methods are for (i) sparse optimal decision trees, (ii) sparse linear integer models (also called medical scoring systems), and (iii) interpretable case-based reasoning in deep neural networks for computer vision.
- Do Simpler Models Exist and How Can We Find Them?
Recommendations
On the Existence of Simpler Machine Learning Models
FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and TransparencyIt is almost always easier to find an accurate-but-complex model than an accurate-yet-simple model. Finding optimal, sparse, accurate models of various forms (linear models with integer coefficients, decision sets, rule lists, decision trees) is ...
Can AI Models Capture Natural Language Argumentation?
Formal AI models of argumentation define arguments as reasons that support claims which may be beliefs, decisions, actions, etc.. Such arguments may be attacked by other arguments. The main issue is then to identify the accepted ones. Several semantics ...
Simpler is better: Lifting interpretability-performance trade-off via automated feature engineering
AbstractMachine learning has proved to generate useful predictive models that can and should support decision makers in many areas. The availability of tools for AutoML makes it possible to quickly create an effective but complex predictive ...
Highlights- Complex models are not always better.
- SAFE extracts interpretable features from ...
Comments