The question is, can PA models adapt in an effective and cost-efficient manner? Simply retraining the PA models with fresher facts will not be good enough if the factors leading towards an event have changed. If the models cannot adapt, it is no different than a doctor prescribing treatments based on outdated knowledge.
We can ease this problem the same way we’ve eased other problems where the complexity is such that we need to formally analyze the situation. That is, building OLAP cubes to easily analyze prediction performance patterns. We can use these cubes to:
Optimize the PA models by isolating the subsets of circumstances in which it falters. Analyze, validate, and optimize PA models.
Monitor prediction performance over time, Manage and Monitor the performance of the PA models in a Performance Management style.
Act as a data source to provide PA results to the end user in a robust manner. Ex: Present arrays of ranked possibilities instead of just one answer.
Over the years I’ve implemented a number of PA applications that go beyond the simple target-marketing or “if you like this, you’ll like that” scenarios. In complex PA scenarios where PA is being applied to a complex system or the players involved in what is being predicted are actually trying to undermine your predictions (ex: credit card fraud), the sophistication of the PA must be taken up a notch or two.
Figure 1 – An update of an old slide showing for human and machine intelligence can meet in the middle with PA.