Forecasting, Pt 4

Forecasting’s ultimate goal is to accurately understand how your business will run

The chart below shows the input that a forecast has on any given business decision. As you can see it is only one of six possible inputs (although past market conditions may also be captured in the forecast). Tribal knowledge, gut feel and market conditions are also significant inputs into the decision.

For the purposes of this discussion it is important to define Tribal Knowledge and Gut Feel with regards to forecasting. Tribal Knowledge is the information known by workers of a company that is not captured in any database, qualitative or numerical format. Tribal Knowledge is best classified as the information critical to the function of any business that is simply known and un-captured.

Gut feel is the concept of “knowing” an answer. Any decision maker will hesitate on a decision that they are uncomfortable with. Humans are capable of capturing and processing information subconsciously which leads to the sense of making decisions without solid input. Most effective decision makers will “trust their gut” often enough that it will impact many critical decisions. This also happens to be one of the best error finding methods in model creation.



You will also notice the three inputs that go into the Revised Forecast Model. All three have the ability to be both good and bad. As we’ve discussed previously, past model performance is no indicator of future performance. There are many variables that could unbalance the tenuous nature of the interactions. Past performance does provide the main driver of model improvement as refinements are made to fine tune the variables actually in the model.

Past bad decisions is an important input to the model structure. Any good analyst will design their model to avoid past mistakes. The situations that led to those decisions will have specific avoidance mechanisms built-in. This is both good and bad. It’s always good to avoid past mistakes, just not at the expense of making new ones.

When modeling in past scenarios many analysts fall into the trap of over simplifying the causes of the past error which causes the model to detect the scenario in more situations than would actually be called for. To use a simple example, a modeler in the Northern US state may put in a mechanism that says rain in January will lead to frozen roads which leads to significantly reduced revenue because the model would predict fewer people driving. However, it is entirely possible to have a very mild month where rain either has no impact or positive impact (depending on the business and specific location).

On the other extreme is the analyst who overcomplicates the situation. To use the same example as above, that same modeler instead programs a situation that rain in January but only on Tuesday’s leads to frozen roads. This leads to six days being excluded from the situation. Neither situation leads to effective decision making.

The third input to the model setup is Forecaster Bias which is very similar to the Past Bad Decision input. Any modeler creating a new forecast will have their concept of what should be included or excluded. This will be modified slightly based on the input of others, but ultimately decided by the model-builder. They are in the position they are because they probably have enough experience on the topic to make good predictions of what will be necessary but they may also have blinders to other inputs.

Ultimately, a forecast model is no better than the people who create it, the time it has been used and revised over, the data input and the situations that are predicted to occur. This still leaves quite a large opportunity for any model to be wrong in many situations. For this reason it is important to reiterate that no model should have the final say in any given business decision.