Forecasting, Pt 4

Forecasting’s ultimate goal is to accurately understand how your business will run

The chart below shows the input that a forecast has on any given business decision. As you can see it is only one of six possible inputs (although past market conditions may also be captured in the forecast). Tribal knowledge, gut feel and market conditions are also significant inputs into the decision.

For the purposes of this discussion it is important to define Tribal Knowledge and Gut Feel with regards to forecasting. Tribal Knowledge is the information known by workers of a company that is not captured in any database, qualitative or numerical format. Tribal Knowledge is best classified as the information critical to the function of any business that is simply known and un-captured.

Gut feel is the concept of “knowing” an answer. Any decision maker will hesitate on a decision that they are uncomfortable with. Humans are capable of capturing and processing information subconsciously which leads to the sense of making decisions without solid input. Most effective decision makers will “trust their gut” often enough that it will impact many critical decisions. This also happens to be one of the best error finding methods in model creation.



You will also notice the three inputs that go into the Revised Forecast Model. All three have the ability to be both good and bad. As we’ve discussed previously, past model performance is no indicator of future performance. There are many variables that could unbalance the tenuous nature of the interactions. Past performance does provide the main driver of model improvement as refinements are made to fine tune the variables actually in the model.

Past bad decisions is an important input to the model structure. Any good analyst will design their model to avoid past mistakes. The situations that led to those decisions will have specific avoidance mechanisms built-in. This is both good and bad. It’s always good to avoid past mistakes, just not at the expense of making new ones.

When modeling in past scenarios many analysts fall into the trap of over simplifying the causes of the past error which causes the model to detect the scenario in more situations than would actually be called for. To use a simple example, a modeler in the Northern US state may put in a mechanism that says rain in January will lead to frozen roads which leads to significantly reduced revenue because the model would predict fewer people driving. However, it is entirely possible to have a very mild month where rain either has no impact or positive impact (depending on the business and specific location).

On the other extreme is the analyst who overcomplicates the situation. To use the same example as above, that same modeler instead programs a situation that rain in January but only on Tuesday’s leads to frozen roads. This leads to six days being excluded from the situation. Neither situation leads to effective decision making.

The third input to the model setup is Forecaster Bias which is very similar to the Past Bad Decision input. Any modeler creating a new forecast will have their concept of what should be included or excluded. This will be modified slightly based on the input of others, but ultimately decided by the model-builder. They are in the position they are because they probably have enough experience on the topic to make good predictions of what will be necessary but they may also have blinders to other inputs.

Ultimately, a forecast model is no better than the people who create it, the time it has been used and revised over, the data input and the situations that are predicted to occur. This still leaves quite a large opportunity for any model to be wrong in many situations. For this reason it is important to reiterate that no model should have the final say in any given business decision.

Forecasting, Pt 3

Iterate regularly to test for variable sensitivity

Every model has the ability to be stress tested. Test them regularly to see how the result changes based on varying conditions. Test for sales dropping 10%. What if prices drop 5% and sales rise 10%? What if your entire logistics department is out sick for the month of October?

To go back to the role of forecasts in the economic downturn, most of the models used in financial houses had no capability to test for the possibility of an across the board housing price drop. Their inputs would not allow them to enter a negative number. It’s possible that those institutions wouldn’t be in the situation that they find themselves in if they could test the sensitivity of their outputs based on negative home price increases.

If your forecast model doesn’t allow you to test for a given condition change that could occur, it is time for a revision. Never let yourself be held hostage by the constraints of your overgrown spreadsheet. It is up to you to make decisions based on the criteria you feel you need to make them against.

A common question is “how do you test for sensitivity?” It surprises everyone who asks how simple it actually is: draw up five scenarios that could happen over the next forecast period (typically this will be high variable growth, medium variable growth, no change, medium variable decline and high variable decline). Use your imagination because it changes from business to business and period to period.

Brainstorm how those five scenarios would make the numbers look. Some data points will go up, others down. Use your best estimation abilities; accuracy is not vital since the scenario didn’t actually happen. Put the revised data into the model and see what the outcome is. Compare all five scenarios against what actually happened. You now have a possible range of outcomes that could have occurred.

This gives you a greater ability to understand the impact of shifting market conditions. If you believe that there is a 5% chance of high growth 15% chance of medium growth, 40% chance of no change and 40% chance of medium decline you can apply those probabilities to the outcomes to get a blended decision. You are essentially hedging against the possibilities that are likely to occur. You will be a better decision maker for this process.

Forecasting, Pt 2

Never, ever, over-trust the forecast output

Models are unintelligent. They have no means of deciding to leave out unnecessary information or retrieving information that they actually need. Unfortunately, there is a tendency to trust the outcome of previously successful models. The thought process goes: “It was right before, it will be right now.” Until suddenly it’s not; a market condition has changed leading to massive failure.

Flawed models are at the heart of the recent economic downturn. Every financial and rating institution was using the same flawed models. For more than five years the models worked with enormous success. Analysts would put in their data points, the model would return the likely outcome and the analyst would execute against that decision. A highly effective system…until it broke.

Once a model is discovered to be flawed, analysts often dive for cover under the excuse of “the model told me to.” This takes us back to the concept of trusting a model’s output. Don’t. Do not ever put too much faith in the outcome of a forecast (moving average, Monte Carlo, or anywhere in between).

Forecasting models are only tools in the digital toolbox of decision makers. The moment that they become elevated to a higher status is the moment that they should lose all credibility. Decision makers should be held responsible for their actions no matter their process for reaching the decision. Anything less leads to abuse of the system.

10 Simple Principles to Pass Down

In the recent past it has become clear that the world needs a new set of fundamental commandments if stability is to be achieved. Unfortunately change will only come when enough public support comes together with a single voice. These 10 rules are simple value statements that can create a framework for all decisions.

1. Always seek self-improvement
2. Be modest in all claims
3. Know your performance over time
4. Consider the non-economic impacts of all decisions
5. Participate in your communities
6. Do not select the least efficient option
7. Do not move backward from any improvement
8. Do not claim the improvements of others as your own
9. Seek to improve the areas around you
10. Give the returns of your improvements back to your communities

Forecasting, Pt 1

Forecasts are no more than a cog in a Prediction Process

Any process that ends in a decision involves forecasting the future. No decisions can be made that do not account for possible future actions. At their most basic level, forecasts are simply predictions of what will occur in the future.

Every C-level executive has a job that almost exclusively involves making numerous business decisions. Sometimes they use crude tools such as gut feel or personal preference and sometimes they use highly sophisticated, probabilistic models involving multitudes of disparate variables. The sophistication of a model does not necessarily make it better than its lower level brethren.

Moving Average forecasts are still around for a reason – they work. In environments not prone to sudden, dramatic changes a moving average forecast holds a lot of appeal: it adjusts fairly quickly to new conditions and it is easy to understand/fix. It will never predict future market condition changes but that’s understood when it is implemented.

Sophisticated Monte Carlo simulations that account for internal and external pressures are more likely to predict future condition changes. The issue with using this style model is that it is dependent on the variables it makes decisions on. Forecaster bias is a significant risk with complex forecasts. Forecaster bias is the condition where the forecaster includes only the variable he/she believes will impact the outcome. There is no way to include every possible variable and someone has to make the selection.

The Greening of America

It continues to surprise me how far the green movement has come in even just the past three years. At the same time I'm surprised at how little traction has been gained on the everyday side of green. When you look in any popular magazine or newspaper there is always some new story of a company going "carbon neutral" or completing a high profile LEED project.

Where are the stories about warehouses doing mass transitions away from metal halide lighting? Where are the stories of waterless urinal/low-flush toilets flying off the shelves? Where are the stories highlighting the benefits of simple LED light bulbs for exterior use? Why is there no attention on the small stuff?

Doing the right thing isn't always sexy or easy. I get that but in this case it should be. At the very least companies should feel ashamed to be using energy inefficient bulbs. Why hasn't GE or Sylvania brought out the campaign for energy efficient lighting? Where are the press releases highlighting the cost of not switching?

I follow the environmental industries not closely but regularly. It seems that recently all the focus has been on the macro-industries: energy, fuel, cars. What I want to see are the studies highlighting the little things. I guarantee a national campaign to change light bulbs will do more in the next 2 years than any amount of fuel efficiency will do over the next 5.

It's About the Experience, Not the Technology

Microsoft doesn't quite understand why Google is successful in search. Do you think the average surfer even knows how Google determines what order pages appear on your screen when you run a search? I can tell you right now that they don't care and they don't really want to know. So why do people choose Google over Microsoft if it isn't about the technology?

It's all about the experience. Google has a very clean homepage that is good for one thing - Search. That's it. If you want to do anything else with them you have to go looking for it. Live Search by Microsoft is moving in the right direction but still doesn't quite get there. The trick is to make it clear what users should use your site for - or your business - for.

How does this tie into your business? Keep your pitch simple. Don't offer too many choices, don't drop a menu on their lap and tell them to pick. Tell them what they need. Show them how you will help them. It's an easy to duplicate formula at the end of the day.

The reason Google is successful is that they were able to take their one product offering - Search - and leverage it into a profitable advertising company. They now make money off of nearly every search. If you are a marketing company, remember to first assist companies with their marketing. If you are a distribution center, remember that distribution is the only end result that matters to your customers. Make it fast, make it effective, make it run well but always keep in mind your one singular core priority.

There's a Problem With Consulting

Consulting has become a game that most everyone plays the same. Most consulting firms are set up in the same fashion, hire the same type of people and go after the same types of clients. Really, is one Harvard MBA different than another? Does having 15 of them as Senior Managers make you a better, smarter, more competitive company? Once every company has adopted the same model it won't be long until they can only compete on price. After you become a commodity it's a long, hard road.

This is the trap that most industries fall into after a certain period of time. The best way to categorize it is "Group Think." Once everyone is trained the same way, they work on the same types of projects, then begin to fall into a certain silo of specialty and you can't expect them to actually see the world through a different shade of lens. Out of all this comes your typical consultants that spout buzzwords like it's going out of style and offer generic solution after generic solution to every problem that they see.

Any consultant that you get out of the typical top firm is going to be out of this mold. They certainly have their strengths which usually include sales, client relations, asking the right questions, creating informative presentations and spotting trouble spots in an organization. The problem becomes their weaknesses. Traditional MBA programs are designed to train a certain style of thinking and decision process. That process is typically financially driven and risk averse. It leads to the natural selection of a low risk solution that creates good net present value, internal rate of return and acceptable payback period. This is all great if all you want is to be average.

Great companies all assume some element of risk. Great solutions require people to think outside the box. Great value requires a project manager not just thinking about how to get the next sale. I won't sit here and say that we here at Integrated Strategies don't ever look for the next sale, but I will tell you why we're different. We think our next sale is in the implementation of our recommendation. We'll take more consulting work but what we're really in this business for is to implement our ideas, make them work and then make them successful. Next time you get a consultant in for a meeting ask him if his firm will personally do construction management and implementation of their plan. I'll bet that they won't.

The trick to changing consulting is getting the answer from people who will do the work after they show you their solution. Consulting firms that hand you the solution and walk away accept no responsibility for your success. Wouldn't you rather have a partner than some abstract adviser?