Demand Planners – what do you see when you look at this graph?
NBC 5 Chicago aired this on the 10:00 news on April 18th, 2023. They didn’t discuss it or explain what it’s graphing but they showed it in context of a story about rising hate speech, and particularly antisemitism.
First let me say I am aghast at the increase in hate crimes and hate speech in this country. No one should be vilified for who they are, where they were born, what they look like, etc. The increasing use of personal attacks in our political system are very distressing when we should be discussing issues rationally and looking for solutions.
But the point of this article is about how to interpret the data, whatever it may represent. When reviewing data such as this, one must first discern between common cause and special causes. The peaks circled in figure 2 are clearly outliers driven by special causes. They should be investigated and understood but they should not be included in calculating the trend.
Demand Planning software tools are often designed to just truncate outliers at a certain statistical border, but that is not always the best answer. If this is something that’s important to forecast correctly, then those outliers need to be investigated and understood as part of the business process. They should be excluded from the trend calculation, but they might need to be built into future forecasts as causal factors.
In addition to outliers, the data also appears to have multiple level changes. Note in Figure 3 that there is a level change after the outlier in August 2022. From August to early October, the graph shows a stable process at a higher average and variability. Fitting a trend line that includes data from before this point is misleading at best. Something significantly changed in the entire system. Data from before the level change might be useful in calculating seasonality, but never trend or level.
Beginning with mid-October, the graph shows a system that appears to be unstable. Something is significantly different than before. Or several things. This is akin to the type of instability in demand I saw at several companies during the COVID shutdowns. It is meaningless to project level or trend until the system reaches a new stability. During such a period, once can seek to understand causal factors and use them to drive forecasts based on expert judgement. The new level and trend can not be established from the data until the system reaches stability. Data from before the unstable period might still be helpful in calculating seasonality, but data during the period of instability is of little use for projecting the future.
Between the level changes and the recent instability, the trend line shown in the original graphic is absolutely incorrect. The continuing upward trend is not supported by the data. A demand planner using this kind of a trend line to generate a forecast would be in big trouble. If I had this data, I would be focused on trying to understand the causal factors of the outliers, level changes, and disruption. If forced to provide a forecast based on just this data, I would project an average of most recent data points (excluding outliers), but it would have a very low confidence level.
Key takeaways for me:
- Know the difference between common cause and special cause, investigate special causes, and determine if they should be excluded from trend and forecast calculations.
- Be alert for level changes. If there’s a level change, data prior to that point in time is of little use for predicting future trend or level (but might be useful for seasonality).
- If you see a time of extended instability, data from that period is of little use in projecting the future. Seek to understand the causal factors as possible input to future forecasts.
- When presented with graphs by news and information sources, look with a critical eye and seek to understand what the data really says.
- Hate speech is indeed up and this is very disconcerting.
What other examples do you see in news publications that present misleading graphs?