For many businesses, improving Financial Planning processes (including Financial Planning & Analysis, Sales & Operational Planning, Integrated Business Planning and forecasting), is one of the highest priorities. Automation, improving the integration between systems and increasing process efficiency seems to be an ongoing endeavor. Nowadays, Artificial Intelligence & Machine Learning appear to offer promise in this area but is this really what your company should purchase and/or develop?
In my career, I have seen many examples of the incorrect use of the term Artificial Intelligence. Also, a lot of software is sold with the caption of AI without having any true AI functionality inside. Even where software does contain AI, most companies cannot directly start using it based on the current state of their planning processes. However, companies need to understand that the implementation and development of the most innovative technology can be realized within an acceptable time span and budget. And here lies the crux of the matter: to implement and use AI correctly, a structured approach needs to be taken. This requires time, careful preparation, technological development and in most cases, also quite a change in the way of working. It is difficult to do this all at once, so a step-by-step approach is recommended: the objective being a stable and flexible environment that delivers now but can also adapt quickly to changes in the business, environment, and related decision-making processes.
In this article, 5 different stages in planning & forecasting processes will be addressed. The focus will be on the necessary changes that are required to step up a stage in your journey to reach the end-goal of creating the most value within these processes utilizing all available resources including AI. The terms planning and forecasting will be used interchangeably.
Stage 0
As a starting point, no planning process is in place. This implies that no investment has been done in proper technology and any work is manual without any underlying foundation. Firefighting without centralized analysis is needed all the time and no clear vision about the short and long-term planning is present within the company.
Stage 1
In this stage, a planning process is in place, however it is not high-value or well thought through. The regular planning activities such as reforecasting are basic & relatively naïve, (for example, just extrapolating historic values from previous periods), but at least a baseline is available as a starting point. Since the accuracy of this forecast is likely to be low, lots of effort from planners will be needed to adjust the ‘baseline’ for the final submission. The time spent on firefighting should be lower than in stage 0. No investment in proper technology needed, since the process will still be extremely outcome focused and can be biased to the subjective mindset of the different planners.
Stage 2
For most companies, this is the likely maturity of planning & forecasting processes. In this stage, a company is likely to have disparate systems and lots of data. Not having a well-constructed process in place to automate & integrate, is the main cause of the inefficiencies. However, the big plus in this stage is the availability of more sophisticated forecasting methods; Baselines, trends, irregular fluctuations, cyclicity or seasonality: the available statistical forecast methods will be able to detect the patterns in the data and can be used by the planners. When you are using less advanced (intuitive) statistical forecasting methods, like Moving Average, Historical Average, Regression, or any Driver-Based approach), it can be done in Excel. For the more advanced forecasting methods, like Exponential Smoothing (Single, Double, Triple), Holt’s Linear Trend, Multiplicative Decomposition and Winter’s Multiplicative, more advanced software (statistical computing programming software or tools) are needed. It is recommended to write down requirements and a game plan for the data flow, integration design and functional design of the model before deciding which software to use.
In this stage, a clear shift of labor can be seen. The amount of work for planners becomes less after the reconsolidation of data, because besides recognizing data trends and applying the correct statistical approach, planners only need to tweak the forecast based on knowledge that is not reflected in any system. However, additional time needs to be invested in long term developments of the model and forecasting process in order to increase scaling and efficiency in all functions within the company.
In this article, I will not cover the exact content of advanced forecast methods. For each method, at least three of the following components is accounted for: baseline, trend, seasonality, cyclicality, irregular fluctuations and growth.
Stage 3
In this stage, the integration of transactional systems of record is in place. This implies that a single source of truth state has been reached within the company, which has a positive impact on the daily jobs of the planners. The amount of low-value work (checking, reconciliation, error correcting) for planners becomes less due to the integrated solution. In this saved time, planners can engage in high-value work (all kinds of data analysis) instead of manual monthly repeated reconciliations.
The technology supports the basic end-to-end forecasting process fully. In contrast to stage 2, more data will be available for the different components of the forecasting process, like extensive data analysis, scenario planning, impact analysis etc. The options in terms of forecasting are no longer limited: detecting demand patterns (hierarchical, but also causal) based on an external database with statistics will be the standard. In stage 2, statistical forecasting was based on chosen actual data, but in this stage predictive analytics is used to determine which (additional) data has most predictive value for the forecast throughout the whole company. More and more data-based algorithms will be implemented, which implies even less work for planners, since also standardized extensive data analysis and reports will be created with this technology. Planners only have to tell what kind of analysis would have added value and these analyses will be developed in a standardized manner company-wide. It will take some time for the planners to feel comfortable with their ‘new’ role.
Stage 4
In this stage, the technology supports more advanced end-to-end forecasting process and includes more innovative forecasting techniques. An improvement compared to stage 3 is the ability to use different techniques to forecast on the short term (0-3 months) and the medium term (3-24 months). The capability to make the distinction between short term and medium-term planning decreases the likelihood of having too little downstream data and increases the accuracy of the long term forecast. Both granular and downstream data are in this stage available to detect also patterns in hierarchy or causal relations.
The nature of the labor in terms of the forecasting will be more and more strategical decision-making instead of focusing on the downstream data. Conventional planners will be re-trained to add value higher in the process chain. In this stage, the data will no longer be the main topic of conversation, but the assumptions of the model will be challenged and simulated in scenario planning to increase the accuracy of forecasting and have a good idea of future possibilities. The model will provide all analysis to enable the decision-making process to go smoothly.
Stage 5
This is the final state, where the full forecasting process is done automatically and lead by a centralized RPA (Robot Process Automation) system. A centralized RPA system is an Artificial Intelligence machine that is the core of all automation: everything is connected to this RPA system. This self-learning technology owns the end-to-end forecast process and can execute all steps in the process and take decisions on its own. However, it needs to be trained to improve, which implies that instead of improving a model, feedback must be given to the RPA system. The upside of this, is that you can feed the AI machine detailed information about potential improvements and the RPA system is able to detect the exact root cause of the mismatch and will adjust this accordingly. Where in stage 3 and 4, the intuition behind the forecasting methods fades away, in this stage it is completely gone: the RPA system is trusted fully on its capability and needs to be trained accurately.
With the RPA system leading the forecasting process, employees will be needed for developments and training the AI machine, but near to zero employees are still needed for checking the forecast numbers. The biggest challenge in this stage is letting the control go. And not to another capable colleague, but to an Artificial Intelligence product that is able to see more in data than all the brightest minds in the company combined are able to see (with sufficient training). As in most stages, it is possible to run the old process and new process in parallel. After running this process for a few months and comparing previous months’ forecast against the current actuals within a certain error margin, companies feel more comfortable to let RPA system control the process and only review the high-level assumptions and number from that point on. In the meantime, every low-level deviation and new data will be fed to the machine to make the forecast of the next month’s even more
A real challenge for this roadmap is going to the next step only when the company is truly ready. When going too fast, inefficiencies will become even more inefficient and will lead to an unstable environment. Do proper pilots and let current and new developments run in parallel for a part of the business. Support the business in the transformation because it will not be easy for them. Guide them in the changing responsibilities and how to manage the expectations of the conventional planners. As a company, you can only move as fast as the slowest part of your business.
Another challenge is worth addressing – letting go of control step-by-step. In this process, you cannot let an RPA system do the forecast if there is no game plan or no time invested in setting up integration, cleaning available data, insight in forecasting techniques etc. Experts in the company need to have the knowledge to be able to get the Artificial Intelligence machine ready on their level to add more value to the company.
Lastly, the way of working will absolutely need to change. Some roles will even disappear within the process and re-training employees is unavoidable. Do not underestimate the impact of this and be honest about the extent of changing procedures and what is needed to accomplish the next step in the improvement of the forecasting process.
It is never too late to start and get informed how to gain big wins in the current forecasting process within your company. Create the long-term road map with company-wide support and get started.
