g a r t n e r g r o u p Integrated Logistics Strategies ILS Strategic Planning, SPA-500-075 B. Enslow Research Note July 23, 1996 REVISION Forecasting Systems: Emerging Functionality, Part 1 We outline the methodological advances that will occur in demand forecasting systems during the next five years. Core Topic Applications Cross-Industry: Demand and Deployment Planning Strategies and Systems Key Issue How will demand and deployment functionality change during the next five years? Strategic Planning Assumptions Causal forecasting for demand planning will be adopted by 75 percent of Type A (leading-edge) enterprises by 2001 (0.7 probability). Enterprises that use forecasting systems that automatically determine the correct model form will sustain a competitive advantage in demand planning through 2001 (0.7 probability). Systems robust in forecasting all major types of demand patterns will emerge by 2000 (0.8 probability). Demand forecasting systems will undergo radical change during the next five years. Enterprises that fail to keep pace will find their business models inadequate, and they will fall behind enterprises whose forecasting systems permit them to react quickly to changes in demand (see Note 1). Leading vendors will improve their demand forecasting systems methodology and functionality. Here, we summarize the critical methodological requirements for forecasting systems during the next five years. In ILS Research Note SPA-500-076, July 23, 1996, we summarize the critical functional requirements of the products. Requirement: Generate forecasts using historical data and causal data. Today, forecasting systems range widely in their ability to automatically spot seasonality, cyclical and general trends in historical data and to deal with unusual values. This ability is becoming more important as enterprises move to forecasting at the SKU level and begin using point-of-sale (POS) data. (During the next five years, enterprises will increasingly use POS data because it closes the supply-chain information loop and, by increasing data timeliness, improves forecast accuracy.) Neural networks and expert systems are being embedded in forecasting systems to address this need. Increasingly, forecasting systems must also be able to incorporate causal factors into the forecast, such as sales of related products (e.g., shampoo and conditioner), cannibalization effects and even the price of competitors products. These systems must also account for events like advertising and promotions, including lag effects (e.g., reduced sales the week after a promotion ends). Automatic Forecasting System, Comshare, Manugistics and Neil Thall Associates are among the vendors developing causal forecasting techniques. Causal forecasting for demand planning will be adopted by 75 percent of Type A enterprises (aggressive adopters of technology) by 2001 (O.7 probability). Requirement: Automatically determine the optimal model form. The preset or "pick best" models found in many forecasting systems today (e.g., Think Systems and LPA Software) produce suboptimal forecasts. To produce more-accurate forecasts, the system needs to automatically tailor the model to the problem at hand, including selecting the best lead and lag structures for each input series and the best weightings. It needs to correct for omitted variables (e.g., holidays or price changes that have affected the historical data, but that the system has no knowledge of) by identifying pulses, seasonal pulses, level shifts and local time trends, and then adding the needed structure through surrogate variables. Conversely, it needs to eliminate unneeded structure (e.g., a statistically unimportant causal variable) to keep the model manageable. It should perform all these functions as part of its normal routine without human intervention. It should also report the statistical tests used to determine the model parameters, and let users manipulate the coefficients and model structure if they want. Manugistics and Automatic Forecasting Systems are working on such methods. Enterprises that use forecasting systems that automatically determine the correct model form will sustain a competitive advantage in demand planning through 2001 (0.7 probability). Note 1 The Goals of Forecasting Forecasting s role is to predict how much product will be needed, when and where. Forecasts are typically divided into three time frames long range, intermediate and short range. Long Range: Long-range forecasts (e.g., more than two years) are used to decide whether to enter new markets, develop new products or services, expand or create new facilities (including plants and warehouses), or arrange long-term procurement contracts. Intermediate: Intermediate forecasts (e.g., three months to two years) are used by: 1) finance, for budgetary planning and cost control; 2) marketing, for new product planning and sales force compensation plans; 3) operations, for facility planning, capacity planning and process selection; and 4) logistics, for warehouse and distribution planning (e.g., transportation contracts). Short Range: Short-range or operational forecasts (e.g., fewer than three months) are used to make continual decisions about planning, scheduling, inventory and staffing in an enterprise s production, procurement and logistics activities. These decisions usually involve scheduling shipments and material flow through an enterprise s facilities and on to the customer. Requirement: Address special forecasting needs. Depending on the enterprise, forecasting applications may need to predict product and market life cycles, spare-part demand, product returns, intermittent demand, seasonal products and new products. For example, a leading system will let users forecast demand for a new product by selecting sales histories for several similar products and combining them to create the forecast for the new product (e.g., the new product may be given the geographical demand pattern of one historical product and the seasonality of another). Systems robust in forecasting all major types of demand patterns will emerge by 2000 (0.8 probability). Bottom Line: During the next five years, business demands for accuracy and customer responsiveness will mandate new requirements for forecasting systems. Increasingly, enterprises must: 1) more accurately forecast at the point of demand, 2) incorporate causal factors like promotions and price into the forecast methodology, and 3) handle specialized demand patterns. Incorporate these requirements when evaluating forecasting products, and analyze the business value that improved forecasting capabilities will have on sales, marketing, manufacturing and logistics activities. Copyright 1996 ILS: SPA-500-075 July 23, 1996 Integrated Logistics Strategies ILS Strategic Planning, SPA-500-076 B. Enslow Research Note July 23, 1996 REVISION Forecasting Systems: Emerging Functionality, Part 2 We outline the functionality that will be emerging in demand forecasting systems during the next five years. Core Topic Applications: Cross-Industry: Demand and Deployment Planning Strategies and Systems Key Issue How will demand and deployment functionality change during the next five years? Strategic Planning Assumptions All major demand planning systems will integrate with multidimensional databases or have them embedded by the end of 1998 (0.8 probability). True enterprisewide forecasting systems will emerge in 1999 (0.7 probability). Data handling and automation features rather than forecast accuracy will be the key differentiators for replenishment planning applications through 2000 (0.7 probability). The simulation of demand will become a standard category management activity for Type A enterprises (aggressive adopters of technology) by 2000 (0.8 probability). Mass customization, vendor-managed inventory (VMI) and other business initiatives are pushing many enterprises demand forecasting systems to the breaking point. Leading enterprises are evaluating products that not only alleviate this, but that also let the demand plan be leveraged throughout the enterprise. Here, we summarize the critical functionality requirements for forecasting systems during the next five years. In "Forecasting Systems" Emerging Functionality, Part 1, we summarize the critical methodological requirements. Requirement: Multidimensional capabilities. As enterprises move to forecasting at the SKU level, handling data becomes a challenge. To optimize speed and performance, demand forecasting at the hundreds-of-thousands-of-SKUs level should be done using online analytical processing (OLAP) with data stored in a multidimensional database, not a relational database. This will enable the system to arrange information in multiple levels or definitions (e.g., by product line, SKU, customer, geography and time) so users throughout the enterprise can slice and dice the forecast to make effective plans. Many traditional forecasting systems (e.g., Manugistics and American Software) still rely on relational databases. Products from Think Systems and Comshare work off multidimensional databases. All major demand planning systems will integrate with multidimensional databases or have them embedded by the end of 1998 (0.8 probability). Note 1 Change Propagation Automatic top-down, bottom-up and middle-out change propagation lets users make adjustments to a forecast at any level of the database. For instance, a forecast can be developed at the pack level (e.g., 16 ounces of diet soda) and be aggregated to total soda sales and disaggregated to all component SKUs (e.g., open stock and floor displays). Similarly, top management can mandate, say, a 10 percent sales increase for Sales Region C and the changes will be propagated accordingly. Requirement: Useful for multiple levels of the enterprise. As enterprises move to cross-functional forecasting teams and single-number forecasts, forecasting packages must be flexible enough to meet the needs of everyone from the regional sales manager to the VMI planner to the procurement manager. Among the necessary features: 1) automatic top-down, bottom-up and middle-out change propagation (see Note 1); 2) "bookmarks" for customizing views; 3) intuitive user interfaces; and 4) easy-to-use tools for forecast overrides and clear documentation of overrides. Systems from American Software, Comshare, Manugistics and Think Systems are useful for pieces of the enterprise, but no vendor has a product that enables people throughout the enterprise. True enterprisewide forecasting systems will emerge in 1999 (0.7 probability). Requirement: Is automated and includes an exceptions feature. To reduce the head count needed to create the forecast, the system must process the forecasts automatically, including data input and forecast generation. However, the system must produce alerts or exception reports when odd-looking data (e.g., outliers) are encountered, and it must allow users to manipulate the forecast models when desired. Automation is becoming increasingly important as enterprises adopt VMI and want to do weekly SKU forecasting at the customer level. Data handling and automation features rather than forecast accuracy will be the key differentiators for replenishment planning applications through 2000 (0.7 probability). Requirement: Contain advanced simulation capabilities. Simulation lets users store forecasts and do "what if?" analysis to determine, say, how a price change will affect the flow of goods through a distribution center. Advanced simulation lets a vice-president of sales use drag-and-drop technology to add a promotion and then graphically see the effect on the sales forecast at the national, regional and local levels. Comshare, Manugistics and Think Systems are refining their products simulation capabilities, though the methods of determining incremental lift remain relatively crude. The simulation of demand will become a standard category management activity for Type A enterprises (aggressive adopters of technology) by 2000 (0.8 probability). Bottom Line: During the next five years, advances in forecasting products will enable multiple parties (e.g., sales, marketing, manufacturing and logistics) to enhance, manipulate and use the forecast. Clients facing pressure from their customers to implement VMI and do more-effective category management should pay close attention to the emerging functional enhancements in demand forecasting systems.