«

»

Apr 29

Forecasting Software Comparison

Introduction

At a client I was asked to help with selecting software for a sales forecasting project. The most important factors were the accuracy of the forecasts, how well it integrated into the technical environment and the level of support for the software.  We wanted to forecast sales throughout the sales hierarchy so the final result would include hundreds of predictive models.  The project involved only a small amount of data so it was unnecessary to worry about whether the software could scale. The main packages we looked at were SAS programming system, SAS Forecast Studio for Desktop, R from Revolution Analytics, Autobox and SQL Server Data Mining.  Here’s how they stacked up.

SAS Programming Language

SAS is the gold-standard of corporate statistical software.  It is developed by the SAS Institute which is the largest market-share holder for analytical software.  The forecasting methods are powerful and have extensive documentation on their use.  The software is guaranteed by the vendor and has an excellent product support group.

The strength of SAS comes at a price which makes it the one of the highest cost options among the candidates.

Advantages

  • Complete statistical computation platform
  • Full featured forecasting tools
  • Can completely automate forecasting process
  • Large community knowledge of forecasting techniques in SAS programming language
  • Long history of use
  • Technical support from large and well-established vendor

Disadvantages

  • High cost
  • More technical and statistical expertise required to implement
  • Requires custom programming to compare multiple forecasting algorithms

SAS Forecast Studio Desktop

SAS Forecast Studio for Desktop is a GUI driven forecasting tool.  It is built on top of the base of the SAS programming language, so the statistical foundation is solid.  The GUI makes the tool accessible to business users who don’t have statistical programming experience.  One of its critical features is that it can forecast a different model for each level of a hierarchy such as a product sales hierarchy.

Advantages

  • Compares multiple forecasting algorithms including ARIMA, exponential smoothing and unobserved components to get best model
  • Can model each member of a hierarchy individually
  • Aggregates forecast models up the hierarchy
  • Configured using simple graphical user interface
  • Business events and holidays can be integrated into forecasts
  • Provided by large and well-established vendor

Disadvantages

  • Must be run on desktop workstation / virtual desktop
  • Needs manual user intervention to generate forecasts
  • System functionality limited to forecasting only

R from Revolution Analytics

R is an open source statistical system which is used heavily in the academic and research communities.  Many of the statistical features have been created by top academic researchers.  The forecasting methods in R are powerful and flexible.

Since R is open source, there is no support or warranty for the product by itselft.  Revolution Analytics, however, provides fee-based support for R.

Advantages

  • Complete statistical computation platform
  • Full featured forecasting tools
  • Straight-forward integration and automation
  • Low cost option
  • R is most popular and fastest growing statistical platform
  • Large community of knowledgeable users on Internet

Disadvantages

  • Open source software
  • Will need to hire third party like Revolution Analytics to support at enterprise level
  • More technical and statistical expertise required to implement
  • Requires custom programming to compare multiple forecasting algorithms

Autobox

Autobox is published by a small vendor who has been in the business of forecasting for decades.  They provide both desktop and server versions of their software.  The company includes many statistical options in the software, but also is able to create forecasts automatically without any configuration by the user.

Advantages

  • Unique forecasting algorithm takes many factors into consideration including outliers and level changes
  • Business events and holidays can be integrated into forecasts
  • Models generated using graphical user interface
  • Vendor has long history of forecasting systems and will spend considerable effort helping with set up

Disadvantages

  • Adding business events and holidays takes a larger time investment
  • Tiny community of support on Internet
  • Vendor is smallest in study

Microsoft SQL Server Data Mining

Microsoft SQL Server Data Mining is a feature of the SQL Server database product.  It is already integrated into the database engine and the tools to perform predictive analytics are included with the licensing fees.  However, the forecasting tools have limited functionality and produce models which are obviously incorrect.

Advantages

  • Automation is straight-forward

Disadvantages

  • Forecasting features are limited and flawed

Conclusion

We ended up selecting SAS Forecast Studio for Desktop because of it’s ability to automatically generate a large number of forecast models.  It also automates the selection of models from a variety of algorithms which produces more accurate forecasts.  Finally, it was able to integrate into the client’s current Business Intelligence environment with a minimum amount of hassle.