Tom Reilly

Waging a war against how to model time series vs fitting

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.

The M3 Forecasting Competition Monthly Calculations were off

Posted by on in Forecasting
  • Font size: Larger Smaller
  • Hits: 5571
  • 3 Comments
  • Subscribe to this entry
  • Print
  • PDF

The M3 Forecasting Competition Calculations were off for Monthly Data

Guess What We Uncovered ? The 2001 M3 Competition's Monthly calculations for SMAPE were off for most of the entries.  How did we find this?  We are very detailed.

 

14 off the 24 to be exact. The accuracy rate was underestimated. Some entries were completely right.  ARARMA was almost off by 2%. Theta-SM was off by almost 1%.  Theta-SM's 1 to 18 SMAPE goes from 14.66 to 15.40.   Holt and also Winter were both off by 1/2%.

 

The underlying data wasn't released for many years so this made this check impossible when this was released.  Does it change the rankings?  Of course. The 1 period out forecast and the averaged of 1 to 18 are the two that I look at.  The averaged rankings had the most disruption. Theta went from 13.85 to 13.94. It's not much of a change.

 

The three other seasonalities accuracies were correctly computed.

 

if you saw our release of Autobox for R, you would know that Autobox would place 2nd for the 1 period out forecast.  You can use our spreadsheet and the forecasts from each of the competitors and prove it yourself.

 

See Autobox's performance in the NN3 competition here.  SAS sponsored the competition, but didn't submit any forecasts.

Comments

  • Robinjack
    Robinjack Thursday, 13 June 2019

    Remarkable post! Chatting about how taken pleasure in the exact examining. I’m hoping to see significantly more from your website. It looks you might have very good knowledge combined with perception. My corporation is very much fulfilled with this particular tips. binance

  • Robinjack
    Robinjack Saturday, 15 June 2019

    It’s actually a nice and helpful piece of info. I am happy that you simply shared this useful information with us. Please keep us up to date like this. Thanks for sharing. binance

  • Robinjack
    Robinjack Sunday, 16 June 2019

    The following time I read a weblog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my option to learn, but I really thought youd have something fascinating to say. All I hear is a bunch of whining about one thing that you might fix in the event you werent too busy looking for attention. Coinnama

  • Please login first in order for you to submit comments
Go to top