Tom Reilly

Waging a war against how to model time series vs fitting

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that has been used in the blog.
  • Bloggers
    Bloggers Search for your favorite blogger from this site.

Machine Learning - It might be "machiney", but it's not learning

Posted by on in Forecasting
  • Font size: Larger Smaller
  • Hits: 6823
  • 55 Comments
  • Subscribe to this entry
  • Print
  • PDF

Let's take a look at Microsoft's Azure platform where they offer machine learning. I am not real impressed. Well, I should state that it's not really a Microsoft product as they are just using an R package. There is no learning here with the models being actually built. It is fitting and not intelligent modeling. Not machine learning.

The assumptions when you do any kind of modeling/forecasting is that the residuals are random with a constant mean and variance.  Many aren't aware of this unless you have taken a course in time series.

Azure is using the R package auto.arima to do it's forecasting. Auto.arima doesn't look for outliers or level shifts or changes in trend, seasonality, parameters or variance.

Here is the monthly data used. 3.479,3.68,3.832,3.941,3.797,3.586,3.508,3.731,3.915,3.844,3.634,3.549,3.557,3.785,3.782,3.601,3.544,3.556,3.65,3.709,3.682,3.511, 3.429,3.51,3.523,3.525,3.626,3.695,3.711,3.711,3.693,3.571,3.509

It is important to note that when presenting examples many will choose a "good example" so that the results can show off a good product.  This data set is "safe" as it is on the easier side to model/forecast, but we need to delve into the details that distinguish the difference between real "machine learning" vs. fitting approaches.  It's important to note that the data looks like it has been scaled down from a large multiple.  Alternatively, if the data isn't scaled and really is 3 digits out then you also are looking for extreme accuracy in your forecast.  The point I am going to make now is that there is a small difference in the actual forecasts, but the level(lower) that Autobox delivers makes more sense and that it delivers residuals that are more random.  The important term here is "is it robust?" and that is what Box-Jenkins stressed and coined the term "robustness".

Here is the model when running this using auto.arima.  It's not too different than Autobox's except one major item which we will discuss.

The residuals from the model are not random.  This is a "red flag". They clearly show the first half of the data above 0 and the second half below zero signaling a "level shift" that is missing in the model.

Now, you could argue that there is an outlier R package with some buzz about it called "tsoutliers" that might help.  If you run this using tsoutliers,  a SPURIOUS Temporary Change(TC) up (for a bit and then back to the same level is identified at period #4 and another bad outlier at period #13 (AO). It doesn't identify the level shift down and made 2 bad calls so that is "0 for 3". Periods 22 to 33 are at a new level, which is lower. Small but significant. I wonder if MSFT chose not to test use the tsoutliers package here.

 

Autobox's model is just about the same, but there is a level shift down beginning at period 11 of a magnitude of .107.

Y(T) =  3.7258                                azure                                                                     
       +[X1(T)][(-  .107)]                              :LEVEL SHIFT       1/ 11    11
      +     [(1-  .864B** 1+  .728B** 2)]**-1  [A(T)]

Here are both forecasts.  That gap between green and red is what you pay for.



Note that the Autobox upper confidence limits are much lower in level.

 

Autobox's residuals are random

 

 

 

 

 

Comments

  • Bhaskar
    Bhaskar Monday, 19 September 2016

    Hey Tom, Great Post!!

    So whats your take in Machine Learning in coming 5 years? You fond of Azure ML of Microsoft?? In which sectors can ML spear into other than E-Commerce?

  • Ming Lin
    Ming Lin Sunday, 24 June 2018

    hvit kjole dame mot den onde overvekt.q.gs/551551/mysqueezeq.gs/551551/mysqueeze
    Den følelsesmessige Toll på medisin,hvit kjole dame,sk uaktsomhet ved Ben forlate balkongene må i råd | Publisert 2012-08-30 15:41:25 | 46 leser | Uklassifiserte Sammendrag mens de fysiske effektene av medisinsk uaktsomhet fei,strikkeoppskrifter genser dame,lene er åpenbare, noe som resulterer i nedsatt funksjonsevne, disfiguration eller selv, i verste tilfeller, døden, er det ofte slik at den følelsesmessige effekter er ikke sett eller ignorert både off,Billig,eret og de nærmeste til dem kan ende opp som lider utålelig følelser al lider på grunn av feil ved helse-leverandører som for eksempel leger, sykepleiere, kirurger og terapeuter
    mörkblå jeans dam nte använda benet för långa promenader och vikt lagret eftersom detta skulle påverka läkningen av f,mörkblå jeans dam,oten. Fortsätta resten som krävs för att slutföra indrivning för att få rid av skadan. Mest troligt, kommer din läkare utfärda ant,collegetröja dam vit,ingen en konservativ behandling o kirurgi beroende på resultatet av röntgen. Allt du behöver göra är att följa läkarens råd och du kommer att ha ett h,vita dunjackor dam,ögre chans att få tillbaka på rätt spår. Försök inte ändra några av de metoder som detta skulle göra mer skada än nytta.Jag skriver för TIR-,figursydd kappa dam,Massage sten om att välja rätt basalt stenar och massage stenar för en massage med varma stenar.Jag skriver om basalt stenar & massage stenar fö

  • Ming Lin
    Ming Lin Tuesday, 19 June 2018

    skleppl ólnie jeśli nie masz wiedzy medycznej, jak również doświadczenie. Niech a Professional obsłużyło sprawa gdyż ma większe szanse powodzenia, jak również w zapobieganiu dalszym stratom w tym samym czasi,skleppl,e.piszę dla TIR, Masaż kamieniami o wyborze właściwego bazaltowe kamienie i masaż kamieniami dla hot stone massage.piszę o bazaltowe kamienie & masaż kamieniami dla TirMassageStone.com ,kurtki puchowe damskie z kapturem,
    Jaka jest Lordosis przez Terry małych   w porady | opublikowane 2012-11-26 20:18:21 | 27 czyta | ,spódnice damskie duże rozmiary,Unrated Lordosis Streszczenie jest bardzo zbliżonym do problemów scoliosis, jednak kręgosłup nie wychodzi "S" lub "C" kształt krzywej kręgosłupa są lekko zagięte do wewnątrz w tego rodzaju stan, który

  • Ming Lin
    Ming Lin Sunday, 10 June 2018

    Women's �?nutzen nicht zu kurz kommen. Im Moment ist Photo voltaic macht lediglich Getti ng von Menschen, die informiert und sensible Ökosystem der Erde betreffend sind ausgewählt, während es in der Tat ein,Women's, Problem ist, dass jeder besorgt werden müssen. Es gibt viele großartige Belohnungen, Solarstrom für Ihr Unternehmen und ihre Residenz zu wählen. Im folgenden werden ein paar. Auswahl eine Solaranl,Damen Schuhe,age nutzen sparen Sie Hunderte oder auch Tausende von Dollar jährlich. Sie können gefährliche Greenhouse Gas Schadstoffe vermindern. Solarstrom ist viel besser für die Umgebung im Vergleich zu foss,daunenjacke mit echtpelz,ilen Brennstoffen. Solarenergie ist eine nachwachsenden Ressource, die nie enden kann. In Fällen wo man Energie mehr machen als Sie nutzen, würde sie Ihren Stromanbieter bei Ihnen kaufen. Foto voltaic Power wird nicht beeinflusst von An

  • Bhaskar
    Bhaskar Monday, 19 September 2016

    not fond*:p

  • Administrator
    Administrator Monday, 19 September 2016

    Hi Bhaskar, No, not fond. Do you understand the differences between the model built by Autobox and the Azure/R model? If you understand time series analysis and forecasting the BLOG post is trying to clearly discuss how weak the analysis is being done by the free software.

  • Bhaskar
    Bhaskar Tuesday, 20 September 2016

    Hi Admin! Thank you for the reply... I just need certain insight like (I am repeating my question)
    a) So whats your take in Machine Learning in coming 5 years?
    b) In which sectors can ML spear into other than E-Commerce?

  • Administrator
    Administrator Tuesday, 20 September 2016

    Sorry, we don't care about ML and we don't like how MSFT is trying to portray that they have some kind of ML tool for time series.

  • NelsonMiller
    NelsonMiller Sunday, 02 July 2017

    as the spring, summer time and fall months are all height season in numerous holiday locations, finding a reduction Christmas excursion is frequently much less hard than it might seem. you may should forsake a sizzling hot sun and tolerate some cloudy days, but there are numerous destinations that offer excellent discount deals at some point of the holiday length. If you would like to know about traveling visit us at
    assignment writing australia.

  • NelsonMiller
    NelsonMiller Sunday, 02 July 2017

    Thank you for sharing this information with us, I am working In
    assignment writing australia , Christmas markets originate from East Germany and have a unique zest there even now. Spending a Christmas vacation in Berlin, and journeying Christmas markets inside the Dresden or Leipzig, is going to emerge as an thrilling holiday.

  • NelsonMiller
    NelsonMiller Sunday, 02 July 2017
  • jamessusan104
    jamessusan104 Tuesday, 29 August 2017

    Machine taking in, a branch of computerized reasoning. It is about the development and investigation of frameworks that can gain from information. A-One Assignments For instance, a machine learning framework could be prepared on email messages to figure out how to recognize spam and non-spam messages. In the wake of learning, it would then be able to be utilized to group new email messages into spam and non-spam envelopes.

  • shibu yadav
    shibu yadav Sunday, 10 September 2017

    Likewise besides the free xbox lives gold points Microsoft likewise offers the free robux live gold codes gather the working points and provide to their finest users and consumers.

  • Robert Jack
    Robert Jack Tuesday, 12 September 2017

    Teaching refers to knowledge involvements eased by a real live teacher. Assignment Writing refers to culture experiences toward cooking learners with specific knowledge.

  • Gracie Willow
    Gracie Willow Monday, 25 September 2017

    Amazing analysis you have done Tom Reilly, We coursework writers uk agreed with your point of view which you have done, actually there is no learning with the models being built in a terrible fashion. Its looks like unfitting and non intelligent modeling. Not at all a machine learning.

  • Gracie Willow
    Gracie Willow Monday, 25 September 2017

    Amazing analysis you have done Tom Reilly, We coursework writers uk agreed with your point of view which you have done, actually there is no learning with the models being built in a terrible fashion. Its looks like unfitting and non intelligent modeling. Not at all a machine learning.

  • Maisie Acton
    Maisie Acton Monday, 02 October 2017

    Tom Reilly I am agreed with you that Machine Learning might not be learning because its human who feed everything in machine, best chemistry assignment help everything which machine shows to you is feed by a human.. Though machine help us in an enormous way but still its only just a machine it can't learn on its own...

  • charlesainsley
    charlesainsley Friday, 06 October 2017

    Instructing alludes to learning associations facilitated by a genuine live instructor. Coursework Help Online
    alludes to culture encounters toward cooking students with particular learning.

  • stevewaugh
    stevewaugh Friday, 06 October 2017

    Machine taking in, a branch of electronic thinking. It is about the advancement and examination of structures that can pick up from data. Do my Coursework Online For example, a machine learning structure could be set up on email messages to make sense of how to perceive spam and non-spam messages. In the wake of learning, it would then have the capacity to be used to aggregate new email messages into spam and non-spam envelopes.

  • Dissertation Store
    Dissertation Store Monday, 09 October 2017

    This is really something that you can consider if you want to make the most of the situations you must handle to manage the people in your circle. http://www.assignmenthub.co.uk/psychology-assignment

  • Please login first in order for you to submit comments
Go to top