Re: Generalized Least Squares. What is it? Title: Re: Generalized Least Squares. What is it? Date: 14 Aug 1998 05:26:09 -0700
In article <6qutqm$lku$1@pulp.ucs.ualberta.ca, dspady@gpu.srv.ubcx5.ca says...
Can you please help me with this question? I apologize for my ignorance but I can't find a simple answer anywhere. What is generalized least squares? How does it differ from Ordinary Least Squares?, from HLM? When is it used?
Many thanks
Don Spady Dep't Pediatrics University of Alberta
dspady@gpu.srv.ualberta.ca
Like all things words have different meanings to different people ... even statistical words or terms. My read or understanding as to what Generalized Least Squares is an extension or the application of Least Squares to a more general problem than the trivial case where the errors have constant mean, constant variance and are uncorrelated (in other words the real world ! )
If the errors don't have a constant mean then include Pulses, Seasonal Pulses, Level Shifts or Local Time Trends) ... ( note ignore Seasonal Pulses, Level Shifts and Local Time Trends if you don't have time series data as only pulses would be applicable)
If the errors don't have constant variance then use Weighted Regression which essentially allows you to minimize not the simple sum of squares of the errors but the sum of [w(i)*a(i)]**2 where w(i) is a weight for the ith observation and a(i) is the model error for that ith observation. These weights can be assumed or empirically developed for time series data by a critical examination of points of change in the variance. In cross-sectional data they are akin to the degree of belief that you have in each reading or observation.
Also if the errors don't have constant variance you might need to decouple the dependence of the errors in the Mean of the Y series that is being predicted and the variance of the errors. Sometimes this power transform includes logs, square roots, inverse etc.. Collectively this body of knowledge is sometimes referred to as BOX-COX .
If the errors are not uncorrelated (doesn't apply to cross sectional data) then you might wish to include an ARIMA structure to deal with the autoregressive nature of the a's thus creating a new set of a's which have the property of independence.
If the errors don't have the same distribution then one might wish to segment or to classify the data into homogeneous sub-groups. This happens quite regularly in time series wgere the model/parameters change over time. Detecting the point in time that the model changes leads to more correct modelling.
In summary, Generalized Least Squares deals with orthogonalizing and making idempotent an otherwise Variance-Covariance Matrix of the errors.
As I close it appears that I have explained what AUTOBOX does in it's Tour De Force. Please visit http://www.autobox.com . AUTOBOX develops models which include all of the above generalizations and more.
To paraphrase Martin L. King , "Discourtesy anywhere is Discourtesy Everywhere "
If you wish to discuss these things, please call me at 215-675-0652
Dave Reilly
Senior Vice-President Automatic Forecasting Systems (developers of AUTOBOX)