The Analysis Of Time Series - An Introduction

Citation preview

cover

cover

next page >

cover

next page >

Cover

title author publisher isbn10 | asin print isbn13 ebook isbn13 language subject publication date lcc ddc subject

file:///C:/Documents and Settings/Yang/

: : : : : : : : : : :

面/The analysis of time series an introduction/files/cover.html [5/24/2009 16:50:34]

page_i

< previous page

page_i

next page >

Page i CHAPMAN & HALL/CRC Texts in Statistical Science Series Series Editors C.Chatfield, University of Bath, UK Martin Tanner, Northwestern University, USA J.Zidek, University of British Columbia, Canada Analysis of Failure and Survival Data Peter J.Smith The Analysis and Interpretation of Multivariate Data for Social Scientists David J.Bartholomew, Fiona Steele, Irini Moustaki, and Jane Galbraith The Analysis of Time Series—An Introduction, Sixth Edition Chris Chatfield Applied Bayesian Forecasting and Time Series Analysis A.Pole, M.West and J.Harrison Applied Nonparametric Statistical Methods, Third Edition P.Sprent and N.C.Smeeton Applied Statistics—Handbook of GENSTAT Analysis E.J.Snell and H.Simpson Applied Statistics—Principles and Examples D.R.Cox and E.J.Snell Bayes and Empirical Bayes Methods for Data Analysis, Second Edition Bradley P.Carlin and Thomas A.Louis Bayesian Data Analysis A.Gelman, J.Carlin, H.Stern, and D.Rubin Beyond ANOVA—Basics of Applied Statistics R.G.Miller, Jr. Computer-Aided Multivariate Analysis, Third Edition A.A.Afifi and V.A.Clark A Course in Categorical Data Analysis T.Leonard A Course in Large Sample Theory T.S.Ferguson Data Driven Statistical Methods P.Sprent Decision Analysis—A Bayesian Approach J.Q.Smith Elementary Applications of Probability Theory, Second Edition H.C.Tuckwell Elements of Simulation B.J.T.Morgan Epidemiology—Study Design and Data Analysis M.Woodward Essential Statistics, Fourth Edition D.A.G.Rees A First Course in Linear Model Theory Nalini Ravishanker and Dipak K.Dey Interpreting Data—A First Course in Statistics A.J.B.Anderson An Introduction to Generalized Linear Models, Second Edition A.J.Dobson Introduction to Multivariate Analysis C.Chatfield and A.J.Collins Introduction to Optimization Methods and their Applications in Statistics B.S.Everitt Large Sample Methods in Statistics P.K.Sen and J.da Motta Singer Markov Chain Monte Carlo—Stochastic Simulation for Bayesian Inference

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_i.html (1 of 2) [5/24/2009 16:50:36]

page_i

D.Gamerman Mathematical Statistics K.Knight Modeling and Analysis of Stochastic Systems V.Kulkarni Modelling Binary Data, Second Edition D.Collett Modelling Survival Data in Medical Research, Second Edition D.Collett

< previous page

file:///C:/Documents and Settings/Yang/

page_i

next page >

面/The analysis of time series an introduction/files/page_i.html (2 of 2) [5/24/2009 16:50:36]

page_ii

< previous page

page_ii

next page >

Page ii Multivariate Analysis of Variance and Repeated Measures—A Practical Approach for Behavioural Scientists D.J.Hand and C.C.Taylor Multivariate Statistics—A Practical Approach B.Flury and H.Riedwyl Practical Data Analysis for Designed Experiments B.S.Yandell Practical Longitudinal Data Analysis D.J.Hand and M.Crowder Practical Statistics for Medical Research D.G.Altman Probability—Methods and Measurement A.O’Hagan Problem Solving—A Statistician’s Guide, Second Edition C.Chatfield Randomization, Bootstrap and Monte Carlo Methods in Biology, Second Edition B.F.J.Manly Readings in Decision Analysis S.French Sampling Methodologies with Applications Poduri S.R.S.Rao Statistical Analysis of Reliability Data M.J.Crowder, A.C.Kimber, T.J.Sweeting, and R.L.Smith Statistical Methods for SPC and TQM D.Bissell Statistical Methods in Agriculture and Experimental Biology, Second Edition R.Mead, R.N.Curnow, and A.M.Hasted Statistical Process Control—Theory and Practice, Third Edition G.B.Wetherill and D.W.Brown Statistical Theory, Fourth Edition B.W.Lindgren Statistics for Accountants, Fourth Edition S.Letchford Statistics for Technology—A Course in Applied Statistics, Third Edition C.Chatfield Statistics in Engineering—A Practical Approach A.V.Metcalfe Statistics in Research and Development, Second Edition R.Caulcutt The Theory of Linear Models B.Jørgensen

< previous page

file:///C:/Documents and Settings/Yang/

page_ii

next page >

面/The analysis of time series an introduction/files/page_ii.html [5/24/2009 16:50:36]

page_iii

< previous page

page_iii

next page >

Page iii The Analysis of Time Series An Introduction SIXTH EDITION Chris Chatfield

CHAPMAN & HALL/CRC A CRC Press Company Boca Raton London NewYork Washington, D.C.

< previous page

file:///C:/Documents and Settings/Yang/

page_iii

next page >

面/The analysis of time series an introduction/files/page_iii.html [5/24/2009 16:50:37]

page_iv

< previous page

page_iv

next page >

Page iv This edition published in the Taylor & Francis e-Library, 2005. To purchase your own copy of this or any of Taylor & Francis or Routledge’s collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk. Library of Congress Cataloging-in-Publication Data Chatfield, Christopher. The analysis of time series: an introduction/Chris Chatfield.—6th ed. p. cm.—(Texts in statistical science) Includes bibliographical references and index. ISBN 1-58488-317-0 1. Time-series analysis. I. Title. II. Series. QA280.C4 2003 519.5′5–dc21 2003051472 This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. The consent of CRC Press LLC does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press LLC for such copying. Direct all inquiries to CRC Press LLC, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe. Visit the CRC Press Web site at www.crcpress.com © 2004 by CRC Press LLC No claim to original U.S. Government works ISBN 0-203-49168-8 Master e-book ISBN ISBN 0-203-62042-9 (OEB Format) International Standard Book Number 1-58488-317-0 (Print Edition) Library of Congress Card Number 2003051472

< previous page

file:///C:/Documents and Settings/Yang/

page_iv

next page >

面/The analysis of time series an introduction/files/page_iv.html [5/24/2009 16:50:37]

page_v

< previous page

page_v

next page >

page_v

next page >

Page v To Liz, with love

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_v.html [5/24/2009 16:50:38]

page_vi

< previous page

page_vi

next page >

Page vi Alice sighed wearily. ‘I think you might do something better with the time,’ she said, ‘than waste it in asking riddles that have no answers.’ ‘If you knew time as well as I do,’ said the Hatter, ‘you wouldn’t talk about wasting it. It’s him.’ ‘I don’t know what you mean,’ said Alice. ‘Of course you don’t!’ the Hatter said, tossing his head contemptuously. ‘I dare say you never even spoke to Time!’ ‘Perhaps not,’ Alice cautiously replied, ‘but I know I have to beat time when I learn music.’ Ah! that accounts for it,’ said the Hatter. ‘He won’t stand beating.’ Lewis Carroll, Alice’s Adventures in Wonderland

< previous page

file:///C:/Documents and Settings/Yang/

page_vi

next page >

面/The analysis of time series an introduction/files/page_vi.html [5/24/2009 16:50:38]

page_vii

page_vii

< previous page

next page >

Page vii Contents Preface to the Sixth Edition Abbreviations and Notation

xi xiii

1 Introduction 1.1 Some Representative Time Series 1.2 Terminology 1.3 Objectives of Time-Series Analysis 1.4 Approaches to Time-Series Analysis 1.5 Review of Books on Time Series 2 Simple Descriptive Techniques 2.1 Types of Variation 2.2 Stationary Time Series 2.3 The Time Plot 2.4 Transformations 2.5 Analysing Series that Contain a Trend 2.6 Analysing Series that Contain Seasonal Variation 2.7 Autocorrelation and the Correlogram 2.8 Other Tests of Randomness 2.9 Handling Real Data 3 Some Time-Series Models 3.1 Stochastic Processes and Their Properties 3.2 Stationary Processes 3.3 Some Properties of the Autocorrelation Function 3.4 Some Useful Models 3.5 The Wold Decomposition Theorem 4 Fitting Time-Series Models in the Time Domain 4.1 Estimating Autocovariance and Autocorrelation Functions 4.2 Fitting an Autoregressive Process 4.3 Fitting a Moving Average Process 4.4 Estimating Parameters of an ARMA Model 4.5 Estimating Parameters of an ARIMA Model 4.6 Box-Jenkins Seasonal ARIMA Models

< previous page

file:///C:/Documents and Settings/Yang/

page_vii

1 1 5 6 8 8 11 11 13 13 14 15 20 22 28 29 33 33 34 36 37 50 55 55 59 62 64 65 66

next page >

面/The analysis of time series an introduction/files/page_vii.html [5/24/2009 16:50:39]

page_viii

< previous page

page_viii

next page >

Page viii 4.7 Residual Analysis 4.8 General Remarks on Model Building 5 Forecasting 5.1 Introduction 5.2 Univariate Procedures 5.3 Multivariate Procedures 5.4 Comparative Review of Forecasting Procedures 5.5 Some Examples 5.6 Prediction Theory 6 Stationary Processes in the Frequency Domain 6.1 Introduction 6.2 The Spectral Distribution Function 6.3 The Spectral Density Function 6.4 The Spectrum of a Continuous Process 6.5 Derivation of Selected Spectra 7 Spectral Analysis 7.1 Fourier Analysis 7.2 A Simple Sinusoidal Model 7.3 Periodogram Analysis 7.4 Some Consistent Estimation Procedures 7.5 Confidence Intervals for the Spectrum 7.6 Comparison of Different Estimation Procedures 7.7 Analysing a Continuous Time Series 7.8 Examples and Discussion 8 Bivariate processes 8.1 Cross-Covariance and Cross-Correlation 8.2 The Cross-Spectrum 9 Linear Systems 9.1 Introduction 9.2 Linear Systems in the Time Domain 9.3 Linear Systems in the Frequency Domain 9.4 Identification of Linear Systems 10 State-Space Models and the Kalman Filter 10.1 State-Space Models 10.2 The Kalman Filter 11 Non-Linear Models 11.1 Introduction 11.2 Some Models with Non-Linear Structure

< previous page

file:///C:/Documents and Settings/Yang/

page_viii

67 70 73 73 75 87 90 98 103 107 107 107 111 113 114 121 121 122 126 130 139 140 144 146 155 155 159 169 169 171 175 190 203 203 211 217 217 222

next page >

面/The analysis of time series an introduction/files/page_viii.html [5/24/2009 16:50:39]

page_ix

page_ix

next page >

Page ix 11.3 Models for Changing Variance 11.4 Neural Networks 11.5 Chaos 11.6 Concluding Remarks 11.7 Bibliography 12 Multivariate Time-Series Modelling 12.1 Introduction 12.2 Single Equation Models 12.3 Vector Autoregressive Models 12.4 Vector ARMA Models 12.5 Fitting VAR and VARMA Models 12.6 Co-integration 12.7 Bibliography 13 Some More Advanced Topics 13.1 Model Identification Tools 13.2 Modelling Non-Stationary Series 13.3 Fractional Differencing and Long-Memory Models 13.4 Testing for Unit Roots 13.5 Model Uncertainty 13.6 Control Theory 13.7 Miscellanea 14 Examples and Practical Advice 14.1 General Comments 14.2 Computer Software 14.3 Examples 14.4 More on the Time Plot 14.5 Concluding Remarks 14.6 Data Sources and Exercises

227 230 235 238 240 241 241 245 246 249 250 252 253 255 255 257 260 262 264 266 268 277 277 278 280 290 292 292

< previous page

A B C D

Fourier, Laplace and z-Transforms Dirac Delta Function Covariance and Correlation Some MINITAB and S-PLUS Commands Answers to Exercises References Index

< previous page

file:///C:/Documents and Settings/Yang/

page_ix

295 299 301 303 307 315 329

next page >

面/The analysis of time series an introduction/files/page_ix.html [5/24/2009 16:50:40]

page_x

< previous page

page_x

next page >

page_x

next page >

Page x This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_x.html [5/24/2009 16:50:41]

page_xi

< previous page

page_xi

next page >

Page xi Preface to the Sixth Edition My aim in writing this text has been to provide an accessible book, which is wide-ranging and up-to-date and which covers both theory and practice. Enough theory is given to introduce essential concepts and make the book mathematically interesting. In addition, practical problems are addressed and worked examples are included so as to help the reader tackle the analysis of real data. The book can be used as a text for an undergraduate or a postgraduate course in time series, or it can be used for self-tuition by research workers. The positive feedback I have received over the years (plus healthy sales figures!) has encouraged me to continue to update, revise and improve the material. However, I do plan that this should be the sixth, and final edition! The book assumes a knowledge of basic probability theory and elementary statistical inference. A reasonable level of mathematics is required, though I have glossed over some mathematical difficulties, especially in the advanced Sections 3.4.8 and 3.5, which the reader is advised to omit at a first reading. In the sections on spectral analysis, the reader needs some familiarity with Fourier analysis and Fourier transforms, and I have helped the reader here by providing a special appendix on the latter topic. I am lucky in that I enjoy the rigorous elegance of mathematics as well as the very different challenge of analysing real data. I agree with David Williams (2001, Preface and p. 1) that “common sense and scientific insight are more important than Mathematics” and that “intuition is much more important than rigour”, but I also agree that this should not be an excuse for ignoring mathematics. Practical ideas should be backed up with theory whenever possible. Throughout the book, my aim is to teach both concepts and practice. In the process, I hope to convey the notion that Statistics and Mathematics are both fascinating, and I will be delighted if you agree. Although intended as an introductory text on a rather advanced topic, I have nevertheless provided appropriate references to further reading and to more advanced topics, especially in Chapter 13. The references are mainly to comprehensible and readily accessible sources, rather than to the original attributive references. This should help the reader to further the study of any topics that are of particular interest. One difficulty in writing any textbook is that many practical problems contain at least one feature that is ‘nonstandard’ in some sense. These cannot all be envisaged in a book of reasonable length. Rather the task of an author, such as myself, is to introduce generally applicable concepts and models, while

< previous page

file:///C:/Documents and Settings/Yang/

page_xi

next page >

面/The analysis of time series an introduction/files/page_xi.html [5/24/2009 16:50:41]

page_xii

< previous page

page_xii

next page >

Page xii making clear that some versatility may be needed to solve problems in practice. Thus the reader must always be prepared to use common sense when tackling real problems. Example 5.1 is a typical situation where common sense has to be applied and also reinforces the recommendation that the first step in any timeseries analysis should be to plot the data. The worked examples in Chapter 14 also include candid comments on practical difficulties in order to complement the general remarks in the main text. The first 12 chapters of the sixth edition have a similar structure to the fifth edition, although substantial revision has taken place. The notation used is mostly unchanged, but I note that the h-step-ahead forecast to to better reflect modern of the variable XN+h, made at time N, has been changed from usage. Some new topics have been added, such as Section 2.9 on Handling Real Data and Section 5.2.6 on Prediction Intervals. Chapter 13 has been completely revised and restructured to give a brief introduction to a variety of topics and is primarily intended to give readers an overview and point them in the right direction as regards further reading. New topics here include the aggregation of time series, the analysis of time series in finance and discrete-valued time series. The old Appendix D has been revised and extended to become a new Chapter 14. It gives more practical advice, and, in the process reflects the enormous changes in computing practice that have taken place over the last few years. The references have, of course, been updated throughout the book. I would like to thank Julian Faraway, Ruth Fuentes Garcia and Adam Prowse for their help in producing the graphs in this book. I also thank Howard Grubb for providing Figure 14.4. I am indebted to many other people, too numerous to mention, for assistance in various aspects of the preparation of the current and earlier editions of the book. In particular, my colleagues at Bath have been supportive and helpful over the years. Of course any errors, omissions or obscurities which remain are my responsibility and I will be glad to hear from any reader who wishes to make constructive comments. I hope you enjoy the book and find it helpful. Chris Chatfield Department of Mathematical Sciences University of Bath Bath, Avon, BA2 7AY, U.K. e-mail: [email protected]

< previous page

file:///C:/Documents and Settings/Yang/

page_xii

next page >

面/The analysis of time series an introduction/files/page_xii.html [5/24/2009 16:50:42]

page_xiii

< previous page

page_xiii

next page >

Page xiii Abbreviations and Notation AR Autoregressive MA Moving average ARMA Autoregressive moving average ARIMA Autoregressive integrated moving average SARIMA Seasonal ARIMA TAR Threshold autoregressive GARCH Generalized autoregressive conditionally heteroscedastic SWN Strict white noise UWN Uncorrelated white noise MMSE Minimum mean square error P.I. Prediction interval FFT Fast Fourier transform ac.f. Autocorrelation function acv.f. Autocovariance function N Sample size, or length of observed time series

N(h) B

Forecast of xN+h made at time N Backward shift operator such that BXt=Xt−1

E Var I N(µ, σ2)

First differencing operator, (1−B), such that Xt=Xt−Xt−1 Expectation or expected value Variance Identity matrix—a square matrix with ones on the diagonal and zeroes otherwise Normal distribution, mean µ and variance σ2

Chi-square distribution with v degrees of freedom {Zt} or {εt} Purely random process of independent random variables, usually N(0, σ2) AT or XT Transpose of a matrix A or vector X—vectors are indicated by boldface, but not scalars or matrices

< previous page

file:///C:/Documents and Settings/Yang/

page_xiii

next page >

面/The analysis of time series an introduction/files/page_xiii.html [5/24/2009 16:50:42]

page_xiv

< previous page

page_xiv

next page >

page_xiv

next page >

Page xiv This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_xiv.html [5/24/2009 16:50:43]

page_1

< previous page

page_1

next page >

Page 1 CHAPTER 1 Introduction A time series is a collection of observations made sequentially through time. Examples occur in a variety of fields, ranging from economics to engineering, and methods of analysing time series constitute an important area of statistics. 1.1 Some Representative Time Series We begin with some examples of the sort of time series that arise in practice. Economic and financial time series

Figure 1.1 The Beveridge wheat price annual index series from 1810 to1864. Many time series are routinely recorded in economics and finance. Examples include share prices on successive days, export totals in successive months, average incomes in successive months, company profits in successive years and so on.

< previous page

file:///C:/Documents and Settings/Yang/

page_1

next page >

面/The analysis of time series an introduction/files/page_1.html [5/24/2009 16:50:44]

page_2

< previous page

page_2

next page >

Page 2 The classic Beveridge wheat price index series consists of the average wheat price in nearly 50 places in various countries measured in successive years from 1500 to 1869. This series is of particular interest to economic historians, and is available in many places (e.g. at www.york.ac.uk/depts/maths/data/ts/ts04.dat). Figure 1.1 shows part of this series and some apparent cyclic behaviour can be seen. Physical time series Many types of time series occur in the physical sciences, particularly in meteorology, marine science and geophysics. Examples are rainfall on successive days, and air temperature measured in successive hours, days or months. Figure 1.2 shows the average air temperature at Recife (Brazil) in successive months over a 10-year period and seasonal fluctuations can be clearly seen.

Figure 1.2 Average air temperature (deg C) at Recife, Brazil, in successive months from 1953 to 1962. Some mechanical recorders take measurements continuously and produce a continuous trace rather than observations at discrete intervals of time. For example, in some laboratories it is important to keep temperature and humidity as constant as possible and so devices are installed to measure these variables continuously. Action may be taken when the trace goes outside pre-specified limits. Visual examination of the trace may be adequate for many purposes, but, for more detailed analysis, it is customary to convert the continuous trace to a series in discrete time by sampling the

< previous page

file:///C:/Documents and Settings/Yang/

page_2

next page >

面/The analysis of time series an introduction/files/page_2.html [5/24/2009 16:50:45]

page_3

< previous page

page_3

next page >

Page 3 trace at appropriate equal intervals of time. The resulting analysis is more straightforward and can readily be handled by standard time-series software. Marketing time series The analysis of time series arising in marketing is an important problem in commerce. Observed variables could include sales figures in successive weeks or months, monetary receipts, advertising costs and so on. As an example, Figure 1.3 shows the sales1 of an engineering product by a certain company in successive months over a 7-year period, as originally analysed by Chatfield and Prothero (1973). Note the trend and seasonal variation which is typical of sales data. As the product is a type of industrial heater, sales are naturally lower in the summer months. It is often important to forecast future sales so as to plan production. It may also be of interest to examine the relationship between sales and other time series such as advertising expenditure.

Figure 1.3 Sales of an industrial heater in successive months from January 1965 to November 1971. Demographic time series Various time series occur in the study of population change. Examples include the population of Canada measured annually, and monthly birth totals in England. Demographers want to predict changes in population for as long 1 The plotted series includes the six later observations added by Chatfield and Prothero (1973) in their reply to comments by G.E.P.Box and G.M.Jenkins.

< previous page

file:///C:/Documents and Settings/Yang/

page_3

next page >

面/The analysis of time series an introduction/files/page_3.html [5/24/2009 16:50:45]

page_4

< previous page

page_4

next page >

Page 4 as 10 or 20 years into the future, and are helped by the slowly changing structure of a human population. Standard time-series methods are usually inappropriate for tackling this problem and we refer the reader to a specialist book on demography. Process control data In process control, the problem is to detect changes in the performance of a manufacturing process by measuring a variable, which shows the quality of the process. These measurements can be plotted against time as in Figure 1.4. When the measurements stray too far from some target value, appropriate corrective action should be taken to control the process. Special techniques have been developed for this type of timeseries problem, and the reader is referred to a book on statistical quality control (e.g. Montgomery, 1996).

Figure 1.4 A process control chart. Binary processes A special type of time series arises when observations can take one of only two values, usually denoted by 0 and 1 (see Figure 1.5). For example, in computer science, the position of a switch, either ‘on’ or ‘off’, could be recorded as one or zero, respectively. Time series of this type, called binary processes, occur in many situations, including the study of communication theory.

Figure 1.5 A realization of a binary process. Point processes A completely different type of time series occurs when we consider a series of events occurring ‘randomly’ through time. For example, we could record

< previous page

file:///C:/Documents and Settings/Yang/

page_4

next page >

面/The analysis of time series an introduction/files/page_4.html [5/24/2009 16:50:46]

page_5

< previous page

page_5

next page >

Page 5 the dates of major railway disasters. A series of events of this type is usually called a point process (see Figure 1.6). For observations of this type, we are interested in such quantities as the distribution of the number of events occurring in a given time period and distribution of time intervals between events. Methods of analysing point process data are generally very different to those used for analysing standard time series data and the reader is referred, for example, to Cox and Isham (1980).

Figure 1.6 A realization of a point process (× denotes an event). 1.2 Terminology A time series is said to be continuous when observations are made continuously through time as in Figure 1.5. The adjective ‘continuous’ is used for series of this type even when the measured variable can only take a discrete set of values, as in Figure 1.5. A time series is said to be discrete when observations are taken only at specific times, usually equally spaced. The term ‘discrete’ is used for series of this type even when the measured variable is a continuous variable. This book is mainly concerned with discrete time series, where the observations are taken at equal intervals. We also consider continuous time series more briefly, while Section 13.7.4 gives some references regarding the analysis of discrete time series taken at unequal intervals of time. Discrete time series can arise in several ways. Given a continuous time series, we could read off (or digitize) the values at equal intervals of time to give a discrete time series, sometimes called a sampled series. The sampling interval between successive readings must be carefully chosen so as to lose little information (see Section 7.7). A different type of discrete series arises when a variable does not have an instantaneous value but we can aggregate (or accumulate) the values over equal intervals of time. Examples of this type are monthly exports and daily rainfalls. Finally, some time series are inherently discrete, an example being the dividend paid by a company to shareholders in successive years. Much statistical theory is concerned with random samples of independent observations. The special feature of time-series analysis is the fact that successive observations are usually not independent and that the analysis must take into account the time order of the observations. When successive observations are dependent, future values may be predicted from past observations. If a time series can be predicted exactly, it is said to be deterministic. However, most time series are stochastic in that the future is only partly determined by past values, so that exact predictions are impossible and must be replaced by the idea that future values have a probability distribution, which is conditioned by a knowledge of past values.

< previous page

file:///C:/Documents and Settings/Yang/

page_5

next page >

面/The analysis of time series an introduction/files/page_5.html [5/24/2009 16:50:46]

page_6

< previous page

page_6

next page >

Page 6 1.3 Objectives of Time-Series Analysis There are several possible objectives in analysing a time series. These objectives may be classified as description, explanation, prediction and control, and will be considered in turn. (i) Description When presented with a time series, the first step in the analysis is usually to plot the observations against time to give what is called a time plot, and then to obtain simple descriptive measures of the main properties of the series. This is described in detail in Chapter 2. The power of the time plot as a descriptive tool is illustrated by Figure 1.3, which clearly shows that there is a regular seasonal effect, with sales ‘high’ in winter and ‘low’ in summer. The time plot also shows that annual sales are increasing (i.e. there is an upward trend). For some series, the variation is dominated by such ‘obvious’ features, and a fairly simple model, which only attempts to describe trend and seasonal variation, may be perfectly adequate to describe the variation in the time series. For other series, more sophisticated techniques will be required to provide an adequate analysis. Then a more complex model will be constructed, such as the various types of stochastic processes described in Chapter 3. This book devotes a greater amount of space to the more advanced techniques, but this does not mean that elementary descriptive techniques are unimportant. Anyone who tries to analyse a time series without plotting it first is asking for trouble. A graph will not only show up trend and seasonal variation, but will also reveal any ‘wild’ observations or outliers that do not appear to be consistent with the rest of the data. The treatment of outliers is a complex subject in which common sense is as important as theory (see Section 13.7.5). An outlier may be a perfectly valid, but extreme, observation, which could, for example, indicate that the data are not normally distributed. Alternatively, an outlier may be a freak observation arising, for example, when a recording device goes wrong or when a strike severely affects sales. In the latter case, the outlier needs to be adjusted in some way before further analysis of the data. Robust methods are designed to be insensitive to outliers. Other features to look for in a time plot include sudden or gradual changes in the properties of the series. For example, a step change in the level of the series would be very important to notice, if one exists. Any changes in the seasonal pattern should also be noted. The analyst should also look out for the possible presence of turning points, where, for example, an upward trend suddenly changes to a downward trend. If there is some sort of discontinuity in the series, then different models may need to be fitted to different parts of the series. (ii) Explanation When observations are taken on two or more variables, it may be possible to use the variation in one time series to explain the variation in another series.

< previous page

file:///C:/Documents and Settings/Yang/

page_6

next page >

面/The analysis of time series an introduction/files/page_6.html [5/24/2009 16:50:47]

page_7

< previous page

page_7

next page >

Page 7 This may lead to a deeper understanding of the mechanism that generated a given time series. Although multiple regression models are occasionally helpful here, they are not really designed to handle time-series data, with all the correlations inherent therein, and so we will see that alternative classes of models should be considered. Chapter 9 considers the analysis of what are called linear systems. A linear system converts an input series to an output series by a linear operation. Given observations on the input and output to a linear system (see Figure 1.7), the analyst wants to assess the properties of the linear system. For example, it is of interest to see how sea level is affected by temperature and pressure, and to see how sales are affected by price and economic conditions. A class of models, called transfer function models, enable us to model time-series data in an appropriate way.

Figure 1.7 Schematic representation of a linear system. (iii) Prediction Given an observed time series, one may want to predict the future values of the series. This is an important task in sales forecasting, and in the analysis of economic and industrial time series. Many writers, including myself, use the terms ‘prediction’ and ‘forecasting’ interchangeably, but note that some authors do not. For example, Brown (1963) uses ‘prediction’ to describe subjective methods and ‘forecasting’ to describe objective methods. (iv) Control Time series are sometimes collected or analysed so as to improve control over some physical or economic system. For example, when a time series is generated that measures the ‘quality’ of a manufacturing process, the aim of the analysis may be to keep the process operating at a ‘high’ level. Control problems are closely related to prediction in many situations. For example, if one can predict that a manufacturing process is going to move off target, then appropriate corrective action can be taken. Control procedures vary considerably in style and sophistication. In statistical quality control, the observations are plotted on control charts and the controller takes action as a result of studying the charts. A more complicated type of approach is based on modelling the data and using the model to work out an ‘optimal’ control strategy—see, for example, Box et al. (1994). In this approach, a stochastic model is fitted to the series, future values of the series are predicted, and then the input process variables are adjusted so as to keep the process on target. Many other contributions to control theory have been made by control engineers and mathematicians rather than statisticians. This topic is rather outside the scope of this book but is briefly introduced in Section 13.6.

< previous page

file:///C:/Documents and Settings/Yang/

page_7

next page >

面/The analysis of time series an introduction/files/page_7.html [5/24/2009 16:50:48]

page_8

< previous page

page_8

next page >

Page 8 1.4 Approaches to Time-Series Analysis This book will describe various approaches to time-series analysis. Chapter 2 describes simple descriptive techniques, which include plotting the data and looking for trends, seasonal fluctuations and so on. Chapter 3 introduces a variety of probability models for time series, while Chapter 4 discusses ways of fitting these models to time series. The major diagnostic tool that is used in Chapter 4 is a function called the autocorrelation function, which helps to describe the evolution of a process through time. Inference based on this function is often called an analysis in the time domain. Chapter 5 goes on to discuss a variety of forecasting procedures, but this chapter is not a prerequisite for the rest of the book. Chapter 6 introduces a function called the spectral density function, which describes how the variation in a time series may be accounted for by cyclic components at different frequencies. Chapter 7 shows how to estimate the spectral density function by means of a procedure called spectral analysis. Inference based on the spectral density function is often called an analysis in the frequency domain. Chapter 8 discusses the analysis of two time series, while Chapter 9 extends this work by considering linear systems in which one series is regarded as the input, while the other series is regarded as the output. Chapter 10 introduces an important class of models, called state-space models. It also describes the Kalman filter, which is a general method of updating the best estimate of the ‘signal’ in a time series in the presence of noise. Chapters 11 and 12 introduce non-linear and multivariate time-series models, respectively, while Chapter 13 briefly reviews some more advanced topics. Chapter 14 completes the book with further examples and practical advice. 1.5 Review of Books on Time Series This section gives a brief review of some alternative books on time series that may be helpful to the reader. The literature has expanded considerably in recent years and so a selective approach is necessary. Alternative general introductory texts include Brockwell and Davis (2002), Harvey (1993), Kendall and Ord (1990) and Wei (1990). Diggle (1990) is aimed primarily at biostatisticians, while Enders (1995) and Mills (1990, 1999) are aimed at economists. There are many more advanced texts including Anderson (1971), Brockwell and Davis (1991), Fuller (1996) and Priestley (1981). The latter is particularly strong on spectral analysis and multivariate time-series modelling. Brillinger (2001) is a classic reprint of a book concentrating on the frequency domain. Kendall et al. (1983), now in the fourth edition, is a valuable reference source, but note that earlier editions are somewhat dated. Hamilton (1994) is aimed at econometricians. The classic book by Box and Jenkins (1970) describes an approach to time-series analysis, forecasting and control that is based on a particular

< previous page

file:///C:/Documents and Settings/Yang/

page_8

next page >

面/The analysis of time series an introduction/files/page_8.html [5/24/2009 16:50:48]

page_9

< previous page

page_9

next page >

Page 9 class of models, called autoregressive integrated moving average (ARIMA) models. This important book is not really suitable for the beginner, who is recommended to read Chapters 3–5 in this book, Vandaele (1983) or the relevant chapters of Wei (1990). The 1976 revised edition of Box and Jenkins (1970) was virtually unchanged, but the third edition (Box et al., 1994), with G.Reinsel as third author, has changed substantially. Chapters 12 and 13 have been completely rewritten to incorporate new material on intervention analysis, outlier detection and process control. The first 11 chapters of the new edition have a very similar structure to the original edition, though there are some relatively minor additions such as new material on model estimation and testing for unit roots. For historical precedence and reader convenience we sometimes refer to the 1970 edition for material on ARIMA modelling, but increasingly refer to the third edition, especially for material in the later chapters. Additional books will be referenced, as appropriate, in later chapters.

< previous page

file:///C:/Documents and Settings/Yang/

page_9

next page >

面/The analysis of time series an introduction/files/page_9.html [5/24/2009 16:50:49]

page_10

< previous page

page_10

next page >

page_10

next page >

Page 10 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_10.html [5/24/2009 16:50:49]

page_11

< previous page

page_11

next page >

Page 11 CHAPTER 2 Simple Descriptive Techniques Statistical techniques for analysing time series range from relatively straightforward descriptive methods to sophisticated inferential techniques. This chapter introduces the former, which will often clarify the main properties of a given series. Descriptive methods should generally be tried before attempting more complicated procedures, because they can be vital in ‘cleaning’ the data, and then getting a ‘feel’ for them, before trying to generate ideas as regards a suitable model. Before doing anything, the analyst should make sure that the practical problem being tackled is properly understood. In other words, the context of a given problem is crucial in time-series analysis, as in all areas of statistics. If necessary, the analyst should ask questions so as to get appropriate background information and clarify the objectives. These preliminary questions should not be rushed. In particular, make sure that appropriate data have been, or will be, collected. If the series are too short, or the wrong variables have been measured, it may not be possible to solve the given problem. For a chapter on Descriptive Techniques, the reader may be expecting the first section to deal with summary statistics. Indeed, in most areas of statistics, a typical analysis begins by computing the sample mean (or median or mode) and the standard deviation (or interquartile range) to measure ‘location’ and ‘dispersion’. However, Time-series analysis is different! If a time series contains trend, seasonality or some other systematic component, the usual summary statistics can be seriously misleading and should not be calculated. Moreover, even when a series does not contain any systematic components, the summary statistics do not have their usual properties (see Section 4.1.2). Thus, this chapter focuses on ways of understanding typical time-series effects, such as trend, seasonality and correlations between successive observations. 2.1 Types of Variation Traditional methods of time-series analysis are mainly concerned with decomposing the variation in a series into components representing trend, seasonal variation and other cyclic changes. Any remaining variation is attributed to ‘irregular’ fluctuations. This approach is not always the best but is particularly valuable when the variation is dominated by trend and seasonality. However, it is worth noting that a decomposition into trend and

< previous page

file:///C:/Documents and Settings/Yang/

page_11

next page >

面/The analysis of time series an introduction/files/page_11.html [5/24/2009 16:50:50]

page_12

< previous page

page_12

next page >

Page 12 seasonal variation is generally not unique unless certain assumptions are made. Thus some sort of modelling, either explicit or implicit, may be involved in carrying out these descriptive techniques, and this demonstrates the blurred borderline that always exists between descriptive and inferential techniques in Statistics. The different sources of variation will now be described in more detail. Seasonal variation Many time series, such as sales figures and temperature readings, exhibit variation that is annual in period. For example, unemployment is typically ‘high’ in winter but lower in summer. This yearly variation is easy to understand, and can readily be estimated if seasonality is of direct interest. Alternatively, seasonal variation can be removed from the data, to give deseasonalized data, if seasonality is not of direct interest. Other cyclic variation Apart from seasonal effects, some time series exhibit variation at a fixed period due to some other physical cause. An example is daily variation in temperature. In addition some time series exhibit oscillations, which do not have a fixed period but which are predictable to some extent. For example, economic data are sometimes thought to be affected by business cycles with a period varying from about 3 or 4 years to more than 10 years, depending on the variable measured. However, the existence of such business cycles is the subject of some controversy, and there is increasing evidence that any such cycles are not symmetric. An economy usually behaves differently when going into recession, rather than emerging from recession. Trend This may be loosely defined as ‘long-term change in the mean level’. A difficulty with this definition is deciding what is meant by ‘long term’. For example, climatic variables sometimes exhibit cyclic variation over a very long time period such as 50 years. If one just had 20 years of data, this long-term oscillation may look like a trend, but if several hundred years of data were available, then the long-term cyclic variation would be visible. Nevertheless in the short term it may still be more meaningful to think of such a long-term oscillation as a trend. Thus in speaking of a ‘trend’, we must take into account the number of observations available and make a subjective assessment of what is meant by the phrase ‘long term’. As for seasonality, methods are available either for estimating the trend, or for removing it so that the analyst can look more closely at other sources of variation. Other irregular fluctuations After trend and cyclic variations have been removed from a set of data, we are left with a series of residuals that may or may not be ‘random’. In due course, we will examine various techniques for analysing series of this

< previous page

file:///C:/Documents and Settings/Yang/

page_12

next page >

面/The analysis of time series an introduction/files/page_12.html [5/24/2009 16:50:51]

page_13

< previous page

page_13

next page >

Page 13 type, either to see whether any cyclic variation is still left in the residuals, or whether apparently irregular variation may be explained in terms of probability models, such as moving average (MA) or autoregressive (AR) models, which will be introduced in Chapter 3. 2.2 Stationary Time Series A mathematical definition of a stationary time-series model will be given in Section 3.2. However, it may be helpful to introduce here the idea of stationarity from an intuitive point of view. Broadly speaking a time series is said to be stationary if there is no systematic change in mean (no trend), if there is no systematic change in variance and if strictly periodic variations have been removed. In other words, the properties of one section of the data are much like those of any other section. Strictly speaking, there is no such thing as a ‘stationary time series’, as the stationarity property is defined for a model. However, the phrase is often used for time-series data meaning that they exhibit characteristics that suggest a stationary model can sensibly be fitted. Much of the probability theory of time series is concerned with stationary time series, and for this reason time-series analysis often requires one to transform a non-stationary series into a stationary one so as to use this theory. For example, it may be of interest to remove the trend and seasonal variation from a set of data and then try to model the variation in the residuals by means of a stationary stochastic process. However, it is also worth stressing that the non-stationary components, such as the trend, may be of more interest than the stationary residuals. 2.3 The Time Plot The first, and most important, step in any time-series analysis is to plot the observations against time. This graph, called a time plot, will show up important features of the series such as trend, seasonality, outliers and discontinuities. The plot is vital, both to describe the data and to help in formulating a sensible model, and several examples have already been given in Chapter 1. Plotting a time series is not as easy as it sounds. The choice of scales, the size of the intercept and the way that the points are plotted (e.g. as a continuous line or as separate dots or crosses) may substantially affect the way the plot ‘looks’, and so the analyst must exercise care and judgement. In addition, the usual rules for drawing ‘good’ graphs should be followed: a clear title must be given, units of measurement should be stated and axes should be properly labelled. Nowadays, graphs are usually produced by computers. Some are well drawn but packages sometimes produce rather poor graphs and the reader must be prepared to modify them if necessary or, better, give the computer appropriate

< previous page

file:///C:/Documents and Settings/Yang/

page_13

next page >

面/The analysis of time series an introduction/files/page_13.html [5/24/2009 16:50:52]

page_14

< previous page

page_14

next page >

Page 14 instructions to produce a clear graph in the first place. For example, the software will usually print out the title you provide, and so it is your job to provide a clear title. It cannot be left to the computer. Further advice and examples are given in Chapter 14. 2.4 Transformations Plotting the data may suggest that it is sensible to consider transforming them, for example, by taking logarithms or square roots. The three main reasons for making a transformation are as follows. (i) To stabilize the variance If there is a trend in the series and the variance appears to increase with the mean, then it may be advisable to transform the data. In particular, if the standard deviation is directly proportional to the mean, a logarithmic transformation is indicated. On the other hand, if the variance changes through time without a trend being present, then a transformation will not help. Instead, a model that allows for changing variance should be considered. (ii) To make the seasonal effect additive If there is a trend in the series and the size of the seasonal effect appears to increase with the mean, then it may be advisable to transform the data so as to make the seasonal effect constant from year to year. The seasonal effect is then said to be additive. In particular, if the size of the seasonal effect is directly proportional to the mean, then the seasonal effect is said to be multiplicative and a logarithmic transformation is appropriate to make the effect additive. However, this transformation will only stabilize the variance if the error term is also thought to be multiplicative (see Section 2.6), a point that is sometimes overlooked. (iii) To make the data normally distributed Model building and forecasting are usually carried out on the assumption that the data are normally distributed. In practice this is not necessarily the case; there may, for example, be evidence of skewness in that there tend to be ‘spikes’ in the time plot that are all in the same direction (either up or down). This effect can be difficult to eliminate with a transformation and it may be necessary to model the data using a different ‘error’ distribution. The logarithmic and square-root transformations, mentioned above, are special cases of a general class of transformations called the Box-Cox transformation. Given an observed time series {xt} and a transformation parameter , the transformed series is given by

< previous page

file:///C:/Documents and Settings/Yang/

page_14

next page >

面/The analysis of time series an introduction/files/page_14.html [5/24/2009 16:50:52]

page_15

< previous page

page_15

next page >

Page 15 This is effectively just a power transformation when , as the constants are introduced to make yt a continuous function of at the value . The ‘best’ value of . can be ‘guesstimated’, or alternatively estimated by a proper inferential procedure, such as maximum likelihood. It is instructive to note that Nelson and Granger (1979) found little improvement in forecast performance when a general Box-Cox transformation was tried on a number of series. There are problems in practice with transformations in that a transformation, which makes the seasonal effect additive, for example, may fail to stabilize the variance. Thus it may be impossible to achieve all the above requirements at the same time. In any case a model constructed for the transformed data may be less than helpful. It is more difficult to interpret and forecasts produced by the transformed model may have to be ‘transformed back’ in order to be of use. This can introduce biasing effects. My personal preference is to avoid transformations wherever possible except where the transformed variable has a direct physical interpretation. For example, when percentage increases are of interest, then taking logarithms makes sense (see Example 14.3). Further general remarks on transformations are given by Granger and Newbold (1986, Section 10.5). 2.5 Analysing Series that Contain a Trend In Section 2.1, we loosely defined trend as a ‘long-term change in the mean level’. It is much more difficult to give a precise definition of trend and different authors may use the term in different ways. The simplest type of trend is the familiar ‘linear trend+noise’, for which the observation at time t is a random variable Xt, given by (2.1) where α, β are constants and εt denotes a random error term with zero mean. The mean level at time t is given by mt=(α+βt); this is sometimes called ‘the trend term’. Other writers prefer to describe the slope β as the trend, so that trend is the change in the mean level per unit time. It is usually clear from the context as to what is meant by ‘trend’. The trend in Equation (2.1) is a deterministic function of time and is sometimes called a global linear trend. In practice, this generally provides an unrealistic model, and nowadays there is more emphasis on models that allow for local linear trends. One possibility is to fit a piecewise linear model where the trend line is locally linear but with change points where the slope and intercept change (abruptly). It is usually arranged that the lines join up at the change points, but, even so, the sudden changes in slope often seem unnatural. Thus, it often seems more sensible to look at models that allow a smooth transition between the different submodels. Extending this idea, it seems even more natural to allow the parameters α and β in Equation (2.1) to evolve through time. This could be done deterministically, but it is more common to assume that a and β evolve stochastically giving rise to what is called a stochastic trend. Some examples of suitable models, under

< previous page

file:///C:/Documents and Settings/Yang/

page_15

next page >

面/The analysis of time series an introduction/files/page_15.html [5/24/2009 16:50:53]

page_16

< previous page

page_16

next page >

Page 16 the general heading of state-space models, are given in Chapter 10. Another possibility, depending on how the data look, is that the trend has a non-linear form, such as quadratic growth. Exponential growth can be particularly difficult to handle, even if logarithms are taken to transform the trend to a linear form. Even with present-day computing aids, it can still be difficult to decide what form of trend is appropriate in a given context (see Ball and Wood (1996) and the discussion that followed). The analysis of a time series that exhibits trend depends on whether one wants to (1) measure the trend and/ or (2) remove the trend in order to analyse local fluctuations. It also depends on whether the data exhibit seasonality (see Section 2.6). With seasonal data, it is a good idea to start by calculating successive yearly averages, as these will provide a simple description of the underlying trend. An approach of this type is sometimes perfectly adequate, particularly if the trend is fairly small, but sometimes a more sophisticated approach is desired. We now describe some different general approaches to describing trend. 2.5.1 Curve fitting A traditional method of dealing with non-seasonal data that contain a trend, particularly yearly data, is to fit a simple function of time such as a polynomial curve (linear, quadratic, etc.), a Gompertz curve or a logistic curve (e.g. see Meade, 1984; Franses, 1998, Chapter 4). The global linear trend in Equation (2.1) is the simplest type of polynomial curve. The Gompertz curve can be written in the form where a, b, r are parameters with 0

面/The analysis of time series an introduction/files/page_16.html [5/24/2009 16:50:54]

page_17

page_17

< previous page

next page >

Page 17 2.5.2 Filtering A second procedure for dealing with a trend is to use a linear filter, which converts one time series, {xt}, into another, {yt}, by the linear operation

where {ar} is a set of weights. In order to smooth out local fluctuations and estimate the local mean, we should clearly choose the weights so that ∑ar=1, and then the operation is often referred to as a moving average. Moving averages are discussed in detail by Kendall et al. (1983, Chapter 46), and we will only provide a brief introduction. Moving averages are often symmetric with s=q and aj=a−j.The simplest example of a symmetric smoothing filter is the simple moving average, for which ar=1/(2q+1) for r=−q,…, +q, and the smoothed value of xt is given by

The simple moving average is not generally recommended by itself for measuring trend, although it can be useful for removing seasonal variation. Another symmetric example is provided by the case where the {ar} are successive terms in the expansion of

. Thus when q=1, the weights are

. As q gets large, the weights approximate to a normal curve. A third example is Spencer’s 15-point moving average, which is used for smoothing mortality statistics to get life tables. This covers 15 consecutive points with q=7, and the symmetric weights are

A fourth example, called the Henderson moving average, is described by Kenny and Durbin (1982) and is widely used, for example, in the X-11 and X-12 seasonal packages (see Section 2.6). This moving average aims to follow a cubic polynomial trend without distortion, and the choice of q depends on the degree of irregularity. The symmetric nine-term moving average, for example, is given by The general idea is to fit a polynomial curve, not to the whole series, but to a local set of points. For example, a polynomial fitted to the first (2q+1) data points can be used to determine the interpolated value at the middle of the range where t=(q+1), and the procedure can then be repeated using the data from t=2 to t=(2q+2) and so on. Whenever a symmetric filter is chosen, there is likely to be an end-effects problem (e.g. Kendall et al., 1983, Section 46.11), since Sm(xt) can only be calculated for t=(q+1) to t=N−q. In some situations this may not

< previous page

file:///C:/Documents and Settings/Yang/

page_17

next page >

面/The analysis of time series an introduction/files/page_17.html [5/24/2009 16:50:54]

page_18

< previous page

page_18

next page >

Page 18 be important, as, for example, in carrying out some retrospective analyses. However, in other situations, such as in forecasting, it is particularly important to get smoothed values right up to t=N. The analyst can project the smoothed values by eye or, alternatively, can use an asymmetric filter that only involves present and past values of xt. For example, the popular technique known as exponential smoothing (see Section 5.2.2) effectively assumes that

where α is a constant such that 0

Page 19 As an example, two filters in series may be represented as on the opposite page. It is easy to show that a series of linear operations is still a linear filter overall. Suppose filter I, with weights {ar}, acts on {xt} to produce {yt}. Then filter II with weights {bj} acts on {yt} to produce {zt}. Now

where

are the weights for the overall filter. The weights {ck} are obtained by a procedure called convolution, and we may write

where the symbol * represents the convolution operator. For example, the filter

may be written as

The Spencer 15-point moving average is actually a convolution of four filters, namely

and this may be the best way to compute it. 2.5.3 Differencing A special type of filtering, which is particularly useful for removing a trend, is simply to difference a given time series until it becomes stationary. We will see that this method is an integral part of the so-called BoxJenkins procedure. For non-seasonal data, first-order differencing is usually sufficient to attain apparent stationarity. Here a new series, say {y2,…, yN}, is formed from the original observed series, say {x1,…, xN}, for t=2, 3,…, N. Occasionally second-order differencing is required using the

by operator

, where

First differencing is widely used and often works well. For example, Franses and Kleibergen (1996) show that better out-of-sample forecasts are usually obtained with economic data by using first differences rather than fitting

< previous page

file:///C:/Documents and Settings/Yang/

page_19

next page >

面/The analysis of time series an introduction/files/page_19.html [5/24/2009 16:50:56]

page_20

< previous page

page_20

next page >

Page 20 a deterministic (or global) trend. Seasonal differencing, to remove seasonal variation, will be introduced in the next section. 2.5.4 Other approaches Some alternative, more complicated, approaches to handling trend will be introduced later in the book. In particular, several state-space models involving trend terms will be introduced in Chapter 10, while the distinction between difference-stationary and trend-stationary series is discussed much later in Section 13.4. 2.6 Analysing Series that Contain Seasonal Variation In Section 2.1 we introduced seasonal variation, which is generally annual in period, while Section 2.4 distinguished between additive seasonality, which is constant from year to year, and multiplicative seasonality. Three seasonal models in common use are A Xt=mt+St+εt B Xt=mtSt+εt C Xt=mtStεt where mt is the deseasonalized mean level at time t, St is the seasonal effect at time t and εt is the random error. Model A describes the additive case, while models B and C both involve multiplicative seasonality. In model C the error term is also multiplicative, and a logarithmic transformation will turn this into a (linear) additive model, which may be easier to handle. The time plot should be examined to see which model is likely to give the better description. The seasonal indices {St} are usually assumed to change slowly through time so that , where s is the number of observations per year. The indices are usually normalized so that they sum to zero in the additive case, or average to one in the multiplicative case. Difficulties arise in practice if the seasonal and/or error terms are not exactly multiplicative or additive. For example, the seasonal effect may increase with the mean level but not at such a fast rate so that it is somewhere ‘in between’ being multiplicative or additive. A mixed additive-multiplicative seasonal model is described by Durbin and Murphy (1975). The analysis of time series, which exhibit seasonal variation, depends on whether one wants to (1) measure the seasonal effect and/or (2) eliminate seasonality. For series showing little trend, it is usually adequate to estimate the seasonal effect for a particular period (e.g. January) by finding the average of each January observation minus the corresponding yearly average in the additive case, or the January observation divided by the yearly average in the multiplicative case. For series that do contain a substantial trend, a more sophisticated approach is required. With monthly data, the most common way of eliminating the

< previous page

file:///C:/Documents and Settings/Yang/

page_20

next page >

面/The analysis of time series an introduction/files/page_20.html [5/24/2009 16:50:56]

page_21

< previous page

page_21

next page >

Page 21 seasonal effect is to calculate

Note that the two end coefficients are different from the rest but that the coefficients sum to unity. A simple moving average cannot be used, as this would span 12 months and would not be centered on an integer value of t. A simple moving average over 13 months cannot be used, as this would give twice as much weight to the month appearing at both ends. For quarterly data, the seasonal effect can be eliminated by calculating

For 4-weekly data, one can use a simple moving average over 13 successive observations. These smoothing procedures all effectively estimate the local (deseasonalized) level of the series. The seasonal effect itself can then be estimated by calculating xt−Sm(xt) or xt/Sm(xt) depending on whether the seasonal effect is thought to be additive or multiplicative. A check should be made that the seasonals are reasonably stable, and then the average monthly (or quarterly etc.) effects can be calculated. A seasonal effect can also be eliminated by a simple linear filter called seasonal differencing. For example, with monthly data one can employ the operator 12 where Further details on seasonal differencing will be given in Sections 4.6 and 5.2.4. Two general reviews of methods for seasonal adjustment are Butter and Fase (1991) and Hylleberg (1992). Without going into great detail, we should mention the widely used X-11 method, now updated as the X12 method (Findley et al., 1998), which is used for estimating or removing both trend and seasonal variation. It is a fairly complicated procedure that employs a series of linear filters and adopts a recursive approach. Preliminary estimates of trend are used to get preliminary estimates of seasonal variation, which in turn are used to get better estimates of trend and so on. The new software for X-12 gives the user more flexibility in handling outliers, as well as providing better diagnostics and an improved user interface. X-12 also allows the user to deal with the possible presence of calendar effects, which should always be considered when dealing with seasonal data (e.g. Bell and Hillmer, 1983). For example, if Easter falls in March one year, and April the next, then this will alter the seasonal effect on sales for both months. The X-11 or X-12 packages are often combined with ARIMA modelling, as introduced in the next three chapters, to interpolate the values near the end of the series and avoid the end-effects problem arising from using symmetric linear filters alone. The package is called X-12-ARIMA. On mainland Europe, many governments use an alternative approach, based on packages called SEATS (Signal Extraction in

< previous page

file:///C:/Documents and Settings/Yang/

page_21

next page >

面/The analysis of time series an introduction/files/page_21.html [5/24/2009 16:50:57]

page_22

page_22

< previous page

next page >

Page 22 ARIMA Time Series) and TRAMO (Time-Series Regression with ARIMA Noise). They are described in Gómez and Maravall (2001). 2.7 Autocorrelation and the Correlogram An important guide to the properties of a time series is provided by a series of quantities called the sample autocorrelation coefficients. They measure the correlation, if any, between observations at different distances apart and provide useful descriptive information. In Chapter 4, we will see that they are also an important tool in model building, and often provide valuable clues to a suitable probability model for a given set of data. We assume that the reader is familiar with the ordinary correlation coefficient.1 Given N pairs of observations on two variables x and y, say {(x1, y1), (x2, y2),…, (xN, yN)}, the sample correlation coefficient is given by (2.2) This quantity lies in the range [−1, 1] and measures the strength of the linear association between the two variables. It can easily be shown that the value does not depend on the units in which the two variables are measured. The correlation is negative if ‘high’ values of x tend to go with ‘low’ values of y. If the two variables are independent, then the true correlation is zero. Here, we apply an analogous formula to timeseries data to measure whether successive observations are correlated. Given N observations x1,…, xN, on a time series, we can form N−1 pairs of observations, namely, (x1, x2), (x2, x3),…, (xN−1, xN), where each pair of observations is separated by one time interval. Regarding the first observation in each pair as one variable, and the second observation in each pair as a second variable, then, by analogy with Equation (2.2), we can measure the correlation coefficient between adjacent observations, xt and xt+1, using the formula

(2.3)

where

is the mean of the first observation in each of the (N−1) pairs and so is the mean of the first N−1 observations, while

1This topic is briefly revised in Appendix C.

< previous page

file:///C:/Documents and Settings/Yang/

page_22

next page >

面/The analysis of time series an introduction/files/page_22.html [5/24/2009 16:50:58]

page_23

page_23

< previous page

next page >

Page 23 is the mean of the last N−1 observations. As the coefficient given by Equation (2.3) measures correlation between successive observations, it is called an autocorrelation coefficient or a serial correlation coefficient at lag one. , it is usually approximated by

Equation (2.3) is rather complicated, and so, as

(2.4) where is the overall mean. It is often convenient to further simplify this expression by dropping the factor N/(N−1), which is close to one for large N. This gives the even simpler formula

(2.5) and this is the form that will be used in this book. In a similar way, we can find the correlation between observations that are k steps apart, and this is given by

(2.6) This is called the autocorrelation coefficient at lag k. In practice the autocorrelation coefficients are usually calculated by computing the series of autocovariance coefficients, {ck}, which we define by analogy with the usual covariance formula2 as

(2.7)

This is the autocovariance coefficient at lag k. We then compute (2.8)

for k=1, 2,…, M, where M

面/The analysis of time series an introduction/files/page_23.html [5/24/2009 16:50:59]

page_24

< previous page

page_24

next page >

Page 24 prefer to use Equations (2.7) and (2.5) for reasons explained later in Section 4.1. We note, once again, that there are only small differences between the different formulae for large N. 2.7.1 The correlogram A useful aid in interpreting a set of autocorrelation coefficients is a graph called a correlogram in which the sample autocorrelation coefficients rk are plotted against the lag k for k=0, 1,…, M, where M is usually much less than N. For example if N=200, then the analyst might look at the first 20 or 30 coefficients. Examples are given in Figures 2.1–2.4. A visual inspection of the correlogram is often very helpful. Of course, r0 is always unity, but is still worth plotting for comparative purposes. The correlogram may alternatively be called the sample autocorrelation function (ac.f.). 2.7.2 Interpreting the correlogram Interpreting the meaning of a set of autocorrelation coefficients is not always easy. Here we offer some general advice. Random series A time series is said to be completely random if it consists of a series of independent observations having the same distribution. Then, for large N, we expect to find that for all non-zero values of k. In fact we will see later that, for a random time series, rk is approximately N(0, 1/N). Thus, if a time series is random, we can expect 19 out of 20 of the values of rk to lie between . As a result, it is common practice to regard any values of rk outside these limits as being ‘significant’. However, if one plots say the first 20 values of rk, then one can expect to find one ‘significant’ value on average even when the time series really is random. This spotlights one of the difficulties in interpreting the correlogram, in that a large number of coefficients are quite likely to contain one (or more) ‘unusual’ results, even when no real effects are present. (See also Section 4.1.) Short-term correlation Stationary series often exhibit short-term correlation characterized by a fairly large value of r1 followed by one or two further coefficients, which, while greater than zero, tend to get successively smaller. Values of rk for longer lags tend to be approximately zero. An example of such a correlogram is shown in Figure 2.1. A time series that gives rise to such a correlogram is one for which an observation above the mean tends to be followed by one or more further observations above the mean, and similarly for observations below the mean.

< previous page

file:///C:/Documents and Settings/Yang/

page_24

next page >

面/The analysis of time series an introduction/files/page_24.html [5/24/2009 16:50:59]

page_25

< previous page

page_25

next page >

Page 25

Figure 2.1 A time series showing short-term correlation together with its correlogram. Alternating series If a time series has a tendency to alternate, with successive observations on different sides of the overall mean, then the correlogram also tends to alternate. With successive values on opposite sides of the mean, the value of r1 will naturally be negative, but the value of r2 will be positive, as observations at lag 2 will tend to be on the same side of the mean. An alternating time series together with its correlogram is shown in Figure 2.2.

Figure 2.2 An alternating time series together with its correlogram.

< previous page file:///C:/Documents and Settings/Yang/

page_25

next page >

面/The analysis of time series an introduction/files/page_25.html [5/24/2009 16:51:00]

page_26

page_26

< previous page

next page >

Page 26 Non-stationary series If a time series contains a trend, then the values of rk will not come down to zero except for very large values of the lag. This is because an observation on one side of the overall mean tends to be followed by a large number of further observations on the same side of the mean because of the trend. A typical nonstationary time series together with its correlogram is shown in Figure 2.3. Little can be inferred from a correlogram of this type as the trend dominates all other features. In fact the sample ac.f. {rk} is only meaningful for data from a stationary time-series model (see Chapters 3 and 4) and so any trend should be removed before calculating {rk}. Of course, if the trend itself is of prime interest, then it should be modelled, rather than removed, and then the correlogram is not helpful.

Figure 2.3 A non-stationary time series together with its correlogram. Seasonal series If a time series contains seasonal variation, then the correlogram will also exhibit oscillation at the same frequency. For example, with monthly observations, we can expect r6 to be ‘large’ and negative, while r12 will be ‘large’ and positive. In particular if xt follows a sinusoidal pattern, then so does rk. For example, if where a is a constant and the frequency

is such that 0<

面/The analysis of time series an introduction/files/page_26.html [5/24/2009 16:51:00]

page_27

< previous page

page_27

next page >

Page 27 extra information, as the seasonal pattern is clearly evident in the time plot of the data. Note that it is generally wise to look at coefficients covering at least three seasons. For example, coefficients up to lag 36 may be useful for monthly data. If the seasonal variation is removed from seasonal data, then the correlogram may provide useful information. The seasonal variation was removed from the air temperature data by the simple, but rather crude, procedure of calculating the 12 monthly averages and subtracting the appropriate one from each individual observation. The correlogram of the resulting series (Figure 2.4(b)) shows that the first three coefficients are significantly different from zero. This indicates short-term correlation in that a month, which is say colder than the average for that month, will tend to be followed by one or two further months that are colder than average.

Figure 2.4 The correlogram of monthly observations on air temperature at Recife: (a) for the raw data; (b) for the seasonally adjusted data. The dotted lines in (b) are at . Values outside these lines are said to be significantly different from zero. Outliers If a time series contains one or more outliers, the correlogram may be seriously affected and it may be advisable to adjust outliers in some way before starting the formal analysis. For example, if there is one outlier in the time series at, say, time t0, and if it is not adjusted, then the plot of xt against xt+k will contain two ‘extreme’ points, namely, and depress the sample correlation coefficients towards zero. If

< previous page

file:///C:/Documents and Settings/Yang/

page_27

. The effect of these two points will be to

next page >

面/The analysis of time series an introduction/files/page_27.html [5/24/2009 16:51:01]

page_28

< previous page

page_28

next page >

Page 28 there are two outliers, this effect is even more noticeable, except when the lag equals the distance between the outliers when a spuriously large correlation may occur. General remarks Considerable experience is required to interpret sample autocorrelation coefficients. In addition it is necessary to study the probability theory of stationary series and learn about the classes of models that may be appropriate. It is also necessary to investigate the sampling properties of rk. These topics will be covered in the next two chapters and we will then be in a better position to interpret the correlogram of a given time series. 2.8 Other Tests of Randomness In most cases, a visual examination of the graph of a time series is enough to see that the series is not random, as, for example, if trend or seasonality is present or there is obvious short-term correlation. However, it is occasionally desirable to assess whether an apparently stationary time series is ‘random’. One type of approach is to carry out what is called a test of randomness in which one tests whether the observations x1,…, xN could have arisen in that order by chance by taking a simple random sample size N from a population assumed to be stationary but with unknown characteristics. Various tests exist for this purpose as described, for example, by Kendall et al. (1983, Section 45.15) and by Kendall and Ord (1990, Chapter 2). It is convenient to briefly mention such tests here. One type of test is based on counting the number of turning points, meaning the number of times there is a local maximum or minimum in the time series. A local maximum is defined to be any observation xt such that xt>xt−1 and also xt>xt+1. A converse definition applies to local minima. If the series really is random, one can work out the expected number of turning points and compare it with the observed value. An alternative type of test is based on runs of observations. For example, the analyst can count the number of runs where successive observations are all greater than the median or all less than the median. This may show up shortterm correlation. Alternatively, the analyst can count the number of runs where successive observations are (monotonically) increasing or are (monotonically) decreasing. This may show up trend. Under the null hypothesis of randomness, the expected number of such runs can be found and compared with the observed value, giving tests that are non-parametric or distribution-free in character. Tests of the above types will not be described here, as I have generally found it more convenient to simply examine the correlogram (and possibly the spectral density function) of a given time series to see whether it is random. This can often be done visually, but, if a test is required, then the so-called portmanteau test can be used (see Section 4.7). The latter test can also be used when assessing models by means of a residual analysis, where the residual

< previous page

file:///C:/Documents and Settings/Yang/

page_28

next page >

面/The analysis of time series an introduction/files/page_28.html [5/24/2009 16:51:02]

page_29

< previous page

page_29

next page >

Page 29 of an observation is the difference between the observation and its fitted value from the model. Thus, having fitted a model to a non-random series, one wants to see if the residuals are random, as they should be if the correct model has been fitted. Testing residuals for randomness will be discussed in Section 4.7. 2.9 Handling Real Data We close this chapter with some important comments on how to handle real data. Analysts generally like to think they have ‘good’ data, meaning that the data have been carefully collected with no outliers or missing values. In reality, this does not always happen, so that an important part of the initial examination of the data is to assess the quality of the data and consider modifying them, if necessary. An even more basic question is whether the most appropriate variables have been measured in the first place, and whether they have been measured to an appropriate accuracy. Assessing the structure and format of the data is a key step. Practitioners will tell you that these types of questions often take longer to sort out than might be expected, especially when data come from a variety of sources. It really is important to avoid being driven to bad conclusions by bad data. The process of checking through data is often called cleaning the data, or data editing. It is an essential precursor to attempts at modelling data. Data cleaning could include modifying outliers, identifying and correcting obvious errors and filling in (or imputing) any missing observations. This can sometimes be done using fairly crude devices, such as downweighting outliers to the next most extreme value or replacing missing values with an appropriate mean value. However, more sophisticated methods may be needed, requiring a deeper understanding of time-series models, and so we defer further remarks until Sections 13.7.4 and 13.7.5, respectively. The analyst should also deal with any other known peculiarities, such as a change in the way that a variable is defined during the course of the data-collection process. Data cleaning often arises naturally during a simple preliminary descriptive analysis. In particular, in time-series analysis, the construction of a time plot for each variable is the most important tool for revealing any oddities such as outliers and discontinuities. After cleaning the data, the next step for the time-series analyst is to determine whether trend and seasonality are present. If so, how should such effects be modelled, measured or removed? In my experience, the treatment of such effects, together with the treatment of outliers and missing values, is often more important than any subsequent choices as regards analysing and modelling time-series data. The context of the problem is crucial in deciding how to modify data, if at all, and how to handle trend and seasonality. This explains why it is essential to get background knowledge about the problem, and in particular to clarify the study objectives. A corollary is that it is difficult to make any general remarks or give general recommendations on data cleaning. It is essential to

< previous page

file:///C:/Documents and Settings/Yang/

page_29

next page >

面/The analysis of time series an introduction/files/page_29.html [5/24/2009 16:51:02]

page_30

page_30

< previous page

next page >

Page 30 combine statistical theory with sound common sense and knowledge of the particular problem being tackled. We close by giving the following checklist of possible actions, while noting that the list is not exhaustive and needs to be adapted to the particular problem under consideration. • Do you understand the context? Have the ‘right’ variables been measured? • Have all the time series been plotted? • Are there any missing values? If so, what should be done about them? • Are there any outliers? If so, what should be done about them? • Are there any obvious discontinuities in the data? If so, what does this mean? • Does it make sense to transform any of the variables? • Is trend present? If so, what should be done about it? • Is seasonality present? If so, what should be done about it? Exercises 2.1 The following data show the coded sales of company X in successive 4-week periods over 1995–1998. I II III IV V VI VII VIII IX X XI XII XIII 1995 153 189 221 215 302 223 201 173 121 106 86 87 108 1996 133 177 241 228 283 255 238 164 128 108 87 74 95 1997 145 200 187 201 292 220 233 172 119 81 65 76 74 1998 111 170 243 178 248 202 163 139 120 96 95 53 94 (a) Plot the data. (b) Assess the trend and seasonal effects. 2.2 Sixteen successive observations on a stationary time series are as follows: 1.6, 0.8, 1.2, 0.5, 0.9, 1.1, 1.1, 0.6, 1.5, 0.8, 0.9, 1.2, 0.5, 1.3, 0.8, 1.2 (a) Plot the observations. (b) Looking at the graph plotted in (a), guess an approximate value for the autocorrelation coefficient at lag 1. (c) Plot xt against xt+1, and again try to guess the value of r1. (d) Calculate r1. where a is a constant and is a constant in (0, π), show that 2.3 If . (Hint: You will need to use the trigonometrical results listed in Section 7.2. Using Equation (7.2) it can be shown that that

as N→∞ so that

< previous page

file:///C:/Documents and Settings/Yang/

. Now use the result

page_30

next page >

面/The analysis of time series an introduction/files/page_30.html [5/24/2009 16:51:03]

page_31

< previous page

page_31

next page >

Page 31 2cos A cos B=cos(A+B)+cos(A−B) together with the result that for a suitably chosen N.) 2.4 A computer generates a series of 400 observations that are supposed to be random. The first 10 sample autocorrelation coefficients of the series are r1=0.02, r2=0.05, r3=−0.09, r4=0.08, r5=−0.02, r6= 0.00, r7=0.12, r8=0.06, r9=0.02, r10=−0.08. Is there any evidence of non-randomness? 2.5 Suppose we have a seasonal series of monthly observations {Xt}, for which the seasonal factor at time t is denoted by {St}. Further suppose that the seasonal pattern is constant through time so that St=St−12 for all t. Denote a stationary series of random deviations by {εt}. (a) Consider the model Xt=a+bt+St+εt having a global linear trend and additive seasonality. Show that the seasonal difference operator acts on Xt to produce a stationary series. (b) Consider the model Xt=(a+bt)St+εt having a global linear trend and multiplicative seasonality. Does the operator transform Xt to stationarity? If not, find a differencing operator that does. (Note: As stationarity is not formally defined until Chapter 3, you should use heuristic arguments. A stationary process may involve a constant mean value (that could be zero) plus any linear combination of the stationary series {εt}, but should not include terms such as trend and seasonality.)

< previous page

file:///C:/Documents and Settings/Yang/

page_31

next page >

面/The analysis of time series an introduction/files/page_31.html [5/24/2009 16:51:04]

page_32

< previous page

page_32

next page >

page_32

next page >

Page 32 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_32.html [5/24/2009 16:51:04]

page_33

< previous page

page_33

next page >

Page 33 CHAPTER 3 Some Time-Series Models This chapter introduces various probability models for time series. Some tools for describing the properties of such models are specified and the important notion of stationarity is formally defined. 3.1 Stochastic Processes and Their Properties We concentrate on various types of time-series models that collectively come under the general title of ‘stochastic processes’. Most physical processes in the real world involve a random element in their structure and a stochastic process can be described as ‘a statistical phenomenon that evolves in time according to probabilistic laws’. Examples include the length of a queue, the number of accidents in a particular town in successive months and the air temperature at a particular site on successive days. The word ‘stochastic’, which is of Greek origin, is used to mean ‘pertaining to chance’, and many writers use ‘random process’ as a synonym for stochastic process. Mathematically, a stochastic process may be defined as a collection of random variables that are ordered in time and defined at a set of time points, which may be continuous or discrete. We will denote the random variable at time t by X(t) if time is continuous (usually −∞

Page 34 t. Nevertheless we may regard the observed time series as just one example of the infinite set of time series that might have been observed. This infinite set of time series is sometimes called the ensemble. Every member of the ensemble is a possible realization of the stochastic process. The observed time series can be thought of as one particular realization, and will be denoted by x(t) for (0≤t≤T) if time is continuous, and by xt for t=1,…, N if time is discrete. Time-series analysis is essentially concerned with evaluating the properties of the underlying probability model from this observed time series, even though this single realization is the only one we will ever observe. Many models for stochastic processes are expressed by means of an algebraic formula relating the random variable at time t to past values of the process, together with values of an unobservable ‘error’ process. From this model, it may be possible to specify the joint probability distribution of X(t1),…, X(tk) for any set of times t1,…, tk and any value of k. However, this is rather complicated and is not usually attempted in practice. A simpler, more useful way of describing a stochastic process is to give the moments of the process, particularly the first and second moments that are called the mean and autocovariance function (acv.f.), respectively. The variance function is a special case of the acv.f. These functions will now be defined for continuous time, with similar definitions applying in discrete time. Mean. The mean function µ(t) is defined for all t by

Variance. The variance function σ2(t) is defined for all t by Autocovariance. The variance function alone is not enough to specify the second moments of a sequence of random variables. More generally, we define the acv.f. γ(t1, t2) to be the covariance1 of X(t1) with X(t2), namely Clearly, the variance function is a special case of the acv.f. when t1=t2. Higher moments of a stochastic process may be defined in an obvious way, but are rarely used in practice. 3.2 Stationary Processes An important class of stochastic processes are those that are stationary. A heuristic idea of stationarity was introduced in Section 2.2. A time series is said to be strictly stationary if the joint distribution of X(t1),…, X(tκ) is the same as the joint distribution X (t1+т),…,X (tκ+т) 1 Readers who are unfamiliar with the term ‘covariance’ should read Appendix C. When applied to a sequence of random variables, it is called an autocovariance.

< previous page

file:///C:/Documents and Settings/Yang/

page_34

next page >

面/The analysis of time series an introduction/files/page_34.html [5/24/2009 16:51:05]

page_35

< previous page

page_35

next page >

Page 35 for all t1,…, tk, т. In other words, shifting the time origin by an amount т has no effect on the joint distributions, which must therefore depend only on the intervals between t1, t2 ,…, tk. The above definition holds for any value of k. In particular, if k=1, strict stationarity implies that the distribution of X(t) is the same for all t, so that, provided the first two moments are finite, we have

are both constants, which do not depend on the value of t. Furthermore, if k=2 the joint distribution of X(t1) and X(t2) depends only on the time difference (t2−t1)=т, which is called the lag. Thus the acv.f. γ(t1, t2) also depends only on (t2−t1) and may be written as γ(т), where

is called the autocovariance coefficient at lag т. The size of an autocovariance coefficient depends on the units in which X(t) is measured. Thus, for interpretative purposes, it is helpful to standardize the acv.f. to produce a function called the autocorrelation function (ac.f.), which is defined by This quantity measures the correlation between X(t) and X(t+т). Its empirical counterpart was introduced in Section 2.7. Note that the argument т of γ(т) and ρ(т) is diserete if time is diserete, but continuous if time is continuous. We typically use γ(κ) and ρ(κ) to denote these functions in the discrete-time case. At first sight it may seem surprising to suggest that there are processes for which the distribution of X(t) should be the same for all t. However, readers with some knowledge of stochastic processes will know that there are many processes {X(t)}, which have what is called an equilibrium distribution as t→∞, whereby the probability distribution of X(t) tends to a limit that does not depend on the initial conditions. Thus once such a process has been running for some time, the distribution of X(t) will change very little. Indeed if the initial conditions are specified to be identical to the equilibrium distribution, the process is stationary in time and the equilibrium distribution is then the stationary distribution of the process. Of course the conditional distribution of X(t2) given that X(t1) has taken a particular value, say x(t1), may be quite different from the stationary distribution, but this is perfectly consistent with the process being stationary.

< previous page

file:///C:/Documents and Settings/Yang/

page_35

next page >

面/The analysis of time series an introduction/files/page_35.html [5/24/2009 16:51:06]

page_36

< previous page

page_36

next page >

Page 36 3.2.1 Second-order stationarity In practice it is often useful to define stationarity in a less restricted way than that described above. A process is called second-order stationary (or weakly stationary) if its mean is constant and its acv.f. depends only on the lag, so that and No requirements are placed on moments higher than second order. By letting т=0, we note that the form of a stationary acv.f. implies that the variance, as well as the mean, is constant. The definition also implies that both the variance and the mean must be finite. This weaker definition of stationarity will generally be used from now on, as many of the properties of stationary processes depend only on the structure of the process as specified by its first and second moments. One important class of processes where this is particularly true is the class of normal processes where the joint distribution of X(t1),…, X(tk) is multivariate normal for all t1 ,…, tk. The multivariate normal distribution is completely characterized by its first and second moments, and hence by µ(t) and γ(t1, t2), and so it follows that second-order stationarity implies strict stationarity for normal processes. However, µ and γ (т) may not adequately describe stationary processes, which are very ‘non-normal’. 3.3 Some Properties of the Autocorrelation Function We have already noted in Section 2.7 that the sample autocorrelation coefficients of an observed time series are an important set of statistics for describing the time series. Similarly the (theoretical) ac.f. of a stationary stochastic process is an important tool for assessing its properties. This section investigates the general properties of the ac.f. Suppose a stationary stochastic process X(t) has mean µ, variance σ2, acv.f. γ(т) and ac.f. ρ(т). Then Note that ρ(0)=1. Property 1: The ac.f. is an even function of lag, so that ρ(т)=ρ(−т). This property simply says that the correlation between X(t) and X(t+т) is the same as that between X(t) and X (t−т). The result is easily proved using γ(т)=ρ(т)σ2 by

< previous page

file:///C:/Documents and Settings/Yang/

page_36

next page >

面/The analysis of time series an introduction/files/page_36.html [5/24/2009 16:51:07]

page_37

< previous page

page_37

next page >

Page 37

Property 2: . This is the ‘usual’ property of correlation, namely, that it lies between ±1. It is proved by noting that for any constants

1,

2, since a variance is always non-negative. The variance is equal to

When

1= 2=1, we find

When

1=1,

2=−1, we find

, so that , so that

. .

as required. Thus The standardized nature of a correlation coefficient means that the value of ρ(т) does not depend on the units in which the time series is measured, as can readily be demonstrated by multiplying all values in a series by the same constant and showing that the resulting autocorrelations are unchanged. Property 3: The ac.f. does not uniquely identify the underlying model. Although a given stochastic process has a unique covariance structure, the converse is not in general true. It is usually possible to find many normal and non-normal processes with the same ac.f. and this creates further difficulty in interpreting sample ac.f.s. Jenkins and Watts (1968, p. 170) give an example of two different stochastic processes, which have exactly the same ac.f. Even for stationary normal processes, which are completely determined by the mean, variance and ac.f., we will see in Section 3.4.3 that a requirement, called the invertibility condition, is needed to ensure uniqueness. 3.4 Some Useful Models This section describes several different types of stochastic processes that may be appropriate when setting up a model for a time series. 3.4.1 Purely random processes A discrete-time process is called a purely random process if it consists of a sequence of random variables, {Zt}, which are mutually independent and identically distributed. We normally further assume that the random variables are normally distributed with mean zero and variance . From the definition it follows that the process has constant mean and variance. Moreover, the independence assumption means that

< previous page

file:///C:/Documents and Settings/Yang/

page_37

next page >

面/The analysis of time series an introduction/files/page_37.html [5/24/2009 16:51:08]

page_38

< previous page

page_38

next page >

Page 38 This means that different values are uncorrelated so that the ac.f. is given by

As the mean and acv.f. do not depend on time, the process is second-order stationary. In fact the independence assumption implies that the process is also strictly stationary. Purely random processes are useful in many situations, particularly as building blocks for more complicated processes such as moving average processes—see Section 3.4.3. are mutually uncorrelated, rather than Some authors prefer to make the weaker assumption that the independent. This is adequate for linear, normal processes, but the stronger independence assumption is needed when considering non-linear models (see Chapter 11). Note that a purely random process is sometimes called white noise, particularly by engineers. The possibility of defining a continuous-time purely random process is discussed in Section 3.4.8. 3.4.2 Random walks Suppose that {Zt} is a discrete-time, purely random process with mean µ and variance said to be a random walk if The process is customarily started at zero when t=0, so that

. A process {Xt} is (3.1)

and

Then we find that E(Xt)=tµ and that Var(Xt)= . As the mean and variance change with t, the process is non-stationary. However, it is interesting to note that the first differences of a random walk, given by form a purely random process, which is therefore stationary. The best-known examples of time series, which behave like random walks, are share prices on successive days. A model, which often gives a good approximation to such data, is share price on day t=share price on day (t−1)+random error 3.4.3 Moving average processes Suppose that {Zt} is a purely random process with mean zero and variance to be a moving average process of order q

< previous page

file:///C:/Documents and Settings/Yang/

page_38

. Then a process {Xt} is said

next page >

面/The analysis of time series an introduction/files/page_38.html [5/24/2009 16:51:08]

page_39

page_39

< previous page

next page >

Page 39 (abbreviated to an MA(q) process) if (3.2)

where {βi} are constants. The Zs are usually scaled so that β0=1. We find immediately that

since the Zs are independent. We also have

since

As γ(k) does not depend on t, and the mean is constant, the process is second-order stationary for all values of the {βi}. Furthermore, if the Zs are normally distributed, then so are the Xs, and we have a strictly stationary normal process. The ac.f. of the above MA(q) process is given by

Note that the ac.f. ‘cuts off’ at lag q, which is a special feature of MA processes. In particular, the MA(1) process with β0=1 has an ac.f. given by

No restrictions on the {βi} are required for a (finite-order) MA process to be stationary, but it is generally desirable to impose restrictions on the {βi} to ensure that the process satisfies a condition called invertibility. This

< previous page

file:///C:/Documents and Settings/Yang/

page_39

next page >

面/The analysis of time series an introduction/files/page_39.html [5/24/2009 16:51:09]

page_40

page_40

< previous page

next page >

Page 40 condition may be explained in the following way. Consider the following first-order MA processes:

It can easily be shown that these two different processes have exactly the same ac.f. (Are you surprised? Then check ρ(k) for models A and B.) Thus we cannot identify an MA process uniquely from a given ac.f., as we noted in Property 3 in Section 3.3. Now, if we ‘invert’ models A and B by expressing Zt in terms of Xt, Xt −1,…, we find by successive substitution that

If |θ|

面/The analysis of time series an introduction/files/page_43.html [5/24/2009 16:51:12]

page_44

< previous page

page_44

next page >

Page 44 The relationship between the as and the βs may then be found. Having expressed Xt as an MA process, it follows that E(Xt)=0. The variance is finite provided that for stationarity. The acv.f. is given by

converges, and this is a necessary condition

A sufficient condition for this to converge, and hence for stationarity, is that ∑|βi| converges. We can in principle find the ac.f. of the general-order AR process using the above procedure, but the {βi} may be algebraically hard to find. The alternative simpler way is to assume the process is stationary, multiply through Equation (3.3) by Xt−k, take expectations and divide by finite. Then, using the fact that ρ(k)=ρ(−k) for all k, we find

assuming that the variance of Xt is

This set of equations is called the Yule-Walker equations after G.U.Yule and Sir Gilbert Walker. It is a set of difference equations and has the general solution where {πi} are the roots of the so-called auxiliary equation The constants {Ai} are chosen to satisfy the initial conditions depending on ρ(0)=1, which means that ΣAi=1. The first (p−1) Yule-Walker equations provide (p−1) further restrictions on the {Ai} using ρ(0)=1 and ρ(k)=ρ (−k). From the general form of ρ(k), it is clear that ρ(k) tends to zero as k increases provided that |πi|

Page 45 from which it can be shown (Exercise 3.6) that the stationarity region is the triangular region satisfying

The roots are real if

, in which case the ac.f. decreases exponentially with k, but the

roots are complex if , in which case the ac.f. turns out to be a damped sinusoidal wave. (See Example 3.1 at the end of this section.) where the constants A1, A2 are also real and

When the roots are real, we have may be found as follows. Since ρ(0)=1, we have while the first Yule-Walker equation gives

This equation may be solved to give ρ(1)=α1/(1−α2), which in turn must equal Hence we find

and this enables us to write down the general form of the ac.f. of an AR(2) process with real roots. For complex roots, the general solution can also be found and an example is given below in Example 3.1. AR processes have been applied to many situations in which it is reasonable to assume that the present value of a time series depends linearly on the immediate past values together with a random error. For simplicity we have only considered processes with mean zero, but non-zero means may be dealt with by rewriting Equation (3.3) in the form This does not affect the ac.f. Example 3.1 Consider the AR(2) process given by

Is this process stationary? If so, what is its ac.f.? In order to answer the first question we find the roots of Equation (3.5), which, in this case, is

< previous page

file:///C:/Documents and Settings/Yang/

page_45

next page >

面/The analysis of time series an introduction/files/page_45.html [5/24/2009 16:51:14]

page_46

page_46

< previous page

next page >

Page 46 The roots of this equation (regarding B as a variable) are complex, namely, 1±i. As the rnodulus of both roots exceeds one, the roots are both outside the unit circle and so the process is stationary. In order to find the ac.f. of the process, we use the first Yule-Walker equation to give

giving ρ(1)=2/3. For k≥2, the Yule-Walker equations are

We could use these equations to find ρ(2), then ρ(3) and so on by successive substitution, but it is easier to find the general solution by solving the set of Yule-Walker equations as a set of difference equations. The general form of the above Yule-Walker equation has the auxiliary equation

with roots y=(1±i)/2 They may be rewritten as

or as

. Since

is less than zero, and the roots are complex, the ac.f. is a damped sinusoidal wave. Using ρ(0)=1 and ρ(1)=2/3, some messy trigonometry and algebra involving complex numbers gives

for k=0, 1, 2,…. Note the values of the ac.f. are all real, even though the roots of the auxiliary equation are complex. 3.4.5 Mixed ARMA models A useful class of models for time series is formed by combining MA and AR processes. A mixed autoregressive/moving-average process containing p AR terms and q MA terms is said to be an ARMA process of order (p, q). It is given by (3.6)

Using the backward shift operator B, Equation (3.6) may be written in the form (3.6a) where

(B), θ(B) are polynomials of order p, q, respectively, such that

< previous page

file:///C:/Documents and Settings/Yang/

page_46

next page >

面/The analysis of time series an introduction/files/page_46.html [5/24/2009 16:51:14]

page_47

page_47

< previous page

next page >

Page 47 and The conditions on the model parameters to make the process stationary and invertible are the same as for a pure AR or pure MA process, namely, that the values of {αi}, which make the process stationary, are such that the roots of lie outside the unit circle, while the values of {βi}, which make the process invertible, are such that the roots of lie outside the unit circle. It is straightforward in principle, though algebraically rather tedious, to calculate the ac.f. of an ARMA process, but this will not be discussed here. (See Exercise 3.11, and Box et al., 1994, Section 3.4.) The importance of ARMA processes lies in the fact that a stationary time series may often be adequately modelled by an ARMA model involving fewer parameters than a pure MA or AR process by itself. This is an early example of what is often called the Principle of Parsimony. This says that we want to find a model with as few parameters as possible, but which gives an adequate representation of the data at hand. It is sometimes helpful to express an ARMA model as a pure MA process in the form (3.6b) where is the MA operator, which may be of infinite order. The weights, { }, can be useful in calculating forecasts (see Chapter 5) and in assessing the properties of a model (e.g. see Exercise 3.11). By comparison with Equation (3.6a), we see that helpful to express an ARMA model as a pure AR process in the form

. Alternatively, it can be (3.6c)

where π(B)= (B)/θ(B). By convention we write AR model is in the form

, since the natural way to write an

By comparing (3.6b) and (3.6c), we see that weights or π weights may be obtained directly by division or by equating powers of B in an equation The such as

< previous page

file:///C:/Documents and Settings/Yang/

page_47

next page >

面/The analysis of time series an introduction/files/page_47.html [5/24/2009 16:51:15]

page_48

page_48

< previous page

next page >

Page 48

Example 3.2 Find the

weights and π weights for the ARMA(1, 1) process given by

Here and θ(B)=(1−0.3B). It follows that the process is stationary and invertible, because both equations have roots greater than one (or are outside the unit circle). Then

Hence Similarly we find Note that both the weights and π weights die away quickly, and this also indicates a stationary, invertible process. 3.4.6 Integrated ARMA (or ARIMA) models In practice most time series are non-stationary. In order to fit a stationary model, such as those discussed in Sections 3.4.3–3.4.5, it is necessary to remove non-stationary sources of variation. If the observed time series is non-stationary in the mean, then we can difference the series, as suggested in Section 2.5.3. in Equation (3.6), then we have Differencing is widely used for econometric data. If Xt is replaced by a model capable of describing certain types of non-stationary series. Such a model is called an ‘integrated’ model because the stationary model that is fitted to the differenced data has to be summed or ‘integrated’ to provide a model for the original non-stationary data. Writing the general autoregressive integrated moving average (ARIMA) process is of the form (3.7)

By analogy with Equation (3.6a), we may write Equation (3.7) in the form (3.7a)

or

(3.7b) Thus we have an ARMA(p, q) model for Wt, while the model in Equation (3.7b), describing the dth differences of Xt, is said to be an ARIMA process of order (p, d, q). The model for Xt is clearly non-stationary, as the AR operator (B) (1−B)d has d roots on the unit circle (since putting B=1 makes the

< previous page

file:///C:/Documents and Settings/Yang/

page_48

next page >

面/The analysis of time series an introduction/files/page_48.html [5/24/2009 16:51:16]

page_49

< previous page

page_49

next page >

Page 49 AR operator equal to zero). In practice, first differencing is often found to be adequate to make a series stationary, and so the value of d is often taken to be one. Note that the random walk can be regarded as an ARIMA(0, 1, 0) process. ARIMA models can be generalized to include seasonal terms, as discussed in Section 4.6. 3.4.7 The general linear process Expanding Equation (3.6b), a general class of processes may be written as an MA process, of possibly infinite order, in the form

(3.8) A sufficient condition for the sum to converge, and hence for the process to be stationary, is that , in which case we also have . A stationary process described by Equation (3.8) is sometimes called a general linear process, although some authors use this term when the Z’s are merely uncorrelated, rather than independent. Although the model is in the form of an MA process, it is interesting to note that stationary AR and ARMA processes can also be expressed as a general linear process using the duality between AR and MA processes arising, for example, from Equations (3.6b) and (3.6c). 3.4.8 Continuous processes So far, we have only considered stochastic processes in discrete time, because models of this type are nearly always used by the statistician in practice. This subsection2 gives a brief introduction to processes in continuous time. The latter have been used in some applications, notably in the study of control theory by electrical engineers. Here we are mainly concerned with indicating some of the mathematical problems that arise when time is continuous. By analogy with a discrete-time purely random process, we might expect to define a continuous-time purely random process as having an ac.f. given by

However, this is a discontinuous function, and it can be shown that such a process would have an infinite variance and hence be a physically unrealizable phenomenon. Nevertheless, some processes that arise in practice do appear to have the properties of continuous-time white noise even when sampled at quite small discrete intervals. We may approximate continuous-time white noise by considering a purely random process in discrete time at intervals t, and letting Δt→0, or by considering a process in continuous time with ac.f. and letting →∞ so that the ac.f. decays very quickly. 2 This subsection may be omitted at a first reading.

< previous page

file:///C:/Documents and Settings/Yang/

page_49

next page >

面/The analysis of time series an introduction/files/page_49.html [5/24/2009 16:51:17]

page_50

page_50

< previous page

next page >

Page 50 As an example of the difficulties involved with continuous-time processes, we briefly consider a first-order AR process in continuous time. A first-order AR process in diserete time may readily be rewritten3 in terms of Xt, Xt and Zt, rather than Xt, Xt−1 and Zt. Now differencing in diserete time corresponds to differentiation in continuous time, so that an apparently natural way of trying to define a first-order AR process in continuous time is by means of the general equation (3.9) where a is a constant, and Z(t) denotes continuous white noise. However, as {Z(t)} cannot physically exist, it is more legitimate to write Equation (3.9) in a form involving infinitesimal small changes as (3.10) where {U(t)} is a process with orthogonal increments such that the random fsvariables [U(t2)−U(t1)] and [U (t4)−U(t3)] are uncorrelated for any two non-overlapping intervals (t1, t2) and (t3, t4). In the theory of Brownian motion, Equation (3.10) arises in the study of the Ornstein-Uhlenbeck model and is sometimes called the Langevin equation. It can be shown that the process {X(t)} defined in Equation (3.10) has ac.f. which is similar to the ac.f. of a first-order AR process in discrete time in that both decay exponentially. However, the rigorous study of continuous processes, such as that defined by Equation (3.10), requires considerable mathematical machinery, including a knowledge of stochastic integration. Thus we will not pursue this topic here. The reader is referred to a specialist book on probability theory such as Rogers and Williams (1994). 3.5 The Wold Decomposition Theorem This section4 gives a brief introduction to a famous result, called the Wold decomposition theorem, which is of mainly theoretical interest. This essentially says that any discrete-time stationary process can be expressed as the sum of two uncorrelated processes, one purely deterministic and one purely indeterministic. The terms ‘deterministic’ and ‘indeterministic’ are defined as follows. We can regress Xt on (Xt−q, Xt−q−1,…) and denote the residual variance from the resulting linear regression model by that, as q increases,

. As

≤Var(Xt), it is clear

is a non-decreasing bounded sequence and therefore tends to a limit as q→∞. If

then linear regression on the remote past is useless for prediction purposes, and we say that {Xt} is purely indeterministic. Stationary linear stochastic processes, such as AR, MA and ARMA processes, are of this type. However, if 3 For example, if Xt=αXt−1+Zt,then 4 This section may be omitted at a first reading.

< previous page

file:///C:/Documents and Settings/Yang/

.

page_50

next page >

面/The analysis of time series an introduction/files/page_50.html [5/24/2009 16:51:18]

page_51

< previous page

page_51

next page >

Page 51 is zero, then the process can be forecast exactly, and we say that {Xt} is purely deterministic. Note that this definition of ‘deterministic’ involves linear models and bears rather little relation to the usual meaning of the word as used in philosophical debate or common parlance. The Wold decomposition theorem also says that the purely indeterministic component can be written as a linear sum of a sequence of uncorrelated random variables, say {Zt}. This has the same form as the general linear process in Equation (3.8) except that the Zs are merely uncorrelated, rather than independent. Of course, if the Zs are normally distributed, then zero correlation implies independence and we really do have a general linear process. While the concept of a purely indeterministic process may sometimes be helpful, I have rarely found the Wold decomposition itself of much assistance. For a linear purely indeterministic process, such as an AR or ARMA model, it is inappropriate to try to model it as an MA(∞) process, as there will be too many parameters to estimate (although the MA(∞) form may be helpful for computing forecast error variances—see Section 5.2.4). Rather, we normally seek a model that gives an adequate approximation to the given data with as few parameters as possible, and that is where an ARMA representation can help. For processes generated in a non-linear way, the Wold decomposition is usually of even less interest, as the best predictor may be quite different from the best linear predictor. Consider, for example, a sinusoidal process (see Exercise 3.14), such as (3.11) where g is a constant, is a constant in (0, π) called the frequency of the process, and θ is a random variable, called the phase, which is uniformly distributed on (0, 2π) but which is fixed for a single realization. Note that we must include the term θ so that If this is not done, Equation (3.11) would not define a stationary process. As θ is fixed for a single realization, once enough values of Xt have been observed to evaluate θ, all subsequent values of Xt are completely determined. It is then obvious that (3.11) defines a deterministic process. However, it is not ‘purely deterministic’ as defined above because (3.11) is not linear. In fact the process is ‘purely indeterministic’ using a linear predictor, even though it is deterministic using an appropriate non-linear predictor. These are deep waters!

< previous page

file:///C:/Documents and Settings/Yang/

page_51

next page >

面/The analysis of time series an introduction/files/page_51.html [5/24/2009 16:51:18]

page_52

page_52

< previous page

next page >

Page 52 Exercises In all the following questions, {Zt} is a discrete-time, purely random process, such that E(Zt)=0, Var(Zt) = , and successive values of Zt are independent so that Cov(Zt, Zt+k)=0, k≠0. Exercise 3.14 is harder than the other exercises and may be omitted. 3.1 Show that the ac.f. of the second-order MA process is given by

3.2 Consider the MA(m) process, with equal weights 1/(m+1) at all lags (so it is a real moving average), given by

Show that the ac.f. of this process is

3.3 Consider the infinite-order MA process {Xt}, defined by where C is a constant. Show that the process is non-stationary. Also show that the series of first differences {Yt} defined by is a first-order MA process and is stationary. Find the ac.f. of {Yt}. 3.4 Find the ac.f. of the first-order AR process defined by Plot ρ(k) for k=−6, −5,…, −1, 0, +1,…, +6. 3.5 If Xt=µ+Zt+βZt−1, where µ is a constant, show that the ac.f. does not depend on µ. 3.6 Find the values of 1, 2, such that the second-order AR process defined by is stationary. If

< previous page

file:///C:/Documents and Settings/Yang/

, show that the ac.f. of Xt is given by

page_52

next page >

面/The analysis of time series an introduction/files/page_52.html [5/24/2009 16:51:19]

page_53

page_53

< previous page

next page >

Page 53 3.7 Explain what is meant by a weakly (or second-order) stationary process, and define the ac.f. ρ(u) for such a process. Show that ρ(u)=ρ(−u) and that |ρ(u)|≤1. Show that the ac.f. of the stationary second-order AR process

is given by

3.8 Suppose the process {Xt} is stationary and has acv.f. γX(k). A new stationary process {Yt} is defined by Yt=Xt−Xt−1. Obtain the acv.f. of {Yt} in terms of γX(k) and find γY(k) when γX(k)= |k|. 3.9 For each of the following models: (a) Xt=0.3Xt−1+Zt (b) Xt=Zt−1.3Zt−1+0.4Zt−2 (c) Xt=0.5Xt−1+Zt−1.3Zt−1+0.4Zt−2 express the model using B notation and determine whether the model is stationary and/or invertible. For model (a) find the equivalent MA representation. 3.10 Suppose that a stationary process, {Xt}, can be represented in the form . The autocovariance generating function is defined by

where γX(k) is the autocovariance coefficient of Xt at lag k, and s is a dummy variable. Show that 3.11 Show that the ac.f. of the ARMA(1, 1) model

. (Hint: Equate coefficients of sk.)

where |α|

Page 54 (d) Evaluate the first four π weights of the model when expressed as an AR(∞) model. Is the behaviour of the and π weights what you would expect, given the type of model? 3.13 Show that the AR(2) process is stationary provided −1

面/The analysis of time series an introduction/files/page_54.html [5/24/2009 16:51:21]

page_55

page_55

< previous page

next page >

Page 55 CHAPTER 4 Fitting Time-Series Models in the Time Domain Chapter 3 introduced several different types of probability models that may be used to describe time series. This chapter discusses the problem of fitting a suitable model to an observed time series. We restrict attention to the discrete-time case and the major diagnostic tool used in this chapter is the sample autocorrelation function (ac.f.). Inference based on this function is often called an analysis in the time domain. 4.1 Estimating Autocovariance and Autocorrelation Functions We have already noted in Section 3.3 that the theoretical ac.f. is an important tool for describing the properties of a stationary stochastic process. In Section 2.7 we heuristically introduced the sample ac.f. of an observed time series, and this is an intuitively reasonable estimate of the theoretical ac.f., provided the series is stationary. This section investigates the properties of the sample ac.f. more closely. Let us look first at the autocovariance function (acv.f.). Suppose we have N observations on a stationary process, say x1, x2,…, xN. Then the sample autocovariance coefficient at lag k (see Equation (2.7)) given by

(4.1) is the usual estimator for the theoretical autocovariance coefficient γ(k) at lag k. It can be shown (e.g. Priestley, 1981, Chapter 5) that this estimator is biased, but that the bias is of order 1/N. Moreover so that the estimator is asymptotically unbiased. It can also be shown that

(4.2) When m=k, Equation (4.2) gives us the variance of ck. Equation (4.2) also highlights the fact that successive values of ck may be (highly) correlated and this increases the difficulty of interpreting the correlogram.

< previous page

file:///C:/Documents and Settings/Yang/

page_55

next page >

面/The analysis of time series an introduction/files/page_55.html [5/24/2009 16:51:21]

page_56

< previous page

page_56

next page >

Page 56 The estimator (4.1) may be compared with the alternative estimator

(4.3) where the divisor is (N−k), rather than N. This is used by some authors because it is claimed to have a smaller bias, though this is, in fact, not always the case. Indeed some authors call the ‘unbiased estimator’ even though it is biased when, as here, the population mean, µ, is unknown and replaced by the sample mean, , as is usually the case in practice. It turns out that the earlier estimator in Equation (4.1) leads to a sample acv.f. having a useful property called positive semi-definiteness, which means that its finite Fourier transform is non-negative, among other consequences. This property is useful in estimating the spectrum (see Chapter 7) and so we generally prefer to use Equation (4.1), rather than (4.3). Note that when k=0, we get the same estimate of variance using both formulae, but that this estimate, involving , will generally still be biased (Percival, 1993). Having estimated the acv.f., we then take (4.4) as an estimator for ρ(k). The properties of rk are rather more difficult to find than those of ck because it is the ratio of two random variables. It can be shown that rk is generally biased. A general formula for the variance of rk is given by Kendall et al. (1983, Section 48.1) and depends on all the autocorrelation coefficients of the process. We will only consider the properties of rk when sampling from a purely random process, when all the theoretical autocorrelation coefficients are zero except at lag zero. These results help us to decide if the observed values of rk from a given time series are significantly different from zero. Suppose that x1,…, xN are observations on independent and identically distributed random variables with arbitrary mean. Then it can be shown (Kendall et al., 1983, Chapter 48) that

and that rk is asymptotically normally distributed under weak conditions. Thus having plotted the correlogram, as described in Section 2.7, we can check for randomness by plotting approximate 95% confidence limits at , which are often further approximated to . Observed values of rk which fall outside these limits are ‘significantly’ different from zero at the 5% level. However, when interpreting a correlogram, it must be remembered that the overall probability of getting at least one coefficient outside these limits, given that the data really are random, increases with the number of coefficients plotted. For example, if the first 20 values of rκ are plotted, then one expects one ‘significant’ value (at the 5% level) on average even if the data really are random. Thus, if only one or two coefficients are

< previous page

file:///C:/Documents and Settings/Yang/

page_56

next page >

面/The analysis of time series an introduction/files/page_56.html [5/24/2009 16:51:22]

page_57

< previous page

page_57

next page >

Page 57 ‘significant’, the size and lag of these coefficients must be taken into account when deciding if a set of data is random. A single coefficient just outside the ‘null’ 95% confidence limits may be ignored, but two or three values well outside the ‘null’ limits will be taken to indicate non-randomness. A single ‘significant’ coefficient at a lag which has some physical interpretation, such as lag 1 or a lag corresponding to seasonal variation, will also provide plausible evidence of non-randomness. Figure 4.1 shows the correlogram for 100 observations, generated on a computer, which are supposed to be independent normally distributed variables. The ‘null’ 95% confidence limits are approximately ±0.2. We see that 2 of the first 20 values of rk are just ‘significant’. However, they occur at apparently arbitrary lags (namely, 12 and 17). Thus we conclude that there is no firm evidence to reject the hypothesis that the observations are independently distributed. This in turn means that the way that the computer generates ‘random’ numbers appears to be satisfactory.

Figure 4.1 The correlogram of 100 ‘independent’ normally distributed observations. The dotted lines are at ±0.2. 4.1.1 Using the correlogram in modelling We have already given some general advice on interpreting correlograms in Section 2.7.2, while their use in assessing randomness was considered above. The correlogram is also helpful in trying to identify a suitable class of models for a given time series, and, in particular, for selecting the most appropriate type of autoregressive integrated moving average (ARIMA) model. A correlogram like that in Figure 2.3, where the values of rk do not come down to zero reasonably quickly, indicates non-stationarity and so the series needs to be differenced. For stationary series, the correlogram is compared with the theoretical ac.f.s of different ARMA processes in order to choose the one which seems to be the ‘best’ representation. In Section 3.4.3, we saw that the ac.f. of an MA(q) process is easy to recognize as it ‘cuts

< previous page

file:///C:/Documents and Settings/Yang/

page_57

next page >

面/The analysis of time series an introduction/files/page_57.html [5/24/2009 16:51:23]

page_58

< previous page

page_58

next page >

Page 58 off’ at lag q. However, the ac.f. of an AR(p) process is rather more difficult to categorize, as it is a mixture of damped exponentials and sinusoids and dies out slowly (or attenuates). The ac.f. of a mixed ARMA model will also generally attenuate rather than ‘cut off’. Suppose, for example, that we find that r1 is significantly different from zero but that subsequent values of rk are all close to zero. Then an MA(1) model is indicated because its theoretical ac.f. is of this form. Alternatively, if r1, r2, r3,… appear to be decreasing exponentially, then an AR(1) model may be appropriate. The interpretation of correlograms is one of the hardest aspects of time-series analysis and practical experience is a ‘must’. Inspection of the partial autocorrelation function (see Section 4.2.2) can provide additional help. 4.1.2 Estimating the mean The first inferential problem considered in most statistics texts is the estimation of a population mean using a sample mean. However, in time-series analysis, the topic is often overlooked and omitted, because of the special problems that relate to using a time-series sample mean as an estimate of some underlying population mean. Although we have used the sample mean in computing the sample acv.f. and ac.f.—see Equations (4.1) and (4.4)—we have already noted, at the start of Chapter 2, that the sample mean is a potentially misleading summary statistic unless all systematic components have been removed. Thus the sample mean should only be considered as a summary statistic for data thought to have come from a stationary process. Even when this is so, it is important to realize that the statistical properties of the sample mean are quite different from those that usually apply. Suppose we have time-series data {xi for i=1,…, N} from a stationary process having mean µ, variance σ2 and theoretical ac.f. ρ(k). Let

denote the sample mean value expressed as a random

variable. Then the usual result for independent observations is that correlated observations, it can be shown (e.g. Priestley, 1981, p. 319) that

. However, for

and this quantity can differ considerably from σ2/N when autocorrelations are substantial. In particular, for an AR(1) process with parameter α, the formula reduces to for large N. Put another way, the equivalent number of independent observations is N(1−α)/(1+α). When α>0 (the usual case), the positive autocorrelation means that there is less information in the data than might be expected in regard to estimating the mean. On the other hand, when a

面/The analysis of time series an introduction/files/page_58.html [5/24/2009 16:51:24]

page_59

< previous page

page_59

next page >

Page 59 Another way that estimating a time-series mean differs from the rest of Statistics is that any results depend upon the underlying process having a property called ergodicity. The difficult ideas involved are introduced briefly in the next optional subsection. 4.1.3 Ergodicity This subsection1 gives a brief introduction to the idea of ergodicity. It is not immediately obvious that one can obtain consistent estimates of the properties of a stationary process from a single finite realization. This requires that an average over time for a single time series, like , can be used to estimate the ensemble properties of the underlying process at a particular time, like E(Xt). This means that, as we only ever get one observation actually at time t, the properties of the series at time t are estimated using data collected at other time points. Fortunately, some theorems, called ergodic theorems, have been proved, showing that for most stationary processes, which are likely to be met in practice, the sample moments of an observed time series do indeed converge to the corresponding population moments. In other words, a time average like converges to a population quantity like E(Xt) as N→∞. A sufficient condition for this to happen is that ρk→0 as k→∞ and the process is then called ‘ergodic in the mean’. We will not pursue the topic here but rather simply assume that appropriate ergodic properties are satisfied when estimating the properties of stationary processes. More details may be found, for example, in Hamilton (1994, p. 46). 4.2 Fitting an Autoregressive Process Having estimated the ac.f. of a given time series, we should have some idea as to which stochastic process will provide a suitable model. If an autoregressive (AR) process is thought to be appropriate, there are two related questions: (1) What is the order of the process? (2) How can we estimate the parameters of the process? It is convenient to consider the second question first. 4.2.1 Estimating parameters of an AR process Suppose we have an AR process of order p, with mean µ, given by (4.5) Given N observations x1,…, xN, the parameters µ, α1,…, αp may be estimated by least squares by minimizing

1 This subsection may be omitted at a first reading.

< previous page

file:///C:/Documents and Settings/Yang/

page_59

next page >

面/The analysis of time series an introduction/files/page_59.html [5/24/2009 16:51:24]

page_60

page_60

< previous page

next page >

Page 60 with respect to µ, α1,…, αp. If the Zt process is normal, then the least squares estimates are also maximum likelihood estimates (Jenkins and Watts, 1968, Section 5.4) conditional on the first p values in the time series being fixed. In the first-order case, with p=1, we find (see Exercise 4.1) (4.6)

and

(4.7) where

are the means of the first and last (N−1) observations. Now

and so we have approximately that (4.8) This approximate estimator is intuitively appealing and is nearly always preferred to Equation (4.6). Substituting this estimator into Equation (4.7) gives

(4.9) It is interesting to note that this is exactly the same estimator that would arise if we were to treat the autoregressive equation as the ‘independent’ variable (which of course it isn’t). In fact H. as an ordinary regression with B.Mann and A.Wald showed as long ago as 1943 that, asymptotically, much classical regression theory can be applied to AR models. A further approximation, which is often used, is obtained by changing the denominator of (4.9) slightly to

1 is also intuitively appealing since r1 is an estimator for ρ(1) and ρ(1)=α1 This approximate estimator for for a first-order AR process. A confidence interval for α1 may be obtained from the fact that the asymptotic standard error of

is

, although the confidence interval will not be symmetric for

away from zero. When α1=0, the standard error of whether lies within

< previous page

file:///C:/Documents and Settings/Yang/

is

page_60

and so a test for α1=0 is given by seeing

next page >

面/The analysis of time series an introduction/files/page_60.html [5/24/2009 16:51:25]

page_61

page_61

< previous page

next page >

Page 61 . This is equivalent to the test for ρ(1)=0 already noted in Section 4.1. the range For a second-order AR process, where p=2, similar approximations may be made to give (4.10) (411) These results are also intuitively reasonable in that if we fit a second-order model to what is really a firstorder process, then as α2=0 we have and so . Thus Equations (4.10) and (4.11) become and , as we would hope. The coefficient is called the (sample) partial autocorrelation coefficient of order two, as it measures the excess correlation between observations two steps apart (e.g. between Xt and Xt+2) not accounted for by the autocorrelation at lag 1, namely, r1—see Section 4.2.2 below. In addition to point estimates of α1 and α2 it is also possible to find a confidence region in the (α1, α2) plane (Jenkins and Watts, 1968, p. 192). Higher-order AR processes may also be fitted by least squares in a straightforward way. Two alternative approximate methods are commonly used, which both involve taking to the model

. The first method fits the data

treating it as if it were an ordinary regression model. A standard multiple regression computer program may be used with appropriate modification. The second method involves substituting the sample autocorrelation coefficients into the first p Yule-Walker equations (see Section 3.4.4) and solving for ( where

,…,

). In matrix form these equations are (4.12) and

is a (p×p) matrix. For N reasonably large, both methods will give estimated values ‘very close’ to the exact least squares estimates for which is close to, but not necessarily equal to, . However, for smaller N (e.g. less than about 50), the Yule-Walker estimates are not so good, especially when the parameter values are such that the model is ‘close’ to being non-stationary.

< previous page

file:///C:/Documents and Settings/Yang/

page_61

next page >

面/The analysis of time series an introduction/files/page_61.html [5/24/2009 16:51:26]

page_62

< previous page

page_62

next page >

Page 62 4.2.2 Determining the order of an AR process It is often difficult to assess the order of an AR process from the sample ac.f. alone. For a first-order process the theoretical ac.f. decreases exponentially and the sample function should have a similar shape. However, for higher-order processes, the ac.f. may be a mixture of damped exponential or sinusoidal functions and is difficult to identify. One approach is to fit AR processes of progressively higher order, to calculate the residual sum of squares for each value of p and to plot this against p. It may then be possible to see the value of p where the curve ‘flattens out’ and the addition of extra parameters gives little improvement in fit. Another aid to determining the order of an AR process is the partial autocorrelation function, which is defined as follows. When fitting an AR(p) model, the last coefficient αp will be denoted by πp and measures the excess correlation at lag p which is not accounted for by an AR(p−1) model. It is called the pth partial autocorrelation coefficient and, when plotted against p, gives the partial ac.f. The first partial autocorrelation coefficient π1 is simply equal to ρ(1), and this is equal to α1 for an AR(1) process. It can be shown (see Exercise 4.3 and the discussion following Equation (4.11) above) that the second partial correlation coefficient is [ρ(2)−ρ(1)2]/[1−ρ(1)2], and we note that this is zero for an AR(1) process where ρ(2)=ρ(1)2. The sample partial ac.f. is usually estimated by fitting AR processes of successively higher order and taking when an AR(1) process is fitted, taking

when an AR(2) process is fitted, and so on.

Values of j, which are outside the range , are significantly different from zero at the 5% level. It can be shown that the partial ac.f. of an AR(p) process ‘cuts off’ at lag p so that the ‘correct’ order is assessed as that value of p beyond which the sample values of {πj} are not significantly different from zero. In contrast, the partial ac.f. of an MA process will generally attenuate (or die out slowly). Thus the partial ac. f. has reverse properties to those of the ac.f. in regard to identifying AR and MA models. Note that McCullough (1998) recommends that Yule-Walker estimates should not be used to estimate partial autocorrelations. Some additional tools to aid model identification are discussed in Section 13.1. 4.3 Fitting a Moving Average Process Suppose now that a moving average (MA) process is thought to be an appropriate model for a given time series. As for an AR process, we have two problems: (1) Finding the order of the process (2) Estimating the parameters of the process. As for an AR process, it is convenient to consider the second problem first.

< previous page

file:///C:/Documents and Settings/Yang/

page_62

next page >

面/The analysis of time series an introduction/files/page_62.html [5/24/2009 16:51:27]

page_63

< previous page

page_63

next page >

Page 63 4.3.1 Estimating parameters of an MA process Estimation is more difficult for an MA process than an AR process, because efficient explicit estimators cannot be found. Instead some form of numerical iteration must be performed, albeit greatly facilitated by modern computers. Let us begin by considering the first-order MA process (4.13) where µ, β1 are constants and Zt denotes a purely random process. We would like to be able to write the residual sum of squares solely in terms of the observed xs and the parameters µ, β1, as we did for the AR process. Then we could differentiate with respect to µ and β1, and hence find the least squares estimates. Unfortunately this cannot be done and so explicit least squares estimates cannot be found; nor is it wise to simply equate sample and theoretical first-order autocorrelation coefficients by (4.14) and choose the solution , such that , because it can be shown that this gives rise to an inefficient estimator. One possible alternative approach is as follows: (i) Select suitable starting values for µ and β1, such as and the value of β1 given by the solution of Equation (4.14) (see Box et al., 1994, Part 5, Table A). (ii) Calculate the corresponding residual sum of squares using Equation (4.13) recursively in the form (4.15) Taking z0=0, we calculate z1=x1−µ, and then z2=x2−µ−β1z1, and so on until zN=xN−µ−β1zN−1. Then the residual sum of squares is calculated conditional on the given values of the parameters and z0=0. (iii) Repeat this procedure for other neighbouring values of µ and β1 so that the residual sum of squares is computed on a grid of points in the (µ, β1) plane. (iv) Determine by inspection the values of µ and β1 that minimize . The above procedure gives least squares estimates, which are also maximum likelihood estimates conditional on z0=0 provided that Zt is normally distributed. The procedure can be further refined by back-forecasting the value of z0 (see Box et al., 1994, Section 6.4.3), but this is unnecessary except when N is small or when

β1 is 'close' to plus or minus one. Nowadays the values of µ and β, which minimize , would normally be found by some iterative optimization procedure, such as hill-climbing, although a grid search can still sometimes be useful to see what the sum of squares surface looks like. For higher-order processes a similar type of iterative procedure to that described above could be used. For example, with a second-order MA process one would guess starting values for µ, β1, β2, compute the residuals recursively using and compute

Then other values of µ, β1, β2 could be tried, perhaps over a

< previous page

file:///C:/Documents and Settings/Yang/

page_63

next page >

面/The analysis of time series an introduction/files/page_63.html [5/24/2009 16:51:28]

page_64

< previous page

page_64

next page >

Page 64 grid of points, until the minimum value of is found. Nowadays computers can take advantage of an appropriate numerically efficient optimization procedure to minimize the residual sum of squares. Box et al. (1994, Section 7.2) describe such a procedure, which they call ‘non-linear estimation’, because the residuals are non-linear functions of the parameters. However, this term could give rise to confusion. For a completely new set of data, it may be a good idea to use the method based on evaluating the residual sum of squares at a grid of points. A visual examination of the sum of squares surface will sometimes provide useful information. In particular, it is interesting to see how ‘flat’ the surface is; whether the surface is approximately quadratic; and whether the parameter estimates are approximately uncorrelated (because the major axes of the ‘hill’ are parallel to the coordinate axes). In addition to point estimates, an approximate confidence region for the model parameters may be found as described by Box et al. (1994, Chapter 7) by assuming that the Zt are normally distributed. However, there is some doubt as to whether the asymptotic normality of maximum likelihood estimators will apply even for moderately large sample sizes (e.g. N=200). It should now be clear that it is much harder to estimate the parameters of an MA model than those of an AR model, because the ‘errors’ in an MA model are non-linear functions of the parameters and iterative methods are required to minimize the residual sum of squares. Because of this, many analysts prefer to fit an AR model to a given time series even though the resulting model may contain more parameters than the ‘best’ MA model. Indeed the relative simplicity of AR modelling is the main reason for its use in the stepwise autoregression forecasting technique (see Section 5.2.5) and in autoregressive spectrum estimation (see Section 13.7.1). 4.3.2 Determining the order of an MA process Although parameter estimation is harder for a given MA process than for a given AR process, the reverse is true as regards determining the order of the process. If an MA process is thought to be appropriate for a given set of data, the order of the process is usually evident from the sample ac.f. The theoretical ac.f. of an MA(q) process has a very simple form in that it ‘cuts off’ at lag q (see Section 3.4.3), and so the analyst should look for the lag beyond which the values of rk are close to zero. The partial ac.f. is generally of little help in identifying pure MA models because of its attenuated form. 4.4 Estimating Parameters of an ARMA Model Suppose now that a model with both AR and MA terms is thought to be appropriate for a given time series. The estimation problems for an ARMA model are similar to those for an MA model in that an iterative procedure has to be used. The residual sum of squares can be calculated at every point on a

< previous page

file:///C:/Documents and Settings/Yang/

page_64

next page >

面/The analysis of time series an introduction/files/page_64.html [5/24/2009 16:51:28]

page_65

< previous page

page_65

next page >

Page 65 suitable grid of the parameter values, and the values, which give the minimum sum of squares may then be assessed. Alternatively some sort of optimization procedure may be used. As an example, consider the ARMA(1, 1) process whose ac.f. decreases exponentially after lag 1 (see Exercise 3.11). This model may be recognized as appropriate if the sample ac.f. has a similar form. The model is given by Given N observations, x1,…, xN, we guess values for µ, α1, β1, set z0=0 and x0=µ, and then calculate the residuals recursively by

The residual sum of squares may then be calculated. Then other values of µ, α1, β1 may be tried until the minimum residual sum of squares is found. Many variants of the above estimation procedure have been studied—see, for example, the reviews by Priestley (1981, Chapter 5) and Kendall et al. (1983, Chapter 50). Nowadays exact maximum likelihood estimates are often preferred, since the extra computation involved is no longer a limiting factor. The conditional least squares estimates introduced above are conceptually easier to understand and can also be used as starting values for exact maximum likelihood estimation. The Hannan-Rissanen recursive regression procedure (e.g. see Granger and Newbold, 1986) is primarily intended for model identification but can additionally be used to provide starting values as well. The Kalman filter (see Section 10.1.4) may be used to calculate exact maximum likelihood estimates to any desired degree of approximation. We will say no more about this important, but rather advanced, topic here. Modern computer software for time-series modelling should incorporate sound estimation procedures for fitting MA and ARMA models, though it is worth finding out exactly what method is used, provided this information is given in the software documentation (as it should be, but often isn’t!). For short series, and models with roots near the unit circle, it is worth stressing that different estimation procedures can give noticeably different results. It is also worth noting that least squares estimates can be biased for short series (Ansley and Newbold, 1986) 4.5 Estimating Parameters of an ARIMA Model In practice, many time series are clearly non-stationary, and so the stationary models we have studied so far cannot be applied directly. In Box-Jenkins ARIMA modelling, the general approach is to difference an observed time series until it appears to come from a stationary process. An AR, MA or

< previous page

file:///C:/Documents and Settings/Yang/

page_65

next page >

面/The analysis of time series an introduction/files/page_65.html [5/24/2009 16:51:29]

page_66

< previous page

page_66

next page >

Page 66 ARMA model may then be fitted to the differenced series as described in Sections 4.2–4.4. It is often found that first-order differencing of non-seasonal data is adequate—see Section 3.4.6 and Example 14.2—although second-order differencing is occasionally required. The resulting model for the undifferenced series is the fitted ARIMA model. For seasonal data, seasonal differencing may also be required (see Section 4.6 and Example 14.3). It is unnecessary to say anything further about fitting ARIMA models because the possible use of differencing is the only feature that changes from fitting ARMA models. 4.6 Box-Jenkins Seasonal ARIMA Models In practice, many time series contain a seasonal periodic component, which repeats every s observations. For example, with monthly observations, where s=12, we may typically expect Xt to depend on values at annual lags, such as Xt−12, and perhaps Xt−24, as well as on more recent non-seasonal values such as Xt−1 and Xt −2. Box and Jenkins (1970) generalized the ARIMA model to deal with seasonality, and defined a general multiplicative seasonal ARIMA (SARIMA) model as (4.16) where B denotes the backward shift operator, p, ΦP, θq, ΘQ are polynomials of order p, P, q, Q, respectively, Zt denotes a purely random process and (4.17) denotes the differenced series. If the integer D is not zero, then seasonal differencing is involved (see Section 2.6). The above model is called a SARIMA model of order (p, d, q)×(P, D, Q)s. The SARIMA model looks rather complicated at first sight, so let us look at the different components of the model in turn. First, the differenced series {Wt} is formed from the original series {Xt} by appropriate differencing to remove non-stationary terms. If d is non-zero, then there is simple differencing to remove trend, while seasonal differencing may be used to remove seasonality. For example, if d=D=1 and s=12, then

In practice, the values of d and D are usually zero or one, and rarely two. Now let us look at the seasonal AR term, ΦP(Bs). Suppose for simplicity that P=1. Then Φ1(Bs) will be of the form (1−C×Bs), where C denotes a constant, which simply means that Wt will depend on Wt−s, since BsWt=Wt−s. Similarly, a seasonal MA term of order one means that Wt will depend on Zt−s as well as on Zt. As an example, consider a SARIMA model of order (1,0,0)×(0,1,1)12, where we note s=12. Here we have one non-seasonal AR term, one seasonal

< previous page

file:///C:/Documents and Settings/Yang/

page_66

next page >

面/The analysis of time series an introduction/files/page_66.html [5/24/2009 16:51:30]

page_67

< previous page

page_67

next page >

Page 67 MA term and one seasonal difference. Then Equations (4.16) and (4.17) can be written where and Θ is a constant parameter (rather than a function as in Equation (4.16)). It may help to write this out in terms of the original observed variable Xt as so that Xt depends on Xt−1, Xt−12 and Xt−13 as well as the innovations at times t and (t−12). You may be surprised to see that Xt−13 is involved, but this is a consequence of mixing first-order autocorrelation with seasonal differencing. When fitting a seasonal model to data, the first task is to assess values of d and D, which remove most of the trend and seasonality and make the series appear to come from a stationary process. Then the values of p, P, q and Q need to be assessed by looking at the ac.f. and partial ac.f. of the differenced series and choosing a SARIMA model whose ac.f. and partial ac.f. are of similar form (see Section 5.2.4 and Example 14.3). Finally, the model parameters may be estimated by some suitable iterative procedure. Full details are given by Box et al. (1994, Chapter 9), but the wide availability of suitable computer software means that the average analyst does not need to worry about the practical details of fitting a particular model. Instead, the analyst can concentrate on choosing a sensible model to fit and ensuring that appropriate diagnostic checks are carried out. 4.7 Residual Analysis When a model has been fitted to a time series, it is advisable to check that the model really does provide an adequate description of the data. As with most statistical models, this is usually done by looking at the residuals, which are generally defined by For a univariate time-series model, the fitted value is the one-step-ahead forecast so that the residual is the one-step-ahead forecast error. For example, for the AR(1) model, Xt=αXt−1+Zt where α is estimated by least squares, the fitted value at time t is so that the residual corresponding to the observed value, xt, is Note the ‘hat’ on zt as the residual is the estimated error term. Of course if α were known exactly, then the exact error zt=xt−αxt−1 could be calculated, but this situation never arises in practice (except in simulated exercises). If we have a ‘good’ model, then we expect the residuals to be ‘random’ and ‘close to zero’, and model validation usually consists of plotting residuals in various ways to see whether this is the case. With timeseries models we have

< previous page

file:///C:/Documents and Settings/Yang/

page_67

next page >

面/The analysis of time series an introduction/files/page_67.html [5/24/2009 16:51:31]

page_68

< previous page

page_68

next page >

Page 68 the added feature that the residuals are ordered in time and it is natural to treat them as a time series. Two obvious steps are to plot the residuals as a time plot, and to calculate the correlogram of the residuals. The time plot will reveal any outliers and any obvious autocorrelation or cyclic effects. The correlogram of the residuals will enable autocorrelation effects to be examined more closely. Let rz, k denote the autocorrelation coefficient at lag k of the residuals { }. If we could fit the true model2, with the correct model parameter values, then the true errors {zt} form a purely random process and, from Section 4.1, their correlogram is such that each autocorrelation coefficient is approximately normally distributed, with mean 0 and variance 1/ N, for reasonably large values of N. Of course, in practice, the true model is unknown and the correlogram of the residuals from the fitted model has somewhat different properties. For example, suppose we know the form of the model is an AR(1) process, but have to estimate the parameter. If the true value of the parameter is a=0.7, it can be shown that approximate 95% confidence limits for the estimated residual autocorrelations are at for rz,1, at for rz,2 and at for values of rz,k at higher lags. Thus for lags greater than 2, the confidence limits are the same as for the correlogram of the true errors. On the other hand, if we fit the wrong form of model, then the distribution of residual autocorrelations will be quite different, and we hope to get some ‘significant’ values so that the wrong model is rejected. The analysis of residuals from ARMA processes is discussed more generally by Box et al. (1994, Chapter 8). As in the AR(1) example above, it turns out that

supplies an upper bound for the

, are standard error of the residual autocorrelations, so that values, which lie outside the range significantly different from zero at the 5% level and give evidence that the wrong form of model has been fitted. Instead of looking at the residual autocorrelations one at a time, it is possible to carry out what is called a portmanteau lack-of-fit test. This looks at the first K values of the residual correlogram all at once. The test statistic is

(4.18) where N is the number of terms in the differenced series and K is typically chosen in the range 15 to 30. If the fitted model is appropriate, then Q should be approximately distributed as X2 with (K−p−q) degrees of freedom, where p, q are the number of AR and MA terms, respectively, in the model. Unfortunately the X2 approximation can be rather poor for N

面/The analysis of time series an introduction/files/page_68.html [5/24/2009 16:51:31]

page_69

< previous page

page_69

next page >

Page 69 Several other procedures for looking at residuals have also been proposed (e.g. Newbold, 1988, Section 4), but my own preference is usually just to ‘look’ at the first few values of rz,k, particularly at lags 1, 2 and the first seasonal lag (if any), and see if any are significantly different from zero using the crude limits of . If they are, then I would modify the model in an appropriate way by putting in extra terms to account for the significant autocorrelation(s). However, if only one (or two) values of rz,k are just significant at lags having no obvious physical interpretation, (e.g. k=5), then I would not regard this as compelling evidence to reject the model. Another statistic used for testing residuals is the Durbin-Watson statistic (e.g. Granger and Newbold, 1986, Section 6.2). This often appears in computer output and in my experience few people know what it means. The statistic is defined by

(4.19)

Now since

we find , where is the autocorrelation coefficient of the residuals at lag 1 (since the mean residual should be virtually zero). Thus, in this sort of application, the Durbin-Watson statistic is really the value rz,1 in a slightly different guise. If the true model has been fitted, so that . Thus a ‘typical’ value for d is around two and not zero. then we expect Furthermore, a test on d is asymptotically equivalent to a test on the value of rz,1 for the residuals. The Durbin-Watson statistic was originally proposed for use with multiple regression models as applied to time-series data. Suppose we have N observations on a dependent variable y, and m explanatory variables, say x1,…, xm, and we fit the model Having estimated the parameters {βi} by least squares, we want to see whether the error terms are really independent. The residuals are therefore calculated by The statistic d may now be calculated, and the distribution of d under the null hypothesis that the zt are independent has been investigated. Tables of critical values are available (e.g. Kendall et al., 1983) and they depend on the number of explanatory variables. Since d corresponds to the value of r1 for the residuals, this test implies that we are only considering an AR(1) process as an alternative to a purely random process for zt. Although it may be possible to modify the use of the Durbin-Watson statistic for models other than multiple regression models, it is usually better to look at the correlogram of the residuals as described earlier.

< previous page

file:///C:/Documents and Settings/Yang/

page_69

next page >

面/The analysis of time series an introduction/files/page_69.html [5/24/2009 16:51:32]

page_70

< previous page

page_70

next page >

Page 70 If the residual analysis indicates that the fitted model is inadequate in some way, then alternative models may need to be tried, and there are various tools for comparing the fit of several competing models (see Section 13.1). An iterative strategy for building time-series models, which is an integral part of the BoxJenkins approach, is discussed more fully in Sections 4.8 and 5.2.4. 4.8 General Remarks on Model Building How do we find a suitable model for a given time series? The answer depends on various considerations, including the properties of the series as assessed by a visual examination of the data, the number of observations available, the context and the way the model is to be used. It is important to understand that model building has three main stages, which can be described as: (1) Model formulation (or model specification) (2) Model estimation (or model fitting) (3) Model checking (or model verification). Textbooks often concentrate on estimation, but say rather little about the more important topic of formulating the model. This is unfortunate because modern computer software makes model fitting straightforward for many types of models, so that the real problem is knowing which model to fit in the first place. For example, it is relatively easy nowadays to fit an ARIMA model, but it can still be difficult to know when ARIMA modelling is appropriate and which ARIMA model to fit. Model checking is also very important, and the assessment of residuals is an essential step in the analysis. Modern software makes this relatively painless and may result in an initial model being discredited. Then an alternative model or models will be tried. Sometimes there are several cycles of model fitting as a model is modified and improved in response to residual checks or in response to additional data. Thus model building is an iterative, interactive process (see Section 13.5 and Chatfield, 1995a, Chapter 5). This section concentrates on model formulation. The analyst should consult appropriate ‘experts’ about the given problem, ask questions to get relevant background knowledge, look at a time plot of the data to assess their more important features, and make sure that a proposed model is consistent with empirical and/or theoretical knowledge and with the objectives of the investigation. There are many classes of time-series models to choose from. Chapter 3 introduced a general class of (univariate) models called ARIMA models, which includes AR, MA and ARMA models as special cases. This useful class of processes provides a good fit to many different types of time series and should generally be considered when more than about 50 observations are available. Another general class of models is the trend and seasonal type of model introduced in Chapter 2. Later in this book several more classes of models will

< previous page

file:///C:/Documents and Settings/Yang/

page_70

next page >

面/The analysis of time series an introduction/files/page_70.html [5/24/2009 16:51:33]

page_71

< previous page

page_71

next page >

Page 71 be introduced, including multivariate models of various types, and structural models. In areas such as oceanography and electrical engineering, long stationary series may occur. If a parametric model is required, an ARMA model should be considered. As well as the time plot, the correlogram and the partial ac.f. should be examined in order to identify an appropriate ARMA model. The model can then be fitted, checked and possibly modified in the usual way. However, as we will see later in Chapters 6 and 7, we may be more interested in the frequency properties of the time series, in which case an ARMA model may not be very helpful. In many other areas, such as economics and marketing, non-stationary series often arise and in addition may be fairly short. It is possible to treat non-stationarity by differencing the observed time series until it becomes stationary and then fitting an ARMA model to the differenced series. For seasonal series, the seasonal ARIMA model may be used. However, it should be clearly recognized that when the variation of the systematic part of the time series (i.e. the trend and seasonality) is dominant, the effectiveness of the ARIMA model is mainly determined by the initial differencing operations and not by the subsequent fitting of an ARMA model to the differenced series, even though the latter operation is much more time-consuming. Thus the simple models discussed in Chapter 2 may be preferable for time series with a pronounced trend and/or large seasonal effect. Models of this type have the advantage of being simple, easy to interpret and fairly robust. In addition they can be used for short series where it is impossible to fit an ARIMA model. An alternative approach is to model the non-stationary effects explicitly, rather than to difference them away, and this suggests the use of a state-space or structural model as will be described in Chapter 10. Sometimes the analyst may have several competing models in mind and then it may help to look at a modelselection statistic such as Akaike’s Information Criterion (AIC). These statistics are introduced later in Section 13.1 and try to strike a balance between the need for a ‘parsimonious’ model, which uses as few parameters as possible, and a model that is too simple and overlooks important effects. A useful reference on model building, in general, and model-selection statistics and the Principle of Parsimony, in particular, is Burnham and Anderson (2002). Whatever model is fitted, it is important to realise that it is only an approximation to the ‘truth’, and the analyst should always be prepared to modify a model in the light of new evidence. The effects of model uncertainty are discussed later in Section 13.5. Exercises 4.1 Derive the least squares estimates for an AR(1) process having mean μ (i.e. derive Equations (4.6) and (4.7), and check the approximations in Equations (4.8) and (4.9)).

< previous page

file:///C:/Documents and Settings/Yang/

page_71

next page >

面/The analysis of time series an introduction/files/page_71.html [5/24/2009 16:51:34]

page_72

page_72

< previous page

next page >

Page 72 4.2 Derive the least squares normal equations for an AR(p) process, taking , and compare with the Yule-Walker equations (Equation (4.12)). 4.3 Show that the (theoretical) partial autocorrelation coefficient of order 2, π2, is given by Compare with Equation (4.11). 4.4 Find the partial ac.f. of the AR(2) process given by

(see Exercise 3.6). 4.5 Suppose that the correlogram of a time series consisting of 100 observations has r1=0.31, r2=0.37, r3= −0.05, r4=0.06, r5= −0.21, r6=0.11, r7=0.08, r8=0.05, r9=0.12, r10=−0.01. Suggest an ARMA model, which may be appropriate. 4.6 Sixty observations are taken on a quarterly economic index, xt. The first eight values of the sample ac.f., rk, and the sample partial ac.f., k, of xt, and of the first differences, xt, are shown below: Lag 1 2 3 4 5 6 7 8

rk

0.95

0.91

0.87

0.82

0.79

0.74

0.70

0.67

k rk

0.95 0.02

0.04 0.08

−0.05 0.12

0.07 0.05

0.00 −0.02

0.07 −0.05

−0.04 −0.01

−0.02 0.03

k 0.02 0.08 0.06 0.03 −0.05 −0.06 −0.04 −0.02 Identify a model for the series. What else would you like to know about the data in order to make a better job of formulating a ‘good’ model?

< previous page

file:///C:/Documents and Settings/Yang/

page_72

next page >

面/The analysis of time series an introduction/files/page_72.html [5/24/2009 16:51:34]

page_73

< previous page

page_73

next page >

Page 73 CHAPTER 5 Forecasting Forecasting is the art of saying what will happen, and then explaining why it didn’t!—Anonymous 5.1 Introduction Forecasting the future values of an observed time series is an important problem in many areas, including economics, production planning, sales forecasting and stock control. Suppose we have an observed time series x1, x2,…, xN. Then the basic problem is to estimate future values such as xN+h, where the integer h is called the lead time or forecasting horizon—h for horizon. The forecast of xN+h made at time N for h steps ahead is typically denoted by (N, h) or N(h). This edition changes from the former notation to the latter, as it has become more standard. Note that some authors still use the imprecise notation N+h, which assumes implicitly that the forecast is made at time N. I strongly discourage this notation, as I regard it as essential to specify separately both the time a forecast is made and the length of the forecasting horizon. A wide variety of different forecasting procedures is available and it is important to realize that no single method is universally applicable. Rather, the analyst must choose the procedure that is most appropriate for a given set of conditions. It is also worth bearing in mind that forecasting is a form of extrapolation, with all the dangers that it entails. Forecasts are conditional statements about the future based on specific assumptions. Thus forecasts are not sacred and the analyst should always be prepared to modify them as necessary in the light of any external information. For long-term forecasting, it can be helpful to produce a range of forecasts based on different sets of assumptions so that alternative ‘scenarios’ can be explored. Forecasting methods may be broadly classified into three groups as follows: (1) Subjective Forecasts can be made on a subjective basis using judgement, intuition, commercial knowledge and any other relevant information. Methods range widely from bold freehand extrapolation to the Delphi technique, in which a group of forecasters tries to obtain a consensus forecast with controlled feedback of other analysts’ predictions and opinions as well as other relevant information. These methods will not be described here, as most statisticians

< previous page

file:///C:/Documents and Settings/Yang/

page_73

next page >

面/The analysis of time series an introduction/files/page_73.html [5/24/2009 16:51:35]

page_74

< previous page

page_74

next page >

Page 74 will want their forecasts to be at least partly objective. The interested reader is referred, for example, to Webby and O’Connor (1996), Rowe and Wright (1999) and the relevant sections of Armstrong (2001). However, note that some subjective judgement is often used in a more statistical approach, for example, to choose an appropriate model and perhaps make adjustments to the resulting forecasts. (2) Univariate Forecasts of a given variable are based on a model fitted only to present and past observations of a given time series, so that N(h) depends only on the values of xN, xN−1,…, possibly augmented by a simple function of time, such as a global linear trend. This would mean, for example, that univariate forecasts of the future sales of a given product would be based entirely on past sales, and would not take account of other economic factors. Methods of this type are sometimes called naive or projection methods. (3) Multivariate Forecasts of a given variable depend at least partly on values of one or more additional series, called predictor or explanatory variables1. For example, sales forecasts may depend on stocks and/or on economic indices. Models of this type are sometimes called causal models. In practice, a forecasting procedure may involve a combination of the above approaches. For example, marketing forecasts are often made by combining statistical predictions with the subjective knowledge and insight of people involved in the market. A more formal type of combination is to compute a weighted average of two or more objective forecasts, as this often proves superior on average to the individual forecasts—see Section 13.5. Unfortunately, an informative model may not result. An alternative way of classifying forecasting methods is between an automatic approach requiring no human intervention, and a non-automatic approach requiring some subjective input from the forecaster. The latter applies to subjective methods and most multivariate methods. Most univariate methods can be made fully automatic but can also be used in a non-automatic form, and there can be a surprising difference between the results. The choice of method depends on a variety of considerations, including: • How the forecast is to be used. • The type of time series (e.g. macroeconomic series or sales figures) and its properties (e.g. are trend and seasonality present?). Some series are very regular and hence ‘very predictable’, but others are not. As always, a time plot of the data is very helpful. • How many past observations are available. 1 They are also sometimes called independent variables but this terminology is misleading, as they are typically not independent of each other.

< previous page

file:///C:/Documents and Settings/Yang/

page_74

next page >

面/The analysis of time series an introduction/files/page_74.html [5/24/2009 16:51:36]

page_75

< previous page

page_75

next page >

Page 75 • The length of the forecasting horizon. This book is mainly concerned with short-term forecasting. For example, in stock control the lead time for which forecasts are required is the time between ordering an item and its delivery. • The number of series to . forecast and the cost allowed per series. • The skill and experience of the analyst. Analysts should select a method with which they feel ‘happy’ and for which relevant computer software is available. They should also consider the possibility of trying more than one method. It is particularly important to clarify the objectives (as in any statistical investigation). This means finding out how a forecast will actually be used, and whether it may even influence the future. In the latter case, some forecasts turn out to be self-fulfilling. In a commercial environment, forecasting should be an integral part of the management process leading to what is sometimes called a systems approach. This chapter concentrates on calculating point forecasts, where the forecast for a particular future time period consists of a single number. Point forecasts are adequate for many purposes, but a prediction interval is often helpful to give a better indication of future uncertainty. Instead of a single value, a prediction interval consists of upper and lower limits between which a future value is expected to lie with a prescribed probability. Some methods for calculating prediction intervals are considered in Section 5.2.6. Taking one more step away from a point forecast, it may be desirable to calculate the entire probability distribution of a future value of interest. This is called density forecasting. The reader is referred to Tay and Wallis (2000). A practical halfway house between prediction intervals and density forecasting is the use of fan charts. The latter essentially plot prediction intervals at several different probability levels, by using darker shades for central values, and lighter shades for outer bands, which cover less likely values. These graphs can be very effective for presenting the future range of uncertainty in a simple, visually-effective way. The reader is referred to Wallis (1999). Whatever forecasting method is used, some sort of forecast monitoring scheme is often advisable, particularly with large numbers of series, to ensure that forecast errors are not systematically positive or negative. A variety of tracking signals for detecting ‘trouble’ are discussed, for example, by Gardner (1983) and McLain (1988). 5.2 Univariate Procedures This section introduces the many projection methods that are now available. Further details may be found in Abraham and Ledolter (1983), Chatfield (2001, Chapters 3−4), Diebold (2001), Granger and Newbold (1986) and Montgomery et al. (1990).

< previous page

file:///C:/Documents and Settings/Yang/

page_75

next page >

面/The analysis of time series an introduction/files/page_75.html [5/24/2009 16:51:36]

page_76

< previous page

page_76

next page >

Page 76 5.2.1 Extrapolation of trend curves For long-term forecasting of non-seasonal data, it is often useful to fit a trend curve (or growth curve) to successive values and then extrapolate. This approach is most often used when the data are yearly totals, and hence clearly non-seasonal. A variety of curves may be tried including polynomial, exponential, logistic and Gompertz curves (see also Section 2.5.1). When the data are annual totals, at least 7 to 10 years of historical data are required to fit such curves. The method is worth considering for short annual series where fitting a complicated model to past data is unlikely to be worthwhile. Although primarily intended for longterm forecasting, it is inadvisable to make forecasts for a longer period ahead than about half the number of past years for which data are available. A drawback to the use of trend curves is that there is no logical basis for choosing among the different curves except by goodness-of-fit. Unfortunately it is often the case that one can find several curves that fit a given set of data almost equally well but which, when projected forward, give widely different forecasts. Further details about trend curves are given by Meade (1984). 5.2.2 Simple exponential smoothing Exponential smoothing (ES) is the name given to a general class of forecasting procedures that rely on simple updating equations to calculate forecasts. The most basic form, introduced in this subsection, is called simple exponential smoothing (SES), but this should only be used for non-seasonal time series showing no systematic trend. Of course many time series that arise in practice do contain a trend or seasonal pattern, but these effects can be measured and removed to produce a stationary series for which simple ES is appropriate. Alternatively, more complicated versions of ES are available to cope with trend and seasonality— see Section 5.2.3 below. Thus adaptations of exponential smoothing are useful for many types of time series —see Gardner (1985) for a detailed general review of these popular procedures. Given a non-seasonal time series, say x1, x2 ,…, xN, with no systematic trend, it is natural to forecast xN+1 by means of a weighted sum of the past observations: (5.1) where the {ci} are weights. It seems sensible to give more weight to recent observations and less weight to observations further in the past. An intuitively appealing set of weights are geometric weights, which decrease by a constant ratio for every unit increase in the lag. In order that the weights sum to one, we take where α is a constant such that 0

Page 77 Strictly speaking, Equation (5.2) implies an infinite number of pastobservations, but in practice there will only be a finite number. Thus Equation(5.2) is customarily rewritten in the recurrence form as (5.3) , then Equation (5.3) can be used recursively to compute forecasts. Equation (5.3) If we set also reduces the amount of arithmetic involved since forecasts can easily be updated using only the latest observation and the previous forecast. The procedure defined by Equation (5.3) is called simple exponential smoothing. The adjective ‘exponential’ arises from the fact that the geometric weights lie on an exponential curve, but the procedure could equally well have been called geometric smoothing. Equation (5.3) is sometimes rewritten in the equivalent error-correction form (5.4) is the prediction error at time N. Equations (5.3) and (5.4) look different at where first sight, but give identical forecasts, and it is a matter of practical convenience as to which one should be used. Although intuitively appealing, it is natural to ask when SES is a ‘good’ method to use. It can be shown (see Exercise 5.6) that SES is optimal if the underlying model for the time series is given by (5.5) where {Zt} denotes a purely random process. This infinite-order moving average (MA) process is nonstationary, but the first differences (Xt+1−Xt) form a stationary first-order MA process. Thus Xt is an autoregressive integrated moving average process of order (0,1,1). In fact it can be shown (Chatfield et al., 2001) that there are many other models for which SES is optimal. This helps to explain why SES appears to be such a robust method. The value of the smoothing constant α depends on the properties of the given time series. Values between 0.1 and 0.3 are commonly used and produce a forecast that depends on a large number of past observations. Values close to one are used rather less often and give forecasts that depend much more on recent observations. When a=1, the forecast is equal to the most recent observation. The value of a may be estimated from past data by a similar procedure to that used for estimating the parameters of an MA process. Given a particular value of a, one-step-ahead forecasts are produced iteratively through the series, and then the sum of squares of the one-step-ahead prediction errors is computed. This can be repeated for different values of a so that the value,

< previous page

file:///C:/Documents and Settings/Yang/

page_77

next page >

面/The analysis of time series an introduction/files/page_77.html [5/24/2009 16:51:38]

page_78

page_78

< previous page

next page >

Page 78 which minimizes the sum of squares, can be found. In more detail, for a given value of α, calculate

and so on until

and then compute

. Repeat this procedure for other values of α between 0 and 1, say in steps of

0.1, and select the value that minimizes , either by inspection or using an algorithmic numerical procedure. Modern computers make this all easy to do. Usually the sum of squares surface is quite flat near the minimum and so the choice of α is not critical. 5.2.3 The Holt and Holt-Winters forecasting procedures Exponential smoothing may readily be generalized to deal with time series containing trend and seasonal variation. The version for handling a trend with non-seasonal data is usually called Holt’s (two-parameter) exponential smoothing, while the version that also copes with seasonal variation is usually referred to as the Holt-Winters (three-parameter) procedure. These names honour the pioneering work of C.C.Holt and P.R. Winters around 1960. The general idea is to generalize the equations for SES by introducing trend and seasonal terms, which are also updated by exponential smoothing. We first consider Holt’s ES. In the absence of trend and seasonality, the one-step-ahead forecast from simple ES can be thought of as an estimate of the local mean level of the series, so that simple ES can be regarded as a way of updating the local level of the series, say Lt. This suggests rewriting Equation (5.3) in the form Suppose we now wish to include a trend term, Tt say, which is the expected increase or decrease per unit time period in the current level. Then a plausible pair of equations for updating the values of Lt and Tt in recurrence form are the following

Then the h-step-ahead forecast at time t will be of the form for h=1, 2, 3,…. There are now two updating equations, involving two

< previous page

file:///C:/Documents and Settings/Yang/

page_78

next page >

面/The analysis of time series an introduction/files/page_78.html [5/24/2009 16:51:38]

page_79

page_79

< previous page

next page >

Page 79 smoothing parameters, α and γ, which are generally chosen to lie in the range(0,1). It is natural to call this the two-parameter version of ES. The above procedure may readily be generalized again to cope with seasonality. Let Lt, Tt, It denote the local level, trend and seasonal index, respectively, at time t. The interpretation of It depends on whether seasonality is thought to be additive or multiplicative—see Section 2.6. In the former case, xt−It is the deseasonalized value, while in the multiplicative case, it is xt/It. The values of the three quantities, Lt, Tt and It, all need to be estimated and so we need three updating equations with three smoothing parameters, say α, γ and δ. As before, the smoothing parameters are usually chosen in the range (0,1). The form of the updating equations is again intuitively plausible. Suppose the observations are monthly, and that the seasonal variation is multiplicative. Then the (recurrence form) equations for updating Lt, Tt, It, when a new observation xt becomes available, are

and the forecasts from time t are then for h=1, 2,…, 12. There are analogous formulae for the additive seasonal case. There are also analogous formulae for the case where the seasonality is of length s say, rather than 12 as for monthly observations. In particular, s=4 for quarterly data, when we would, for example, compare It with It−4. Unfortunately, the literature is confused by many different notations and by the fact that the updating equations may be presented in an equivalent error-correction form, which can look quite different. For example, the above formula for updating the trend in the monthly multiplicative case can be rewritten (after some algebra) in the form denotes the one-step-ahead forecast error as before. This formula looks quite where different and, in particular, it looks as though αγ is the smoothing parameter. Clearly, great care needs to be taken when comparing formulae from different sources expressed in different ways. In order to apply Holt-Winters smoothing to seasonal data, the analyst should carry out the following steps: (1) Examine a graph of the data to see whether an additive or a multiplicative seasonal effect is the more appropriate. (2) Provide starting values for L1, and T1 as well as seasonal values for the first year, say I1, I2,…, Is, using the first few observations in the series in a fairly simple way; for example, the analyst could choose .

< previous page

file:///C:/Documents and Settings/Yang/

page_79

next page >

面/The analysis of time series an introduction/files/page_79.html [5/24/2009 16:51:39]

page_80

< previous page

page_80

next page >

Page 80 (3) Estimate values for α, γ, δ by minimizing over a suitable fitting period for which historical data are available. (4) Decide whether to normalize the seasonal indices at regular intervals by making them sum to zero in the additive case or have an average of one in the multiplicative case. (5) Choose between a fully automatic approach (for a large number of series) and a non-automatic approach. The latter allows subjective adjustments for particular series, for example, by allowing the removal of outliers and a careful selection of the appropriate form of seasonality. Further details on the Holt-Winters method are given by Chatfield and Yar (1988). The method is straightforward and is widely used in practice. Another variation of ES that deserves mention here is the use of a damped trend (Gardner and McKenzie, 1985). This procedure can be used with the Holt and Holt-Winters methods and introduces another smoothing parameter, say where 0<

面/The analysis of time series an introduction/files/page_80.html [5/24/2009 16:51:40]

page_81

< previous page

page_81

next page >

Page 81 5.2.4 The Box-Jenkins procedure This section gives a brief outline of the forecasting procedure, based on autoregressive integrated moving average (ARIMA) models, which is usually known as the Box-Jenkins approach. The beginner may find it easier to get further information from books such as Vandaele (1983), Granger and Newbold (1986) or Jenkins (1979), rather than the original 1970 Box-Jenkins book now revised as Box et al. (1994), although the latter is still an essential reference source. Now AR, MA and ARMA models have been around for many years and are associated, in particular, with early work by G.U.Yule and H.O.Wold. A major contribution of Box and Jenkins has been to provide a general strategy for time-series forecasting, which emphasizes the importance of identifying an appropriate model in an iterative way as outlined briefly in Section 4.8. Indeed the iterative approach to model building that they suggested has since become standard in many areas of statistics. Furthermore, Box and Jenkins showed how the use of differencing can extend ARMA models to ARIMA models and hence cope with non-stationary series. In addition, Box and Jenkins show how to incorporate seasonal terms into seasonal ARIMA (SARIMA) models. Because of all these fundamental contributions, ARIMA models are often referred to as Box-Jenkins models. In brief, the main stages in setting up a Box-Jenkins forecasting model are as follows: (1) Model identification Examine the data to see which member of the class of ARIMA processes appears to be most appropriate. (2) Estimation Estimate the parameters of the chosen model as described in Chapter 4. (3) Diagnostic checking Examine the residuals from the fitted model to see if it is adequate. (d) Consideration of alternative models if necessary If the first model appears to be inadequate for some reason, then alternative ARIMA models may be tried until a satisfactory model is found. When such a model has been found, it is usually relatively straightforward to calculate forecasts as conditional expectations. We now consider these stages in more detail. In order to identify an appropriate ARIMA model, the first step in the Box-Jenkins procedure is to difference the data until they are stationary. This is achieved by examining the correlograms of various differenced series until one is found that comes down to zero ‘fairly quickly’ and from which any seasonal cyclic effect has been largely removed, although there could still be some ‘spikes’ at the seasonal lags s, 2s,

< previous page

file:///C:/Documents and Settings/Yang/

page_81

next page >

面/The analysis of time series an introduction/files/page_81.html [5/24/2009 16:51:40]

page_82

< previous page

page_82

next page >

Page 82 and so on, where s is the number of observations per year. For non-seasonal data, first-order differencing is usually sufficient to attain stationarity. For monthly data (of period 12), the operator 12 is often used if the seasonal effect is additive, while the operator may be used if the seasonal effect is multiplicative. Sometimes the operator 12 by itself will be sufficient. Over-differencing should be avoided. For a seasonal period of length s, the operator s may be used, and, in particular, for quarterly data we may use 4. The differenced series will be denoted by {wt; t=1,…, N−c}, where c terms are ‘lost’ by differencing. For example, if the operator 12 is used, then c=13. For non-seasonal data, an ARMA model can now be fitted to {wt} as described in Chapter 4. If the data are seasonal, then the SARIMA model defined in Equation (4.16) may be fitted as follows. Plausible values of p, P, q, Q are selected by examining the correlogram and the partial autocorrelation function (ac.f.) of the differenced series {wt}. Values of p and q are selected by examining the first few values of rk, as outlined in Chapter 4. Values of P and Q are selected primarily by examining the values of rk at k=12, 24,…, when the seasonal period is given by s=12. If, for example, r12 is ‘large’ but r24 is ‘small’, this suggests one seasonal moving average term, so we would take P=0, Q=1, as this SARIMA model has an ac.f. of similar form. Box et al. (1994, Table A9.1) list the autocovariance functions of various SARIMA models. Having tentatively identified what appears to be a reasonable SARIMA model, least squares estimates of the model parameters may be obtained by minimizing the residual sum of squares in a similar way to that proposed for ordinary ARMA models. In the case of seasonal series, it is advisable to estimate initial values of at and wt by backforecasting (or backcasting) rather than setting them equal to zero. This procedure is described by Box et al. (1994, Section 9.2.4). In fact, if the model contains a seasonal MA parameter that is close to one, several cycles of forward and backward iteration may be needed. Nowadays several alternative estimation procedures are available, based on, for example, the exact likelihood function, on conditional or unconditional least squares, or on a Kalman filter approach (see references in Section 4.4). For both seasonal and non-seasonal data, the adequacy of the fitted model should be checked by what Box and Jenkins call ‘diagnostic checking’. This essentially consists of examining the residuals from the fitted model to see whether there is any evidence of non-randomness. The correlogram of the residuals is calculated and we can then see how many coefficients are significantly different from zero and whether any further terms are indicated for the ARIMA model. If the fitted model appears to be inadequate, then alternative ARIMA models may be tried until a satisfactory one is found. Section 13.1 describes some additional model identification tools to help in choosing an appropriate model.

< previous page

file:///C:/Documents and Settings/Yang/

page_82

next page >

面/The analysis of time series an introduction/files/page_82.html [5/24/2009 16:51:41]

page_83

page_83

< previous page

next page >

Page 83 When a satisfactory model is found, forecasts may readily be computed. Given data up to time N, these forecasts will involve the observations and the fitted residuals (i.e. the one-step-ahead forecast errors) up to and including time N. The minimum mean square error forecast of XN+h at time N is the conditional expectation of XN+h at time N, namely, . In evaluating this conditional expectation, we use the fact that the ‘best’ forecast of all future Zs is simply zero (or more formally that the conditional expectation of ZN+h, given data up to time N, is zero for all h>0). Box et al. (1994) describe three general approaches to computing forecasts. (1) Using the model equation directly Point forecasts are usually computed most easily directly from the ARIMA model equation, which Box et al. is (1994) call the difference equation form. Assuming that the model equation is known exactly, then obtained from the model equation by replacing (i) future values of Z by zero, (ii) future values of X by their conditional expectation and (iii) present and past values of X and Z by their observed values. As an example, consider the SARIMA(1, 0, 0)×(0,1,1)12 model used as an example in Section 4.6, where Then we find

Forecasts further into the future can be calculated recursively in an obvious way. It is also possible to find ways of updating the forecasts as new observations become available. For example, when xN+1 becomes known, we have

(2) Using the

weights

An ARMA model can be rewritten as an infinite-order MA process and the resulting weights, as defined in Equation (3.6b), could also be used to compute forecasts, but are primarily helpful for calculating forecast error variances. Since (5.6) and future Zs are unknown at time N, it is clear that is equal to cannot be included). Thus the h-steps-ahead forecast error is

(since future zs

. Hence the variance of the h-steps-ahead forecast error is .

< previous page

file:///C:/Documents and Settings/Yang/

page_83

next page >

面/The analysis of time series an introduction/files/page_83.html [5/24/2009 16:51:42]

page_84

page_84

< previous page

next page >

Page 84 (3) Using the π weights An ARMA model can also be rewritten as an infinite-order AR process and the resulting π weights, as defined in Equation (3.6c), can also be used for compute point forecasts. Since it is intuitively clear that

N(h) is given by

These forecasts can be computed recursively, replacing future values of X with predicted values as necessary. In general, methods (1) or (3) are used for point forecasts while method (2) is used for forecast error variances. In practice, the model will not be known exactly, and we have to estimate the model parameters (and hence the

and πs if they are needed); we also have to estimate the past observed values of Z,

namely, zt, by the observed residuals or one-step-ahead forecasts errors, namely, (1,0,0)×(0,1,1)12 model given above, we would have, for example, that

. Thus for the SARIMA

Except for short series, this generally makes little difference to forecast error variances. Although some packages have been written to carry out ARIMA modelling and forecasting in an automatic way, the Box-Jenkins procedure is primarily intended for a non-automatic approach where the analyst uses subjective judgement to select an appropriate model from the large family of ARIMA models according to the properties of the individual series being analysed. Thus, although the procedure is more versatile than many competitors, it is also more complicated and considerable experience is required to identify an appropriate ARIMA model. Unfortunately, the analyst may find several different models, which fit the data equally well but give rather different forecasts, while sometimes it is difficult to find any sensible model. Of course, an inexperienced analyst will sometimes choose a ‘silly’ model. Another drawback is that the method requires several years of data (e.g. at least 50 observations for monthly seasonal data). My own view (see also Section 5.4) is that the method should not be used by analysts with limited statistical experience or for series where the variation is dominated by trend and seasonal variation (see Example 5.2 and Example 14.3). However, it can work well for series showing short-term correlation (see Example 14.2). It can also be combined with seasonal adjustment methods as in the X-11 or X-12 ARIMA methods (Findley et al., 1998) and may be generalized to the multivariate case (see Chapter 12).

< previous page

file:///C:/Documents and Settings/Yang/

page_84

next page >

面/The analysis of time series an introduction/files/page_84.html [5/24/2009 16:51:43]

page_85

< previous page

page_85

next page >

Page 85 5.2.5 Other methods Many other univariate forecasting procedures have been proposed, and we briefly mention a few of them. Granger and Newbold (1986, Section 5.4) describe a procedure called stepwise autoregression, which can be regarded as a subset of the Box-Jenkins procedure. It has the advantage of being fully automatic and relies on the fact that AR models are much easier to fit than MA or ARMA models even though an AR model may require extra parameters to give as good a representation of the data. The first step is to take first differences of the data to allow for non-stationarity in the mean. Then a maximum possible lag, say p, is chosen and the best AR model with just one lagged variable at a lag between 1 and p, is found, namely

where

is the autoregression coefficient at lag k when fitting one

lagged variable only, and is the corresponding error term. Then the best AR model with 2, 3,…, lagged variables is found. The procedure is terminated when the reduction in the sum of squared residuals at the jth stage is less than some preassigned quantity. Thus an integrated AR model is fitted, which is a special case of the Box-Jenkins ARIMA class. Granger and Newbold suggest choosing p=13 for quarterly data and p=25 for monthly data. Harrison (1965) has proposed a modification of seasonal exponential smoothing, which consists essentially of performing a Fourier analysis of the seasonal factors and replacing them by smoothed factors. Parzen’s ARARMA approach (Parzen, 1982; Meade and Smith, 1985) relies on fitting an AR model to remove the trend (rather than just differencing the trend away) before fitting an ARMA model. An apparently new method, called the theta method, gave promising results (Makridakis and Hibon, 2000), but subsequent research has shown that it is actually equivalent to a form of exponential smoothing. There are two general forecasting methods, called Bayesian forecasting (West and Harrison, 1997) and structural modelling (Harvey, 1989), which rely on updating model parameters by a technique called Kalman filtering. The latter is introduced in Chapter 10, and so we defer consideration of these methods until then. 5.2.6 Prediction intervals Thus far, we have concentrated on calculating point forecasts, but it is sometimes better to calculate an interval forecast to give a clearer indication of future uncertainty. A prediction interval (P.I.) consists of upper and lower limits between which a future value is expected to lie with a prescribed

< previous page

file:///C:/Documents and Settings/Yang/

page_85

next page >

面/The analysis of time series an introduction/files/page_85.html [5/24/2009 16:51:44]

page_86

< previous page

page_86

next page >

Page 86 probability. This interval can be calculated in several ways, depending on the forecasting method used, the properties of the data and so on. Most P.I.s used in practice are essentially of the following general form. A 100(1−α)% P.I. for XN+h is given by : (5.7) denotes the forecast error made at time N when forecasting h steps where ahead. Here zα/2 denotes the percentage point of a standard normal distribution with a proportion α/2 above it. Equation (5.7) assumes that an appropriate expression for Var[eN(h)] can be found for the method or model being used. As the P.I. in Equation (5.7) is symmetric about N(h), it effectively assumes that the forecast is unbiased. The formula also assumes that the forecast errors are normally distributed. In practice, the forecast errors are unlikely to be exactly normal, because the estimation of model parameters produces small departures from normality, while the assumption of a known, invariant model with normal errors is also unlikely to be exactly true. Nevertheless, because of its simplicity, Equation (5.7) is the formula that is generally used to compute P.I.s, though preferably after checking that the underlying assumptions (e. g. forecast errors are normally distributed) are at least approximately satisfied. For any given forecasting method, the main problem will then lie with evaluating Var[eN(h)]. Fortunately, formulae for this variance are available for many classes of model. Perhaps the best-known formula is for Box-Jenkins ARIMA forecasting, where the variance may be evaluated by writing an ARIMA model in infinite-moving-average form as (5.8) —see Equation (5.6). Then the best forecast at time N of XN+h can only involve the values of Zt up to time

t=N, so that

. Thus

Formulae can also be found for most variations of exponential smoothing, by assuming that the given method is optimal. For example, for simple exponential smoothing, with smoothing parameter α, it can be shown that where denotes the variance of the one-step-ahead forecast errors. Analogous formulae are available in the literature for some other methods and models—see Chatfield (1993; 2001, Chapter 7). A completely different alternative approach to calculating P.I.s is to work empirically by using the ‘forecast’ errors obtained by fitting a model to past data, finding the observed variance of the within-sample ‘forecast’ errors at different lead times and using these values in Equation (5.7). A detailed review of different methods of calculating P.I.s, including simulation, resampling and Bayesian methods, is given by Chatfield (2001, Chapter 7).

< previous page

file:///C:/Documents and Settings/Yang/

page_86

next page >

面/The analysis of time series an introduction/files/page_86.html [5/24/2009 16:51:44]

page_87

< previous page

page_87

next page >

Page 87 Unfortunately, whichever way P.I.s are calculated, they tend to be too narrow in practice. The empirical evidence for this is discussed by Chatfield (2001, Section 7.7), who also suggests various reasons why this phenomenon occurs. The main reason seems to be that the underlying model may change in the future. The forecaster should bear this in mind and should not think that a narrow P.I. is necessarily ‘good’. 5.3 Multivariate Procedures This section provides a brief introduction to some multivariate forecasting procedures. Further details on multivariate modelling are given later in this book, especially in Section 9.4.2 and in Chapter 12. More detailed coverage is given by Chatfield (2001, Chapter 5). The concepts are much more advanced than for univariate modelling and more sophisticated tools, such as the cross-correlation function, need to be developed in later chapters before we can make much progress. 5.3.1 Multiple regression One common forecasting method makes use of the multiple regression model, which will be familiar to many readers. This model assumes that the response variable of interest, say y, is linearly related to p explanatory variables, say x1, x2,…, xp. The usual multiple regression model can be written as (5.9) where {βi} are constants, and u denotes the ‘error’ term. This equation is linear in terms of the parameters {βi}, but could involve non-linear functions of observed variables. When building a regression model, it is helpful to distinguish between explanatory variables that can be controlled (like stock levels) and those that cannot (like air temperature). Now Equation (5.9) does not specifically involve time. Of course, we could regard time as a predetermined variable and introduce it as one of the explanatory variables, but regression on time alone would normally be regarded as a univariate procedure. More generally, we need to specify when each of the variables in Equation (5.9) is measured and so each variable really needs a subscript indicating when the variable is measured. When lagged values of the explanatory variables are included, they may be called leading indicators. Such variables are much more useful for forecasting. If lagged values of the response variable y are included, they are of an autoregressive nature and change the character of the model. Multiple regression is covered in numerous statistics texts and the details need not be repeated here. The models are widely used and sometimes work well. However, there are many dangers in applying such models to time-series data. Modern computer software makes it easy (perhaps too easy!) to fit regression models, and the ease of computation makes it tempting to

< previous page

file:///C:/Documents and Settings/Yang/

page_87

next page >

面/The analysis of time series an introduction/files/page_87.html [5/24/2009 16:51:45]

page_88

< previous page

page_88

next page >

Page 88 include lots of explanatory variables, even though including too many may yield dubious results. In fact applying regression models to time-series data is really not straightforward, especially as standard results assume that successive values of the ‘errors’ {u} are independent, which is unlikely to be the case in practice. Although a high multiple correlation coefficient R2 may result from fitting a regression model to timeseries data, this apparent good fit may be spurious and does not mean that good forecasts necessarily result. This may be demonstrated both theoretically (Phillips, 1986; Hamilton, 1994, Section 18.3) and empirically (Granger and Newbold, 1974; 1986, Section 6.4) for non-stationary data. It is advisable to restrict the value of p to perhaps 3 or 4, and keep back part of the data to check forecasts from the fitted model. When doing this, it is important to distinguish between ex ante forecasts of y, which replace future values of explanatory variables by their forecasts (and so are true out-of-sample forecasts), and ex post forecasts, which use the true values of explanatory variables. The latter can look misleadingly good. Problems can arise when the explanatory variables are themselves correlated, as often happens with timeseries data. It is advisable to begin by looking at the correlations between explanatory variables so that, if necessary, selected explanatory variables can be removed to avoid possible singularity problems. The quality and characteristics of the data also need to be checked. For example, if a crucial explanatory variable has been held more or less constant in the past, then it is impossible to assess its effect using past data. Another type of problem arises when the response variable can, in turn, affect values of the explanatory variables to give what is called a closed-loop system. This is discussed later in Sections 9.4.3 and 12.1. Perhaps the most important danger arises from mistakenly assuming that the ‘error’ terms form an independent sequence. This assumption is often inappropriate and can lead to a badly misspecified model and poor forecasts (Box and Newbold, 1971). The residuals from a regression model should always be cheeked for possible autocorrelation—see Section 4.7. A standard regression model, with independent errors, is usually fitted by Ordinary Least Squares (OLS), but this is seldom applicable directly to time-series data without suitable modification. Several alternative estimation procedures, such as Generalized Least Squares (GLS), have been developed over the years to cope with autocorrelated errors, but in such a way as still to be able to use OLS software. It can be shown that GLS and OLS are sometimes equivalent asymptotically, but such results may have little relevance for short series. Moreover, it is disturbing that autocorrelated errors may arise because certain lagged variables have been omitted from the model so that efforts to overcome such problems (e.g. by using GLS) are likely to lead to failure in the presence of a mis-specified model (e.g. Mizon, 1995). Mis-specifying the error structure also causes problems. Further details about methods, such as GLS, may be found, for example, in an econometrics text such as Hamilton (1994). Nowadays, full maximum likelihood is likely to be used once an appropriate

< previous page

file:///C:/Documents and Settings/Yang/

page_88

next page >

面/The analysis of time series an introduction/files/page_88.html [5/24/2009 16:51:46]

page_89

< previous page

page_89

next page >

Page 89 model for the errors (e.g. AR, MA or ARMA) has been identified (Choudhury et al., 1999). In summary, the use of multiple regression can be dangerous except when there are clear contextual reasons why one or more series should explain variation in another. There are various precautions that should be taken, and various alternative strategies that should be considered. They include: (1) Using the context to choose the explanatory variables with care, and limiting their total number to perhaps 3 or 4; (2) Including appropriate lagged values of variables as variables in their own right; (3) Removing obvious sources of nonstationarity before fitting a regression model; (4) Carrying out careful diagnostic checks on any fitted model; (5) Allowing for correlated errors in the fitting procedure and (6) Considering alternative families of models such as transfer function models, vector AR models or a model allowing for co-integration—see Sections 9.4.2, 12.3 and 12.6, respectively. 5.3.2 Econometric models Econometric models (e.g. Harvey, 1990) often assume that an economic system can be described, not by a single equation, but by a set of simultaneous equations. For example, not only do wage rates depend on prices but also prices depend on wage rates. Economists distinguish between exogenous variables, which affect the system but are not themselves affected, and endogenous variables, which interact with each other. The simultaneous equation system involving k dependent (endogenous) variables {Yi} and g predetermined (exogenous) variables {Xi} may be written for i=1, 2,…, k. Some of the exogenous variables may be lagged values of the Yi. The above set of equations, often called the structural form of the system, can be solved to give what is called the reduced form of the system, namely The principles and problems involved in constructing econometric models are too broad to be discussed in detail here (see, for example, Granger and Newbold, 1986, Section 6.3). A key issue is the extent to which the form of the model should be based on judgement, on economic theory and/or on empirical data. While some econometricians have been scornful2 of univariate time-series models, which do not ‘explain’ what is going on, statisticians have been generally sceptical of some econometric model building in which the structure of the model is determined a priori by economic theory and little attention is paid to identifying an appropriate ‘error’ structure or to using empirical data. Fortunately, mutual understanding has improved in recent years as developments in multivariate time-series modelling have brought statisticians and econometricians closer together to the benefit of both. In 2 This occurs especially when the univariate forecasts are ‘better’ than alternatives!

< previous page

file:///C:/Documents and Settings/Yang/

page_89

next page >

面/The analysis of time series an introduction/files/page_89.html [5/24/2009 16:51:46]

page_90

< previous page

page_90

next page >

Page 90 fact, the uncontrolled nature of much economic data makes it difficult to identify econometric models solely on an empirical statistical basis, while overreliance on economic theory should also be avoided. It is- now widely recognized that a balanced middle way is sensible, and that econometric model building should be an iterative process involving both theory and data. In particular, I note that econometricians have made substantial contributions to multivariate time-series modelling in recent years—see Chapter 12. 5.3.3 Other multivariate models There are many other types of multivariate models that may be used to produce forecasts. The multivariate generalization of ARIMA models is considered in Chapter 12. One special case is the class of vector autoregressive models, while another useful class of models is that called transfer function models (see Section 9.4.2). The latter concentrates on describing the relationship between one ‘output’ variable and one or more ‘input’ or explanatory variables. It is helpful to understand the interrelationships between all these classes of multivariate models (e.g. see Granger and Newbold, 1986, Chapters 6–8; Priestley, 1981, Chapter 9; Chatfield, 2001, Chapter 5). Of course, more specialized multivariate models may occasionally be required. For example, forecasts of births must take account of the number and age of women of child-bearing age. Common sense and background knowledge of the problem context should indicate what is required. 5.4 Comparative Review of Forecasting Procedures We noted in Section 5.1 that there is no such thing as a ‘best’ forecasting procedure, but rather that the choice of method depends on a variety of factors such as the objective in producing forecasts, the degree of accuracy required and the properties of the given time series. This section gives a brief review of relevant research and makes recommendations on which method to use and when. The context and the reason for making a forecast are, of course, of prime importance. Forecasts may be used, for example, for production planning in industry. They can also be used to assess the effects of different strategies by producing several forecasts. They may also be used as a ‘norm’ (or yardstick) against which the actual outcome may be assessed to see whether anything is changing. Univariate forecasts are usually used to provide such a ‘norm’ and are also particularly suitable when there are large numbers of series to be forecast (e.g. in stock control) so that a relatively simple method has to be used. They are also suitable when the analyst’s skill is limited or when they are otherwise judged appropriate for the client’s needs and level of understanding. Multivariate models are particularly appropriate for assessing the effects of explanatory variables, for understanding the economy, and for

< previous page

file:///C:/Documents and Settings/Yang/

page_90

next page >

面/The analysis of time series an introduction/files/page_90.html [5/24/2009 16:51:47]

page_91

< previous page

page_91

next page >

Page 91 evaluating alternative economic policy proposals by constructing a range of ‘what if’ forecasts. The latter strategy of computing more than one forecast is often very helpful in assessing the effects of different assumptions or strategies. 5.4.1 Forecasting competitions First, we review the empirical evidence as to which method is ‘best’. In order to clarify the choice between different univariate methods, there have been several ‘competitions’ to compare the forecasting accuracy of different methods on a given collection of time series. Some of the more important competitions are described by Newbold and Granger (1974), Makridakis and Hibon (1979), Makridakis et al. (1984), Makridakis et al. (1993) and Makridakis and Hibon (2000). The last three studies are commonly known as the Mcompetition, the M2-competition and the M3-competition, respectively. The M-competition was designed to be more wide ranging than earlier studies, and compared 24 methods on 1001 series, while the M3competition compared 24 methods on no fewer than 3003 series—a mammoth task! The M2 competition was much smaller, just 29 series, but this allowed non-automatic procedures to be investigated. Given different analysts and data sets, it is perhaps not too surprising that the results from different competitions have not always been consistent. For example, Newbold and Granger (1974) found that BoxJenkins tended to give more accurate forecasts than other univariate methods, but this was not the case in the M- and M3-competitions. A detailed assessment of the strengths and weaknesses of forecasting competitions is given by Chatfield (1988; 2001, Section 6.4). It is essential that results be replicable and that appropriate criteria are used. Moreover, accuracy is only one aspect of forecasting, and practitioners think that cost, ease of use and ease of interpretation are of almost equal importance. Furthermore, competitions mainly analyse large numbers of series in a completely automatic way. Thus although they tell us something, competitions only tell part of the story and are mainly concerned with comparing automatic forecasts. In practice, large gains in accuracy can often be made by applying a carefully-chosen method that is tailored to a particular context for a particular type of series—see, for example, Tashman and Kruk (1996) and Section 5.4.2 below. This will not be evident in the results from automatic forecasting competitions, where, for example, simple exponential smoothing (SES) has sometimes been applied to all series in a group regardless of whether a particular series exhibits trend or not. This is unfair to SES, which does not pretend to be able to cope with trend. One important question is how forecasting accuracy should be assessed—see, for example, Armstrong and Collopy (1992), Fildes (1992) and Chatfield (2001, Section 6.3). While average prediction mean square error is the obvious measure, care needs to be taken when averaging across series with widely differing variances. Because of this, the mean absolute percentage error is often used, rather than mean square error. Of course, a model that is ‘best’

< previous page

file:///C:/Documents and Settings/Yang/

page_91

next page >

面/The analysis of time series an introduction/files/page_91.html [5/24/2009 16:51:48]

page_92

< previous page

page_92

next page >

Page 92 under one criterion need not be best under some other criterion. In particular, it can be useful to use a robust measure that is not inflated too much by the occasional large forecast error. When comparing results from different competitions, it is also essential to ensure that genuine out-of-sample forecasts have been used throughout. When one method appears to give much better forecasts than all alternative methods, there must be a suspicion that it has an unfair advantage in some way. In my experience, a method that seems to be a clear winner may not be using genuine out-of-sample forecasts, but rather may be ‘cheating’ in some way, perhaps unwittingly. For example, multivariate forecasts sometimes outperform univariate methods only by dubiously using future values of explanatory variables. It is also worth noting that unexciting results tend to be suppressed. For example, forecasters seem eager to publish results that show a new method to be better, but not to publish results that show the reverse—see Chatfield (1995c) for examples. This sort of publication bias is endemic in other areas of statistical investigation, as people like to publish ‘significant’ results, but not the reverse. As a result, new methods tend to have an unfair advantage. Finally, it is worth saying that empirical results tend to be ignored (Fildes and Makridakis, 1995), especially when they run counter to orthodox thinking by showing, for example, that simple forecasting methods do as well as more complicated ones. Choosing an automatic method. If an automatic approach is desirable or unavoidable, perhaps because a large number of series is involved, then my interpretation of the competition results is as follows. While there could be significant gains in being selective, most users will want to apply the same method to all series for obvious practical reasons. Some methods should be discarded, but there are several automatic methods for which average differences in accuracy are small. Thus the choice between them may depend on other practical considerations such as availability of computer programs. The methods include Holt’s exponential smoothing (applied to seasonally adjusted data where appropriate), Holt-Winters and Bayesian forecasting. My particular favourite, partly on grounds of familiarity, is the Holt-Winters method, which can be recommended as a generally reliable, easy to understand method for seasonal data. The results in Chen (1997) have shown that Holt-Winters is robust to departures from the model for which the method is optimal. Holt’s method can be used for non-seasonal data or data that have already been deseasonalized. The use of a damped trend term is often advisable in both the Holt and Holt-Winters procedures. 5.4.2 Choosing a non-automatic method Suppose instead that a non-automatic approach is indicated because the number of series is small and/or because external information is available

< previous page

file:///C:/Documents and Settings/Yang/

page_92

next page >

面/The analysis of time series an introduction/files/page_92.html [5/24/2009 16:51:48]

page_93

< previous page

page_93

next page >

Page 93 that cannot be ignored. Then sensible forecasters will use their skill and knowledge to interact with their clients, incorporate background knowledge, plot the data and generally use all relevant information to build an appropriate model and compute sensible forecasts. The choice then lies between some form of multivariate method and a non-automatic univariate procedure. Here forecasting competitions are of limited value and it is easy to cite case studies where subjective adjustment of automatic forecasts leads to improvements (e.g. Chatfield, 1978). Moreover, the differences in accuracy for different methods when averaged over many series are relatively small compared with the large differences in accuracy that can arise when different methods are applied to individual series. The potential rewards in selecting an appropriate, perhaps non-automatic, method indicate that the distinction between an automatic and a non-automatic approach may be more fundamental than the differences between different forecasting methods. We look first at multivariate methods. It is possible to cite case studies (e.g. Jenkins and McLeod, 1982) where statistician and client collaborate to develop a successful multivariate model. However, it is difficult to make general statements about the relative accuracy of multivariate methods. Many people expect multivariate forecasts to be at least as good as univariate forecasts, but this is not true either in theory or in practice, and univariate methods outperform multivariate ones in about half the studies I have seen. One reason for this is that the computation of multivariate forecasts of a response variable may require the prior computation of forecasts of explanatory variables, and the latter must be sufficiently accurate to make this viable (Ashley, 1988). Multivariate models work best when one or more variables are leading indicators for the response variables of interest and so do not need to be forecast, at least for shorter lead times. For example, when trying to forecast a particular sales series, I found that regressing detrended, deseasonalized sales on detrended, deseasonalized stocks two quarters before, gave much better forecasts than univariate ones. In other words stocks are a leading indicator for sales. Two other general points are that multivariate models are more difficult to identify than univariate models, and that they are generally less robust to departures from model assumptions. For all these reasons, simple methods and models are often better. The empirical evidence is reviewed by Chatfield (1988, 2001). Regression models do rather better on average than univariate methods, though not by any means in every case (Fildes, 1985). Econometric simultaneous equation models have a patchy record and it is easy to cite cases where univariate forecasts are more accurate (e.g. Makridakis and Hibon, 1979, Section 2). There have been some encouraging case studies using transfer function models (e.g. Jenkins, 1979; Jenkins and McLeod, 1982), but such models assume there is no feedback, and this state of affairs will not apply to much multivariate economic data. Vector AR models, introduced later in Chapter 12, have a mixed record with some successes, but some failures as well— see, for example, the special issue of the Journal of Forecasting (1995, No. 3). Moreover,

< previous page

file:///C:/Documents and Settings/Yang/

page_93

next page >

面/The analysis of time series an introduction/files/page_93.html [5/24/2009 16:51:49]

page_94

< previous page

page_94

next page >

Page 94 many researchers fail to compare their multivariate forecasts with those from simpler alternatives, perhaps because they do not wish to be embarrassed by the possibility that univariate forecasts turn out better. It is arguable that multivariate models are more useful for understanding relationships than for forecasting. Of course multivariate models can usually be made to give a better fit to given data than univariate models, but this superiority does not necessarily translate into better forecasts, perhaps because multivariate models are more sensitive to changes in structure. It has to be realized that the nature of economic time-series data is such as to make it difficult to fit reliable multivariate time-series models. Most economic variables are simply observed, rather than controlled, and there are usually high autocorrelations within each series. In addition there may be high correlations between series, not necessarily because of a real relationship but simply because of a mutual correlation with time. Feedback between ‘output’ and ‘input’ variables is another problem. There are special difficulties in fitting regression models to time-series data anyway, as already noted in Section 5.3.1, and an apparent good fit may be spurious. Simultaneous equation and vector AR models are also difficult to construct, and their use seems likely to be limited to the analyst who is as interested in the modelling process as in forecasting. Thus although the much greater effort required to construct multivariate models will sometimes prove fruitful, there are many situations where a univariate method will be preferred. With a non-automatic univariate approach, the main choice is between the Box-Jenkins approach and the non-automatic use of a simple method, such as Holt-Winters, which is more often used in automatic mode. The Box-Jenkins approach has been one of the most influential developments in time-series analysis. However, the accuracy of the resulting forecasts has been rather mixed in practice, particularly when one realizes that forecasting competitions are biased in favour of Box-Jenkins by implementing other methods in a completely automatic way. The advantage of being able to choose from the broad class of ARIMA models is clear, but, as noted in Section 5.2.4, there are also dangers in that considerable experience is needed to interpret correlograms and other indicators. Moreover, when the variation in a series is dominated by trend and seasonality, the effectiveness of the fitted ARIMA model is mainly determined by the differencing procedure rather than by the identification of the autocorrelation structure of the differenced (stationary) series. Yet the latter is what is emphasized in the Box-Jenkins approach. Nevertheless, some writers have suggested that all exponential smoothing models should be regarded as special cases of Box-Jenkins, the implication being that one might as well use Box-Jenkins. However, this view is now discredited (Chatfield and Yar, 1988) because exponential smoothing methods are actually applied in a different way to BoxJenkins.

< previous page

file:///C:/Documents and Settings/Yang/

page_94

next page >

面/The analysis of time series an introduction/files/page_94.html [5/24/2009 16:51:50]

page_95

< previous page

page_95

next page >

Page 95 In some situations, a large expenditure of time and effort can be justified and then Box-Jenkins is worth considering. However, for routine sales forecasting, simple methods are more likely to be understood by managers and workers who have to utilize or implement the results. Thus I suggest that Box-Jenkins is only worth considering when the following conditions are satisfied: (1) the analyst is competent to implement the method and has appropriate software; (2) the objectives justify the additional complexity; and (3) the variation in the series is not dominated by trend and seasonality. If these conditions are not satisfied, then a non-automatic version of a simple method, such as Holt-Winters, may be ‘best’. Of course, another alternative to the Box-Jenkins approach is to use a more complicated method, such as one of the multivariate methods discussed earlier in this subsection. This can be especially rewarding when there is an obvious leading indicator to include. However, there are many situations when it is advisable to restrict attention to univariate forecasts. 5.4.3 A strategy for non-automatic univariate forecasting If circumstances suggest that a non-automatic univariate approach is appropriate, then I suggest that the following eight steps will generally provide a sensible strategy that covers many forecasting situations: 1. Get appropriate background information and carefully define the objectives. 2. Plot the data and look for trend, seasonal variation, outliers and any changes in structure such as slow changes in variance, sudden discontinuities and anything else that looks unusual. 3. ‘Clean’ the data if necessary, for example, by adjusting any suspect observations, preferably after taking account of external information. Consider the possibility of transforming the data. 4. Decide whether the seasonal variation is non-existent, multiplicative, additive or something else. 5. Decide whether the trend is non-existent, global linear, local linear or non-linear. 6. Fit an appropriate model where possible. It can be helpful to distinguish the following four types of series (although the reader may, of course, come across series that do not fit into any of these categories. Then common sense has to be applied):

< previous page

file:///C:/Documents and Settings/Yang/

page_95

next page >

面/The analysis of time series an introduction/files/page_95.html [5/24/2009 16:51:50]

page_96

< previous page

page_96

next page >

Page 96

Figure 5.1 Three types of time series. (a) Discontinuity present: numbers of new insurance policies issued file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_96.html (1 of 2) [5/24/2009 16:51:51]

page_96

by a particular life office (monthly in hundreds). (b) Short-term autocorrelation present: unemployment rate in U.S.A. (quarterly). (c) Exponential growth present: world-wide sales of IBM (yearly).

< previous page

file:///C:/Documents and Settings/Yang/

page_96

next page >

面/The analysis of time series an introduction/files/page_96.html (2 of 2) [5/24/2009 16:51:51]

page_97

< previous page

page_97

next page >

Page 97 (i) Discontinuities present. Figure 5.1(a) shows a series having a major discontinuity, where it is generally unwise to produce any univariate forecasts. There is further discussion of this series in Section 13.2. (ii) Trend and seasonality present. Figures 1.3 and 14.2 show series whose variation is dominated by trend and seasonality. Here the Holt-Winters exponential smoothing method is a suitable candidate. The correct seasonal form must be chosen and the smoothing parameters can then be estimated by optimizing one-stepahead forecasts over the period of fit. Full details are given by Chatfield and Yar (1988). For data showing trend, but no seasonality, Holt’s method may be used. (iii) Short-term correlation present. Figure 5.1(b) shows a non-seasonal series whose variation is dominated by short-term correlation. Many economic indicator series are of this form and it is essential to try to understand the autocorrelation structure. Here, the Box—Jenkins approach is recommended. (See Example 14.2.) (iv) Exponential growth present. Figure 5.1(c) shows a series dominated by an increasing trend. Series of this type are difficult to handle because exponential forecasts are inherently unstable. No one really believes that economic growth or population size, for example, can continue forever to increase exponentially. Two alternative strategies are to fit a model that explicitly includes exponential (or perhaps quadratic) growth terms, or (my preference) to fit a model to the logarithms of the data (or to some other suitable transformation of the data). It may help to damp any trend that is fitted to the transformed data as in Gardner and McKenzie (1985). 7. Check the adequacy of the fitted model. In particular, study the one-step-ahead forecast errors over the period of fit to see whether they have any undesirable properties such as significant autocorrelation. Modify the model if necessary. 8. Compute forecasts. Decide whether the forecasts need to be adjusted subjectively because of anticipated changes in other variables, or because of any other reason. Of course, much of the above procedure could be automated, making the dividing line between automatic and non-automatic procedures rather blurred. We should also mention rule-based forecasting (Collopy and Armstrong, 1992), which is a form of expert system that integrates skilled judgement with domain knowledge and also takes account of the features of the data. The rule base, consisting of as many as 99 rules, entertains a variety of forecasting procedures, which may be combined in an appropriate way. This rule base may be modified (e.g. Adya et al., 2000) depending on the particular situation.

< previous page

file:///C:/Documents and Settings/Yang/

page_97

next page >

面/The analysis of time series an introduction/files/page_97.html [5/24/2009 16:51:52]

page_98

< previous page

page_98

next page >

Page 98 5.4.4 Summary It is difficult to summarize the many empirical findings and other general principles that have been established in regard to forecasting, but I make the following general observations and recommendations: • As in all statistical work, it is essential to formulate the forecasting problem carefully, clarify objectives and be clear exactly how a forecast will be used. • The more frequent and the greater the number of forecasts required, the more desirable it is to use a simple approach. • Fitting the ‘best’ model to historical data does not necessarily lead to the most accurate out-of-sample forecast errors in the future. In particular, complex models usually give a better fit than simpler models but the resulting forecasts need not be more accurate. It is essential that all comparisons between different forecasting models and methods should be made using genuine out-of-sample forecasts. • Prediction intervals, calculated on the assumption that the model fitted to past data will also be true in the future, are generally too narrow. • If an automatic univariate method is required, then the Holt and HoltWinters versions of exponential smoothing are suitable candidates, but there are several close competitors. • When a non-automatic approach is appropriate, there is a wide choice from judgemental and multivariate methods through to (univariate) Box-Jenkins and the ‘thoughtful’ use of univariate methods that are usually regarded as being automatic. A general strategy for non-automatic univariate forecasting has been proposed that involves looking carefully at the data and selecting an appropriate method from a set of candidates including the Box-Jenkins, Holt and Holt-Winters methods. Whatever approach is used, the analyst should be prepared to improvise and modify ‘objective’ forecasts using subjective judgement. • It is often possible to get more accurate results by combining forecasts from different methods, perhaps by using a weighted average, but this does not lead to an informative model. Comprehensive coverage of the general principles involved in time-series forecasting, together with further guidance on empirical results, is given by Armstrong (2001) and Chatfield (2001). 5.5 Some Examples This section discusses three forecasting situations to illustrate some of the problems that arise in practice. Example 5.1 Forecasting in the presence of an outlier. Figure 5.2 shows the (coded) sales of a certain superstore in successive months over 4 years. As part of some consulting work, I was asked to produce point and interval forecasts for up to 1 year ahead. The problem related to an insurance claim and a large sum of money was involved.

< previous page

file:///C:/Documents and Settings/Yang/

page_98

next page >

面/The analysis of time series an introduction/files/page_98.html [5/24/2009 16:51:52]

page_99

< previous page

page_99

next page >

Page 99

Figure 5.2 Coded monthly sales for 4 years starting in August 1988. There was no prior information as to a suitable time-series model other than that the data were thought to be seasonal. So what forecasting method is appropriate? As in any time-series analysis, I began by looking at the time plot to assess trend, seasonality and any unusual features. In this particular study, the time plot shows some upward trend over the first couple of years, which flattens out towards the end of the series. There is also some seasonality, with three January figures being comparatively high. However, the first January observation in 1989 is unusually low compared with other January results, and the treatment of this ‘outlier’ (really an ‘inlier’!) took centre stage in the modelling/inference process. I asked the company why such an unusual value might have occurred. The simple answer was that no one seemed to know, presumably because the observation occurred several years ago. There was speculation that it was due to changes in accounting procedure that resulted in some sales being carried over into neighbouring months. The question now was what to do about it. We could have imputed the value, or used some sort of robust modelling procedure, but, given the proximity to the start of the series, it seemed more sensible to drop the first 6 months of data (which are the least relevant to forecasts). The drawback is that we then had to work with just years of data. This is shorter than one would like and suggests using a simple approach such as Holt-Winters exponential smoothing. However, the client preferred a model-

< previous page

file:///C:/Documents and Settings/Yang/

page_99

next page >

面/The analysis of time series an introduction/files/page_99.html [5/24/2009 16:51:53]

page_100

< previous page

page_100

next page >

Page 100 based approach, and so seasonal ARIMA modelling was tried even though the length of the series was below the usual recommended minimum length. After looking at various forms of differencing ( , 12, 12, ) and at the usual diagnostic tools (mainly the various correlograms), it was unfortunately clear that there was no obvious model. The original data have a correlogram that looks clearly non-stationary, but the seasonal differences still look somewhat non-stationary, probably because the trend is changing. The use of 12 doesn’t improve things much, while increases the variance of the series and ‘loses’ too many observations for such a short series. After trying various models, we found the best model for 12Xt and for 12Xt and took the average of the forecasts given by these two models. Further details will not be given here. This was not entirely satisfactory but seemed the best that could be done with a Box-Jenkins approach, although Holt-Winters might have been ‘safer’. Fortunately the client appeared satisfied (though that is not always a conclusive indicator!). Further details on this example are given in Chatfield (2002). The example demonstrates, once again, the importance of plotting a time series and making a visual examination before deciding on the appropriate forecasting procedure. If an outlier occurs close to the end of a series, the above procedure (omitting some early observations) could not be used. Instead, we would need to ask questions to decide what to do. For example, we could ask whether the outlier indicates a permanent change in the seasonal pattern, in which case earlier observations will have little relevance for forecasting purposes, or it is thought to be a one-off phenomenon that we can allow for in some way, perhaps by adjusting the outlying observation. It is hard to give general advice, except to say that it would be most unwise to produce forecasts from the given series as if nothing were amiss. Example 5.2 Forecasting a series dominated by trend and seasonality. Many time series exhibit an obvious trend and seasonal pattern. For example, looking back to Figure 1.3 or ahead to Figure 14.2, the reader can see two such examples. In order to model and forecast such series, we must choose a method or model that reflects the observed properties. Two obvious candidates are seasonal ARIMA (SARIMA) models and the HoltWinters method. How should we decide which, if either, to use? Consider the data of Figure 1.3 as an example. This depicts the sales of an engineering product in successive months over a 7-year period. A detailed description of an early attempt to model the first 77 observations using SARIMA models is given by Chatfield and Prothero (1973). There were many difficulties in doing this. For example, the seasonality is not additive, so that the data must be transformed in order to use the BoxJenkins approach. A logarithmic transformation was originally used, but other writers have argued that a cube root transformation would be better. Unfortunately, the latter makes model interpretation more difficult. Chatfield and Prothero developed a SARIMA model of order (1,1,0)(0,1,1)12 for the logged data, but this gave

< previous page

file:///C:/Documents and Settings/Yang/

page_100

next page >

面/The analysis of time series an introduction/files/page_100.html [5/24/2009 16:51:54]

page_101

< previous page

page_101

next page >

Page 101 forecasts that seemed intuitively implausible. They therefore looked at several alternative SARIMA models until they found one that gave sensible forecasts and for which the diagnostic checks did not indicate model rejection. However, this meant that the whole model-building exercise was rather unsatisfactory, as pointed out by several discussants to the original 1973 paper. Given that the series has a substantial trend and a large, regular seasonal pattern, there is much to be said for using a simpler method than Box-Jenkins. As noted in Section 4.8, when the variation due to trend and seasonality is dominant, the effectiveness of an ARIMA model is mainly determined by the initial differencing operations and not by the time-consuming ARMA model fitting to the differenced series. For regular seasonal data, nearly any forecasting method will give good results, and we found that the Holt-Winters method explains a high proportion of the variation in the series. It seems doubtful whether the extra expense of the Box-Jenkins method can be justified in this case and my recommendation nowadays would be to use the multiplicative version of Holt-Winters on the untransformed data. The interested reader may enjoy the lively discussion following our 1973 paper! A detailed analysis of a second series dominated by trend and seasonal is given later in Chapter 14 as Example 14.3. Many difficulties arose in carrying out ARIMA modelling, and the Holt-Winters method may be preferred there as well. More generally there have now been many studies comparing forecast accuracy on different time series, and the results show that when trend and seasonality account for more than about 90% of the total variation, then Holt-Winters forecasts are typically comparable to Box-Jenkins. Example 5.3 The importance of context. This example demonstrates the importance of understanding the context of any forecasting problem. A large manufacturing company sought my advice on the best way of forecasting the future values of a particular time series. Despite having many statistical problems, it turned out that the company did not employ a specialist statistician-a sad situation. My client showed me the data, which consisted of just 11 quarterly observations—a very short series. The observed variable was described as the ‘number of services and part orders in one quarter’ for a particular model of a type of electrical appliance. A ‘part order’ was explained as involving the supply of one or more spare part replacements for a single machine. The data have been coded to preserve confidentiality and the scaled data are: 1579, 1924, 1868, 2068, 2165, 1754, 2265, 2288, 2526, 2568 and 2722, starting in 1998, Quarter(Q)1. The series is plotted in Figure 5.3 and shows an upward trend that is approximately linear. There is no apparent seasonality. The value in 1999, Q2, seems unusually low, but my client could provide no explanation for this. The series is rather short to use any sort of sophisticated forecasting procedure. The company had heard of exponential smoothing and wondered whether that would be an appropriate method to use.

< previous page

file:///C:/Documents and Settings/Yang/

page_101

next page >

面/The analysis of time series an introduction/files/page_101.html [5/24/2009 16:51:54]

page_102

page_102

< previous page

next page >

Page 102

Figure 5.3 The number of services required for a particular type of consumer durable in successive quarters from 1998 Q1 to 2000 Q3. My first reaction was that the series was so short that any simple univariate projection method would be reasonable, provided that it allowed for trend. However, before committing myself, I asked some questions to get further background information so as to better understand the context. In particular, I needed to find out why a forecast was required and what would be done with it. I soon ascertained the key information that production of the particular brand was shortly to be stopped so that a new model could be introduced. Forecasts were desired so as to plan the rundown of work on the brand. Sales through shops would continue for a while but would be phased out within 3–5 months. Thus the upward trend in the data would soon cease. Then, almost by chance, I discovered the second crucial item of information, namely, that the observed variable was not as originally described, but was actually the ‘number of services and part orders for machines less than 2 years old’. This is because the company is only directly involved in servicing and supplying spare parts when a machine is covered by warranty. The latter expires when a machine is 2 years years. old. This means that the observed series will soon start decreasing and should die out within Using Figure 5.3 by itself, it seems reasonable to use something like Holt’s linear trend version of exponential smoothing. However, the contextual information shows that this would be completely inappropriate. Instead we know that all values more than

< previous page

file:///C:/Documents and Settings/Yang/

years ahead will be zero, and so these

page_102

next page >

面/The analysis of time series an introduction/files/page_102.html [5/24/2009 16:51:55]

page_103

< previous page

page_103

next page >

Page 103 forecasts can be made with certainty! In the interim, some sort of damping procedure seems appropriate. This would best be carried out in-house by people familiar with the problem, and it is still not clear to me why the company wanted my help. Simple extrapolation is totally inappropriate, and my time-series expertise appears wasted. In fact, it was my more general skills as a problem solver that proved useful—further comments on this aspect of the problem are given in Chatfield (2002). 5.6 Prediction Theory This section3 gives a brief introduction to the general theory of linear prediction, which has been developed by Kolmogorov, Wiener (1949), Yaglom (1962) and Whittle (1983) among others. All these authors avoid the use of the word ‘forecasting’, although most of the univariate methods considered in Section 5.2 are in the general class of linear predictors. The theory of linear prediction has applications in control and communications engineering and is of considerable theoretical interest, but readers who wish to tackle the sort of forecasting problem we have been considering earlier in this chapter will find this literature less accessible, and less relevant to real-life problems, than earlier references, such as those describing the BoxJenkins approach. Two types of problems are often distinguished. In the first type of problem we have data up to time N, say {xN, xN−1,…}, and wish to predict the value of xN+h. One approach is to use the predictor

which is a linear function of the available data. The weights {cj} are chosen so as to minimize the expected mean square prediction error . This is often called the prediction problem (e.g. Cox and Miller, 1968), while Yaglom (1962) refers to it as the extrapolation problem and Whittle (1983) calls it pure prediction. As an example of the sort of result that has been obtained, Wiener (1949) has considered the problem of evaluating the weights {cj}, so as to find the best linear predictor, when the underlying process is assumed to be stationary with a known autocorrelation function, and when the entire past sequence of observations, namely, {xt} for t≤N, is assumed known. It is interesting to compare this sort of approach with the forecasting techniques proposed earlier in this chapter. Wiener, for example, says little or nothing about identifying an appropriate model and then estimating the model parameters from a finite sample, and yet these are the sort of problems that have to be faced in real-life applications. The BoxJenkins approach does tackle this sort of problem, and aims to employ a linear predictor that is optimal for a particular ARIMA process, while recognizing that the form of the model and the model parameters are usually not known beforehand and have to be inferred from the data. 3 This section may be omitted at first reading.

< previous page

file:///C:/Documents and Settings/Yang/

page_103

next page >

面/The analysis of time series an introduction/files/page_103.html [5/24/2009 16:51:56]

page_104

page_104

< previous page

next page >

Page 104 The second type of problem tackled in the more theoretical literature arises when the process of interest, called the signal, is contaminated by noise, and we actually observe the process where s(t) and n(t) denote the signal and noise, respectively. In some situations the noise is simply measurement error; in engineering applications the noise could be an interference process of some kind. The problem now is to separate the signal from the noise. Given measurements on y(t) up to time T we may want to reconstruct the signal up to time T or alternatively make a prediction of s(T+ ). The problem of reconstructing the signal is often called smoothing or filtering. The problem of predicting the signal is also sometimes called filtering (Yaglom, 1962; Cox and Miller, 1968), but is sometimes called prediction (Astrom, 1970). To make progress, it is often assumed that the signal and noise processes are uncorrelated and that s (t) and n(t) have known autocorrelation functions. These assumptions, which are unlikely to hold in practice, make the results of limited practical value. It is clear that both the above types of problems are closely related to the control problem because, if we can predict how a process will behave, then we can adjust the process so that the achieved values are, in some sense, as close as possible to the target value. Control theory is generally outside the scope of this book, although we do make some further brief remarks on the topic later in Section 13.6 after we have studied the theory of linear systems. Exercises 5.1 For the MA(1) model given by show that

and that

Show that the variance of the h-steps-ahead forecast error is given by

h≥2, provided the true model is known. (In practice we would take squares estimate of θ and , is the observed residual at time N.) 5.2 For the AR(1) model given by show that

for h=1, and by

, where

for is the least

for h=1, 2,…. Show that the variance of the h-steps-ahead forecast error is

given by . For the AR(1) model, with non-zero mean µ, given by show that . for h=1, 2,…. (In practice the least squares estimates of α and µ would need to be substituted into the above formulae.)

< previous page

file:///C:/Documents and Settings/Yang/

page_104

next page >

面/The analysis of time series an introduction/files/page_104.html [5/24/2009 16:51:57]

page_105

page_105

< previous page

next page >

Page 105 5.3 Consider the SARIMA(1, 0, 0)×(0, 1, 1)12 model used as an example in Section 5.2.4. Show that 5.4 For the SARIMA(0, 0, 1)×(1, 1, 0)12 model, find forecasts at time N for up to 12 steps ahead in terms of observations and estimated residuals up to time N. 5.5 For the model (1−B)(1−0.2B) Xt=(1−0.5B)Zt in Exercise 3.12, find forecasts for one and two steps ahead, and show that a recursive expression for forecasts three or more steps ahead is given by Find the variance of the one-, two- and three-steps-ahead forecast errors. If zN =1, xN=4, xN−1=3 and , show that and that the standard error of the corresponding forecast error is 1.72. 5.6 Consider the ARIMA(0, 1, 1) process Show that

for h ≥ 2. Express

and

in terms of xN

and show that this is equivalent to exponential smoothing. By considering the an the process, show that the variance of the h-steps-ahead prediction error is

< previous page

file:///C:/Documents and Settings/Yang/

page_105

weights of

next page >

面/The analysis of time series an introduction/files/page_105.html [5/24/2009 16:51:57]

page_106

< previous page

page_106

next page >

page_106

next page >

Page 106 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_106.html [5/24/2009 16:51:58]

page_107

< previous page

page_107

next page >

Page 107 CHAPTER 6 Stationary Processes in the Frequency Domain 6.1 Introduction Chapter 3 described several types of stationary stochastic processes, such as autoregressive processes. When discussing their properties, we emphasised the autocovariance (or autocorrelation) function, as being the natural tool for considering the evolution of a process through time. This chapter introduces a complementary function, called the spectral density function, which is the natural tool for considering the frequency properties of a time series. Inference regarding the spectral density function is called an analysis in the frequency domain. Some statisticians initially have difficulty in understanding the frequency approach, but the advantages of frequency methods are widely appreciated in such fields as electrical engineering, geophysics and meteorology. These advantages will become apparent in the next few chapters. We confine attention to real-valued processes. Some authors consider the more general problem of complexvalued processes, and this has some mathematical advantages. However, in my view, the reader is more likely to understand an approach restricted to real-valued processes. The vast majority of practical problems are covered by this approach. 6.2 The Spectral Distribution Function In order to introduce the idea of a spectral density function, we first consider a function called the spectral distribution function. The approach adopted is heuristic and not mathematically rigorous, but will hopefully give the reader a better understanding of the subject than a more theoretical approach. Suppose we suspect that a time series contains a periodic sinusoidal component with a known wavelength. Then a natural model is (6.1) is called the frequency of the sinusoidal variation, R is called the amplitude of the variation, is called the phase and {Zt} denotes a stationary random series. Note that the angle ( + ) is usually measured in units called radians, where π radians=180°. As is the number of radians per unit time, it is sometimes called the angular frequency, but in keeping with most authors we simply call the frequency. However, some authors, where

< previous page

file:///C:/Documents and Settings/Yang/

page_107

next page >

面/The analysis of time series an introduction/files/page_107.html [5/24/2009 16:51:58]

page_108

< previous page

page_108

next page >

Page 108 notably Jenkins and Watts (1968), use the term ‘frequency’ to refer to the number of cycles per unit time,

, and this form of frequency is easier to interpret from a physical point of view. We namely, usually use the angular frequency in mathematical formulae, because it makes them more concise, but we will often use the frequency

to interpret the results of a data analysis. The period of a

sinusoidal cycle, sometimes called the wavelength, is clearly l/ƒ or

. Figure 6.1 shows an example of

, so that . The wavelength is a sinusoidal function with angular frequency the reciprocal of f, namely, 6, and inspection of Figure 6.1 shows that this is indeed the number of time units between successive peaks or successive troughs of the sinusoid.

Figure 6.1 A graph of R cos

with

and . Model (6.1) is a very simple model, but in practice the variation in a time series may be caused by variation at several different frequencies. For example, sales figures may contain weekly, monthly, yearly and other cyclical variation. In other words the data show variation at high, medium and low frequencies. It is natural therefore to generalize Equation (6.1) to

(6.2) where Rj, j denote the amplitude and phase, respectively, at frequency j. The reader will notice that the models in Equations (6.1) and (6.2) are not stationary if parameters such as

{Rj}, { j} and { j} are all fixed constants, because E(Xt) will change with time. In order to apply the theory of stationary processes to models like Equation (6.2), it is customary to assume that {Rj} are (uncorrelated) random variables with mean zero, or that { j} are random variables with a uniform distribution on (0, 2π), which are fixed for a single realization of the process (see Section 3.5 and Exercise 3.14). This is something

< previous page

file:///C:/Documents and Settings/Yang/

page_108

next page >

面/The analysis of time series an introduction/files/page_108.html [5/24/2009 16:51:59]

page_109

< previous page

page_109

next page >

Page 109 of a ‘mathematical trick’, but it does enable us to treat time series containing one or more deterministic sinusoidal components as stationary series. , the model in Equation (6.2) can alternatively be Since expressed as a sum of sine and cosine terms in the form

(6.3) and . where However, we may now ask why there should only be a finite number of frequencies involved in Equations (6.2) or (6.3). In fact, letting k→∞, the work of Wiener and others has shown that any discrete-time stationary process measured at unit intervals may be represented in the form (6.4) where u( ), υ( ) are uncorrelated continuous processes, with orthogonal increments (see Section 3.4.8), which are defined for all in the range (0, π). Equation (6.4) is called the spectral representation of the process; it involves stochastic integrals, which require considerable mathematical skill to handle properly. It is intuitively more helpful to ignore these mathematical problems and simply regard Xt as a linear combination of orthogonal sinusoidal terms. Thus the derivation of the spectral representation will not be considered here (see, for example, Cox and Miller, 1968, Chapter 8). The reader may wonder why the upper limits of the integrals in Equation (6.4) are π rather than ∞. For a continuous process the upper limits would indeed be ∞, but for a discrete-time process measured at unit intervals of time there is no loss of generality in restricting to the range (0, π), since

and so variation at frequencies higher than π cannot be distinguished from variation at a corresponding frequency in (0, π). The frequency =π is called the Nyquist frequency. We will say more about this in Section 7.2.1. For a discrete-time process measured at equal intervals of time of length Δt, the Nyquist frequency is π/Δt. In the next two sections we consider discrete-time processes measured at unit intervals of time, but the arguments carry over to any discrete-time process, measured at an arbitrary time interval Δt if we replace π by π/Δt. The main point of introducing the spectral representation in Equation (6.4) is to show that every frequency in the range (0, π) may contribute to the variation of the process. However, the processes u( ) and υ( ) in Equation (6.4) are of little direct practical interest. Instead we introduce a single function, F( ), called the (power) spectral distribution function. This function is related to the autocovariance function and provides an intuitively

< previous page

file:///C:/Documents and Settings/Yang/

page_109

next page >

面/The analysis of time series an introduction/files/page_109.html [5/24/2009 16:52:00]

page_110

< previous page

page_110

next page >

Page 110 understandable description of the frequency properties of a stationary process. It arises from a theorem, called the Wiener-Khintchine theorem, named after N.Wiener and A.Y.Khintchine, which says that for any real-valued stationary stochastic process with autocovariance function γ(k), there exists a monotonically1 increasing function F( ) such that (6.5) Equation (6.5) is called the spectral representation of the autocovariance function, and involves a type of integral (called Stieltjes) that may be unfamiliar to some readers. However, it can be shown that the function F( ) has a direct physical interpretation: it is the contribution to the variance of the series, which is accounted for by frequencies in the range (0, ). If variation at negative frequencies is not allowed, then For a discrete-time process measured at unit intervals of time, the highest possible frequency is the Nyquist frequency π and so all the variation is accounted for by frequencies less than π. Thus This last result also comes directly from Equation (6.5) by putting k=0, when we have

In between

and

is monotonically increasing.

If the process contains a deterministic sinusoidal component at frequency

, say

where

R is a constant and is uniformly distributed on (0, 2π), then there will be a step increase in F( ) at equal to the contribution to variance of this particular component. As the component has mean zero, the contribution to variance is just the average squared value, namely, As F( ) is monotonically increasing, it can be decomposed into two functions, F1( ) and F2(

. ), such that

(6.6) where F1( ) is a non-decreasing continuous function and F2( ) is a non-decreasing step function. This decomposition usually corresponds to the Wold decomposition, with F1( ) relating to the purely indeterministic component of the process and F2( ) relating to the deterministic component. We are mainly concerned with purely indeterministic processes, where F2( )≡0, in which case F( ) is a continuous function on (0, π). The adjective ‘power’, which is sometimes prefixed to ‘spectral distribution 1 A function, say ƒ(x), is said to be monotonically increasing with x if ƒ(x) never decreases as x increases. Thus ƒ(x) will either increase or stay constant.

< previous page

file:///C:/Documents and Settings/Yang/

page_110

next page >

面/The analysis of time series an introduction/files/page_110.html [5/24/2009 16:52:01]

page_111

< previous page

page_111

next page >

Page 111 function’, derives from the engineer’s use of the word in connection with the passage of an electric current through a resistance. For a sinusoidal input, the power is directly proportional to the squared amplitude of the oscillation. For a more general input, the power spectral distribution function describes how the power is distributed with respect to frequency. In the case of a time series, the variance may be regarded as the total power. Note that some authors use a normalized form of F( ) given by (6.7) Thus F*( ) is the proportion of variance accounted for by frequencies in the range (0, ). Since F*(0)=0, F* (π)=1, and F*( ) is monotonically increasing in (0, π), F*( ) has similar properties to the cumulative distribution function of a random variable. 6.3 The Spectral Density Function For a purely indeterministic discrete-time stationary process, the spectral distribution function is a continuous (monotone bounded) function in [0, π], and may therefore be differentiated2 with respect to in (0, π). We will denote the derivative by ƒ( ), so that (6.8) This is the power spectral density function. The term ‘spectral density function’ is often shortened to spectrum, and the adjective ‘power’ is sometimes omitted. When f( ) exists, Equation (6.5) can be expressed in the form (6.9) This is an ordinary (Riemann) integral and therefore much easier to handle than (6.5). Putting k=0, we have (6.10) The physical interpretation of the spectrum is that

represents the contribution to variance of

. When the spectrum is drawn, Equation (6.10) components with frequencies in the range indicates that the total area underneath the curve is equal to the variance of the process. A peak in the spectrum indicates an important contribution to variance at may not be differentiable on a set of measure zero, but this is usually of no 2 Strictly speaking, practical importance. In particular, may not be differentiable at the endpoints 0 and π and so it is more accurate to define the derivative over the open interval (0, π) that excludes the endpoints. This does . In fact the formula in Equation (6.12) for , which is given later, not affect any integrals involving usually can be evaluated at the end-points.

< previous page

file:///C:/Documents and Settings/Yang/

page_111

next page >

面/The analysis of time series an introduction/files/page_111.html [5/24/2009 16:52:02]

page_112

< previous page

page_112

next page >

Page 112 frequencies near the value that corresponds to the peak. An example of a spectrum is shown in Figure 6.2, together with the corresponding normalized spectral distribution function. It is important to realise that the autocovariance function (acv.f.) and the power spectral density function are equivalent ways of describing a stationary stochastic process. From a practical point of view, they are complementary to each other. Both functions contain the same information but express it in different ways. In some situations a time-domain approach based on the acv.f. is more useful, while in other situations a frequency-domain approach is preferable.

Figure 6.2 An example of a spectrum, together with the corresponding normalized spectral distribution function. Equation (6.9) expresses γ(k) in terms of f( ) as a cosine transform. It can be shown that the corresponding inverse relationship is given by

(6.11) so that the spectrum is the Fourier transform3 of the acv.f. Since γ(k) is an even function of k, Equation (6.11) is often written in the equivalent form

(6.12) Note that if we try to apply Equation (6.12) to a process containing a deterministic component at a particular frequency , then will not converge when . This arises because has a and so will not be differentiable at . Thus its derivative f( ) will not be defined at step change at . The reader should note that several other definitions of the spectrum are given in the literature, most of which differ from Equation (6.12) by a constant 3 See Appendix A for details on the Fourier transform.

< previous page

file:///C:/Documents and Settings/Yang/

page_112

next page >

面/The analysis of time series an introduction/files/page_112.html [5/24/2009 16:52:03]

page_113

page_113

< previous page

next page >

Page 113 multiple and by the range of definition of range (−π,π) by

. The most popular approach is to define the spectrum in the

(6.13)

whose inverse relationship (see Appendix A) is

(6.14) as the frequency variable Jenkins and Watts (1968) use these equations, except that they take (see Equations (A.3) and (A.4)). Equations (6.13) and (6.14), which form a Fourier transform pair, are the more usual form of the Wiener-Khintchine relations. The formulation is slightly more general in that it can be applied to complex-valued time series. However, for real time series we find that f( ) is an even function of , and then we need only consider f( ) for >0. In my experience the introduction of negative frequencies, while having certain mathematical advantages, serves only to confuse the student. As we are concerned only with real-valued processes, we prefer Equation (6.11) defined on (0, π). It is sometimes useful to use a normalized form of the spectral density function, given by (6.15) This is the derivative of the normalized spectral distribution function (see Equation (6.7)). Then we find that is the Fourier transform of the autocorrelation function (ac.f.), namely,

(6.16) is the proportion4 of variance in the interval This means that 6.4 The Spectrum of a Continuous Process

.

For a continuous purely indeterministic stationary process, X(t), the autocovariance function,

, is

, is defined for all positive . The defined for all and the (power) spectral density function, relationship between these functions is very similar to that in the discrete-time case except that 4 As an example of the difficulties in comparing formulae from different sources, Kendall et al. (1983, Equation 47.20) define the spectral density function in terms of the autocorrelation function over the same range, (0, π), as we do, but omit the constant 1/π from Equation (6.16). This makes it more difficult to give the function a physical interpretation. Instead they introduce an intensity function that corresponds to our power spectrum.

< previous page

file:///C:/Documents and Settings/Yang/

page_113

next page >

面/The analysis of time series an introduction/files/page_113.html [5/24/2009 16:52:04]

page_114

page_114

< previous page

next page >

Page 114 there is no upper bound to the frequency. We have

for 0


Page 115 for 0< 0 the power is concentrated at low frequencies, giving what is called a low-frequency spectrum; if β

面/The analysis of time series an introduction/files/page_115.html [5/24/2009 16:52:06]

page_116

< previous page

page_116

next page >

Page 116 since . The shape of the spectrum depends on the value of α. When a>0, the spectral density function is ‘large’ when is ‘small’, so that power is concentrated at low frequencies—a low-frequency spectrum. On the other hand, when a

is a random variable having a uniform distribution on (0, 2π). As

explained in Section 3.5, is fixed for a single realization of the process and Equation (6.24) defines a purely deterministic process. The acv.f. of the process is given by

which we note does not tend to zero as k increases. This contrasts with the behaviour of the ac.f. of a stationary linearly indeterministic process, which does tend to zero as k increases. Put loosely, we can say that the above deterministic process has a long memory. From the model Equation (6.24), it is obvious that all the ‘power’ of the process is concentrated at the . Since E(Xt)=0, we find that frequency distribution function is given by

, so that the power spectral

and so the spectrum is not defined at . If we Since this is a step function, it has no derivative at nevertheless try to use Equation (6.12) to obtain the spectrum as the Fourier transform of the acv.f., then we find that does not converge at . This confirms that the spectrum is as expected, but that . not defined at (6) A mixture of deterministic and stochastic components Our final example contains a mixture of deterministic and stochastic components, namely, where

,

variance

are as defined in Example 5 above, and {Zt} is a purely random process with mean zero and . Then we find that the acv.f. is given by

As in Example 5 above, note that γ(k) does not tend to zero, as k increases, because Xt contains a periodic deterministic component. We can obtain the power spectral distribution function of Xt by using Equation (6.6), since the deterministic component

has power spectral distribution function

< previous page

file:///C:/Documents and Settings/Yang/

page_117

next page >

面/The analysis of time series an introduction/files/page_117.html [5/24/2009 16:52:08]

page_118

< previous page

page_118

next page >

Page 118 while the stochastic component Zt has power spectral distribution function on integrating Equation (6.19). Thus the combined power spectral distribution function is given by

As in Example 5, the distribution function has a step at . Exercises

and so the power spectrum is not defined at

In the following questions {Zt} denotes a purely random process, mean zero and variance . 6.1 Find (a) the power spectral density function, (b) the normalized spectral density function of the firstorder AR process with | |

面/The analysis of time series an introduction/files/page_118.html [5/24/2009 16:52:08]

page_119

page_119

< previous page

next page >

Page 119 the state variable at time t. If the process is started at time t=0 with the two possible states having equal , show that the process is second-order

probability (so that stationary, with autocorrelation function and spectral density function

6.6 Show that if {Xt} and {Yt} are independent, stationary processes with power spectral density functions and

, then {Vt}={Xt+Yt} is also stationary with power spectral density function . If Xt is a first-order AR process

and {Yt}, {Wt} are independent purely random processes with zero mean and common variance σ2, show that the power spectral density function of {Vt} is given by 6.7 Show that the normalized spectral density function of the ARMA(1, 1) process (with |α|

page_120

next page >

Page 120 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_120.html [5/24/2009 16:52:10]

page_121

< previous page

page_121

next page >

Page 121 CHAPTER 7 Spectral Analysis Spectral analysis is the name given to methods of estimating the spectral density function, or spectrum, of a given time series. Before about 1900, research workers such as A.Schuster were essentially concerned with looking for ‘hidden periodicities’ in data at one or two specific frequencies. Spectral analysis as we know it today is concerned with estimating the spectrum over the whole range of frequencies. The techniques are widely used by many scientists, particularly in electrical engineering, physics, meteorology and marine science. We are mainly concerned with purely indeterministic processes, which have a continuous spectrum, but the techniques can also be used for deterministic processes to pick out periodic components in the presence of noise. 7.1 Fourier Analysis Traditional spectral analysis is essentially a modification of Fourier analysis so as to make it suitable for stochastic rather than deterministic functions of time. Fourier analysis (e.g. Priestley, 1981) is essentially concerned with approximating a function by a sum of sine and cosine terms, called the Fourier series representation. Suppose that a function ƒ(t) is defined on (−π, π]1 and satisfies the so-called Dirichlet conditions. These conditions ensure that ƒ(t) is reasonably ‘well behaved’, meaning that, over the range (−π, π], ƒ(t) is absolutely integrable, has a finite number of discontinuities, and has a finite number of maxima and minima. Then ƒ(t) may be approximated by the Fourier series

where

1 The different-shaped brackets indicate that the lower limit −π is not included in the interval, while the square bracket indicates that the upper limit +π is included.

< previous page

file:///C:/Documents and Settings/Yang/

page_121

next page >

面/The analysis of time series an introduction/files/page_121.html [5/24/2009 16:52:10]

page_122

page_122

< previous page

next page >

Page 122 It can be shown that this Fourier series converges to ƒ(t) as k→∞ except at points of discontinuity, where it converges to halfway2 up the step change. In order to apply Fourier analysis to discrete time series, we need to consider the Fourier series representation of ƒ(t) when ƒ(t) is defined only on the integers 1, 2,…, N. Rather than write down the formula, we demonstrate that the required Fourier series emerges naturally by considering a simple sinusoidal model. 7.2 A Simple Sinusoidal Model Suppose we suspect that a given time series, with observations made at unit time intervals, contains a deterministic sinusoidal component at a known frequency , together with a random error term. Then we will consider the model (7.1) where Zt denotes a purely random process, and µ, α, β are parameters to be estimated from the data. The observations will be denoted by (x1, x2 ,…, xN). The algebra in the next few sections is somewhat simplified if we confine ourselves to the case where N is even. There is no real difficulty in extending the results to the case where N is odd (e.g. Anderson, 1971), and indeed many of the later estimation formulae apply for both odd and even N, but some results require one to consider odd N and even N separately. Thus, if N happens to be odd and a spectral analysis is required, computation can be made somewhat simpler by removing the first observation so as to make N even. If N is reasonably large, little information is lost. Expected values for the model in Equation (7.1) can be represented in matrix notation by where

As this model is linear in the parameters µ, α and β, it is an example of a general linear model. In that case the least squares estimate of θ, which minimizes known’ to be

, is ‘well

2 Mathematicians say that this is the average of the limit from below and the limit from above, sometimes written as

< previous page

file:///C:/Documents and Settings/Yang/

.

page_122

next page >

面/The analysis of time series an introduction/files/page_122.html [5/24/2009 16:52:11]

page_123

page_123

< previous page

next page >

Page 123 where The above formulae hold for any value of the frequency , but they only make practical sense for values of that are not too high or too low. As noted in Section 6.2, the highest frequency we can uniquely fit to the data is the Nyquist frequency, given by =π, which completes one cycle every two observations. At the other end of the spectrum, the lowest frequency we can reasonably fit completes one cycle in the whole length of the time series. These upper and lower limits will be explained further in Section 7.2.1 below. By equating the cycle length

to N, we find that the lowest frequency is given by 2π/N. The formulae for

the least squares estimates of

turn out to be particularly simple if

, is restricted to one of the values

which lie in equal steps from the lowest frequency 2π/N to the Nyquist frequency π. In this case, it turns out that (ATA) is a diagonal matrix in view of the following ‘well-known’ trigonometric results (all summations are for t=1 to N): (7.2)

(7.3)

(7.4) (7.5) With (ATA) diagonal, it is easy to evaluate the least squares estimate of θ, as the inverse (ATA)−1 will also be diagonal. For such that p≠N/2, we find (Exercise 7.2)

If p=N/2, we ignore the term in β sin

, which is zero for all t, and find

(7.6)

(7.7) The model in Equation (7.1) is essentially the one used before about 1900 to search for hidden periodicities, but this model has now gone out of fashion. However, it can still be useful if there is reason to suspect that a time series

< previous page

file:///C:/Documents and Settings/Yang/

page_123

next page >

面/The analysis of time series an introduction/files/page_123.html [5/24/2009 16:52:12]

page_124

< previous page

page_124

next page >

Page 124 does contain a deterministic periodic component at a known frequency and it is desired to isolate this component (e.g. Bloomfield, 2000, Chapters 2, 3). Readers who are familiar with the analysis of variance (ANOVA) technique will be able to work out that the total corrected sum of squared deviations, namely,

can be partitioned into two components, namely, the residual sum of squares and the sum of squares ‘explained’ by the periodic component at frequency . This latter component is given by

which, after some algebra (Exercise 7.2), can be shown to be

(7.8) using Equations (7.2)–(7.5). 7.2.1 The highest (Nyquist) frequency and the lowest (fundamental) frequency When fitting the simple sinusoidal model in Equation (7.1), we restricted the frequency to one of the values (2π/N, 4π/N,…, π), assuming that N is even. Here we examine the practical rationale for the upper and lower limits, namely, π and 2π/N. In Section 6.2, we pointed out that, for a discrete-time process measured at unit intervals of time, there is no loss of generality in restricting the spectral distribution function to the range (0, π). We now demonstrate that the upper bound π, called the Nyquist frequency, is indeed the highest frequency about which we can get meaningful information from a set of data. First, we give a more general form for the Nyquist frequency. If observations are taken at equal intervals of time of length Δt, then the Nyquist (angular) frequency is given by

. The equivalent

. frequency expressed in cycles per unit time is Consider the following example. Suppose that temperature readings are taken every day in a certain town at noon. It is clear that these observations will tell us nothing about temperature variation within a day. In particular, they will not tell us whether nights are hotter or cooler than days. With only one observation per radians per day or cycle per day (or 1 cycle per 2 days). day, the Nyquist frequency is This is lower than the frequencies, which correspond to variation within a day. For example, variation with a period of 1 day has (angular) frequency radians per day or f=1 cycle per day. In order to get information about variation within

< previous page

file:///C:/Documents and Settings/Yang/

page_124

next page >

面/The analysis of time series an introduction/files/page_124.html [5/24/2009 16:52:13]

page_125

< previous page

page_125

next page >

Page 125 a day at these higher frequencies, we must increase the sampling rate and take two or more observations per day. A similar example is provided by yearly sales figures. These will obviously give no information about any seasonal effects, whereas monthly or quarterly observations will give information about seasonality. At the other end of the spectrum, we will now explain why there is a lowest frequency below which it is not sensible to try to fit to a set of data. If we had just 6 months of temperature readings from winter to summer, the analyst would not be able to decide, from the data alone, whether there is an upward trend in the observations or whether winters are colder than summers. However, with 1 year’s data, it would become clear that winters are colder than summers. Thus if we are interested in variation at the low frequency of 1 cycle per year, then we should have at least 1 year’s data, in which case the lowest frequency we can fit is at 1 cycle per year. With weekly observations, for example, 1 year’s data have N=52, Δt=1 week, and the lowest angular frequency of 2π/NΔt corresponds to a frequency of 1/NΔt cycles per week. (Note that all time units must be expressed in terms of the same period, here a week.) The lowest frequency is therefore 1/52 cycles per week, which can now be converted to 1 cycle per year. The lowest frequency, namely, 2π/NΔ, is sometimes called the fundamental Fourier frequency, because for the Fourier series representation of the data is normally evaluated at the frequencies p=1,…, N/2, which are all integer multiples of the fundamental frequency. These integer multiples are often called harmonics. The phrase fundamental frequency is perhaps more typically, and more helpfully, used when a function, ƒ(t) say, is periodic with period T so that f(t+nT)=f(t) for all integer values of n. Then f=1/

T, or , is called the fundamental frequency and the Fourier series representation of ƒ(t) is a sum over integer multiples, or harmonics, of the fundamental frequency. When T= NΔ=(the length of the observed time series), the fundamental frequencies coincide. This raises a practical point, in regard to choosing the length of a time series. Suppose, for example, that you are collecting weekly data and are particularly interested in annual variation. As noted above, you should collect at least 1 year’s data. If you collect exactly 52 weeks of data3, then the fundamental frequency will be at exactly 1 cycle per year. We will see that this makes it much easier to interpret the results of a spectral analysis. The fundamental frequency is at 1 cycle per year and the harmonics are at 2 cycles per year, 3 cycles per year and so on. However, if you have say an extra 12 weeks of data making 64 weeks, then it will be much harder to interpret the results at frequencies . Wherever possible, you should choose the length of the time series so that the harmonics cover the frequencies of particular interest. The easiest option is to collect observations covering an integer multiple of the lowest wavelength of particular interest. This ensures 3 For simplicity, ignore day 365, and day 366 if a leap year.

< previous page

file:///C:/Documents and Settings/Yang/

page_125

next page >

面/The analysis of time series an introduction/files/page_125.html [5/24/2009 16:52:13]

page_126

< previous page

page_126

next page >

Page 126 that this frequency is an integer multiple of the fundamental frequency. Thus, to investigate annual variation, 2 years of data is good and 3 or 4 years of data even better. The reader will notice that the Nyquist frequency does not depend on N, but rather only on the sampling frequency, whereas the lowest frequency does depend on N. Put another way, the lower the frequency we are interested in, the longer the time period over which we need to take measurements, whereas the higher the frequency we are interested in, the more frequently must we take observations. 7.3 Periodogram Analysis Early attempts at discovering hidden periodicities in a given time series basically consisted of repeating the analysis of Section 7.2 at all the frequencies 2π/N, 4π/N,…, π. In view of Equations (7.3)–(7.5), the different terms are orthogonal and we end up with the finite Fourier series representation of the {xt}, namely

(7.9) for t=1, 2,…, N, where the coefficients {ap, bp} are of the same form as Equations (7.6) and (7.7), namely

(7.10) An analysis along these lines is sometimes called a Fourier analysis or a harmonic analysis. The Fourier series representation in Equation (7.9) has N parameters to describe N observations and so can be made to fit the data exactly (just as a polynomial of degree N−1 involving N parameters can be found that goes exactly through N observations in polynomial regression). This explains why there is no error term in Equation (7.9) in contrast to Equation (7.1). Also note that there is no term in sin πt in Equation (7.9) as sin πt is zero for all integer t. It is worth stressing that the Fourier series coefficients in Equation (7.10) at a given frequency are exactly the same as the least squares estimates for the coefficients of the model in Equation (7.1). The overall effect of the Fourier analysis of the data is to partition the variability of the series into components at frequencies 2π/N, 4π/N,…, π. The component at frequency harmonic. For

< previous page

file:///C:/Documents and Settings/Yang/

page_126

is called the pth

next page >

面/The analysis of time series an introduction/files/page_126.html [5/24/2009 16:52:14]

page_127

page_127

< previous page

next page >

Page 127 p≠N/2, it can be useful to write the pth harmonic in the equivalent form (7.11)

where (7.12)

is the amplitude of the pth harmonic, and

(7.13) is the phase of the pth harmonic. We have already noted in Section 7.2 that, for p≠N/2, the contribution of the pth harmonic to the total sum Using Equation (7.12), this is equal to . Extending this of squares is given by result using Equations (7.2)–(7.5) and (7.9), we have, after some algebra (Exercise 7.3), that

Dividing through by N we have

(7.14) which is known as Parseval’s theorem. The left-hand side of Equation (7.14) is effectively the variance4 of is the contribution of the pth harmonic to the variance, and Equation (7.14) the observations. Thus shows how the total variance is partitioned. If we plot against , we obtain a line spectrum. A different type of line spectrum occurs in the physical sciences when light from molecules in a gas discharge tube is viewed through a spectroscope. The light has energy at discrete frequencies and this energy can be seen as bright lines. However, most time series have continuous spectra, and then it is inappropriate to plot a line spectrum. If we regard

as the contribution to variance in the range

height in the range

, we can plot a histogram whose

is such that

Thus the height of the histogram at

, denoted by I(

), is given by (7.15)

As usual, Equation (7.15) does not apply for p=N/2; we may regard the range [(N−1)π/N, π] so that

as the contribution to variance in

4 The divisor is N rather than the more usual (N−1), but this makes little difference for large N.

< previous page

file:///C:/Documents and Settings/Yang/

page_127

next page >

面/The analysis of time series an introduction/files/page_127.html [5/24/2009 16:52:15]

page_128

< previous page

page_128

next page >

Page 128 against is usually called the periodogram, even though is a function of The plot of frequency rather than period. It follows from Parseval’s theorem in Equation (7.14) that the total area under the periodogram is equal to the variance of the time series. and hence for , can be written in several equivalent ways that look quite Note that the formula for different. For example, after some algebra, it can be shown that

(7.16) or we can replace with in Equation (7.16). The usual way to actually calculate the periodogram directly from the data uses the expression (7.17) Equation (7.17) also applies for p=N/2. Other authors define the periodogram in what appear to be slightly different ways, but the differences usually

, rather than

arise from allowing negative frequencies or using the cyclic frequency expressions generally turn out to be some other multiple of I(ωp) or

. The

. For example, Hannan (1970,

Equation (3.8)) and Koopmans (1995, Equation (8.7)) give expressions that correspond to (7.16). As to terminology, Anderson (1971, Section 4.3.2) describes the graph of

×expression

against the period N/p,

against frequency. as the periodogram, and suggests the term spectrogram to describe the graph of Jenkins and Watts (1968) define a similar expression to Equation (7.17) in terms of the variable

, but call it the ‘sample spectrum’. As always, when comparing terms and formulae from different sources, the reader needs to take great care. The periodogram appears to be a natural way of estimating the power spectral density function, but Section 7.3.2 shows that, for a process with a continuous spectrum, it provides a poor estimate and needs to be modified. First, we derive the relationship between the periodogram of a given time series and the corresponding autocovariance function (acv.f.). 7.3.1 The relationship between the periodogram and the acv.f. The periodogram ordinate and the autocovariance coefficient ck are both quadratic forms of the data {xt}. It is therefore natural to enquire how they are related. In fact, we will show that the periodogram is the finite Fourier transform of {ck}. Using Equation (7.2), we may rewrite Equation (7.17) for p≠N/2 as

< previous page

file:///C:/Documents and Settings/Yang/

page_128

next page >

面/The analysis of time series an introduction/files/page_128.html [5/24/2009 16:52:16]

page_129

< previous page

page_129

next page >

Page 129

However, (see Equation (4.1))

and so that

(7.18)

(7.19) The formula in Equation (7.19) is an expression called a discrete finite Fourier transform (assuming that ck=0 for |k| ≥N). Any reader not familiar with the Fourier transform, is recommended to read Appendix A— see especially Equation (A.5). 7.3.2 Properties of the periodogram When the periodogram is expressed in the form of Equation (7.18), it appears to be the ‘obvious’ estimate of the power spectrum

simply replacing γk by its estimate ck for values of k up to (N−1), and putting subsequent estimates of γk equal to zero. However, although we find (7.20) so that the periodogram is asymptotically unbiased, we see below that the variance of

does not

decrease as N increases. Thus is not a consistent estimator for . An example of a periodogram is given later in Figure 7.5(c), and it can be seen that the graph fluctuates wildly. The lack of consistency is perhaps not too surprising when one realizes that the Fourier series representation in Equation (7.9) requires one to evaluate N parameters from N observations, however long the series may be. Thus in Section 7.4 we will consider alternative ways of estimating a power spectrum that are essentially ways of smoothing the periodogram. is not a consistent estimator for in the case where the We complete this section by proving that observations are assumed to be independent N(µ, σ2) variates, so that they form a discrete-time purely random process with a uniform spectrum. This result can be extended to other stationary

< previous page

file:///C:/Documents and Settings/Yang/

page_129

next page >

面/The analysis of time series an introduction/files/page_129.html [5/24/2009 16:52:17]

page_130

< previous page

page_130

next page >

Page 130 processes with continuous spectra, but this does not need to be demonstrated here. If the periodogram estimator does not ‘work’ for a uniform spectrum, it cannot be expected to ‘work’ for more complicated spectra. Given the above assumptions, Equation (7.10) shows that ap and bp are linear combinations of normally distributed random variables and so will themselves be normally distributed. Using Equations (7.2)– (7.4), it can be shown (Exercise 7.4) that ap and bp each have mean zero and variance 2σ2/N for p≠N/2. Furthermore we have

since the observations are assumed to be independent. Thus, using Equation (7.5), we see that ap and bp are uncorrelated. Since (ap, bp) are bivariate normal, zero correlation implies that ap and bp are independent. The variables ap and bp can be standardized by dividing by to give standard N(0, 1) variables. Now a result from distribution theory says that if Y1, Y2 are independent N(0, 1) variables, then has a X2 distribution with two degrees of freedom, which is written

. Thus

is . Put another way, this means that is when as in this case, although this result does, in fact, generalize to spectra that are not constant. Now the variance of a X2 distribution with v degrees of freedom is 2v, so that and As this variance is a constant, it does not tend to zero as N→∞, and hence

is not a consistent

. Furthermore it can be shown that neighbouring periodogram ordinates are estimator for asymptotically independent, which further explains the very irregular form of an observed periodogram. This all means that the periodogram needs to be modified in order to obtain a good estimate of a continuous spectrum. 7.4 Some Consistent Estimation Procedures This section describes several alternative ways of estimating a spectrum. The different methods will be compared in Section 7.6. Each method provides a consistent estimator for the (power) spectral density function, in contrast to the (raw) periodogram. However, although the periodogram is itself an inconsistent estimator, the procedures described in this section are essentially based on smoothing the periodogram in some way. Throughout the section we will assume that any obvious trend and seasonal

< previous page

file:///C:/Documents and Settings/Yang/

page_130

next page >

面/The analysis of time series an introduction/files/page_130.html [5/24/2009 16:52:18]

page_131

< previous page

page_131

next page >

Page 131 variation have been removed from the data. If this is not done, the results of the spectral analysis are likely to be dominated by these effects, making any other effects difficult or impossible to see. Trend produces a peak at zero frequency, while seasonal variation produces peaks at the seasonal frequency and at integer multiples of the seasonal frequency—the seasonal harmonics (see Section 7.2.1). For a nonstationary series, the estimated spectrum of the detrended, deseasonalized data will depend to some extent on the method chosen to remove trend and seasonality. We assume throughout that a ‘good’ method is used to do this. The methods described in this chapter are essentially non-parametric in that no model fitting is involved. It is possible to use a model-based approach and an alternative, parametric approach, called autoregressive spectrum estimation, will be introduced later in Section 13.7.1. 7.4.1 Transforming the truncated acv.f. One type of estimation procedure consists of taking a Fourier transform of the truncated weighted sample acv.f. From Equation (7.18), we know that the periodogram is the discrete finite Fourier transform of the complete sample acv.f. However, it is clear that the precision of the values of ck decreases as k increases, because the coefficients are based on fewer and fewer terms. Thus, it would seem intuitively reasonable to give less weight to the values of ck as k increases. An estimator, which has this property is

(7.21) where {λk} are a set of weights called the lag window, and M(

面/The analysis of time series an introduction/files/page_132.html [5/24/2009 16:52:19]

page_133

page_133

< previous page

next page >

Page 133 , while results from density estimation suggest that a different power of N may be appropriate. Percival and Walden (1993, Chapter 6) point out that an appropriate value of M depends on the properties of the underlying process and give more detailed guidance. However, my advice is to try three or four different values of M. A low value will give an idea where the large peaks in f( ) are, but the curve is likely to be too smooth. A high value is likely to produce a curve showing a large number of peaks, some of which may be spurious. A compromise can then be achieved with an in-between value of M. In principle, Equation (7.21) may be evaluated at any value of in (0, π), but it is usually evaluated at equal for j=0, 1…, Q, where Q is chosen sufficiently large to show up all features of

intervals at

. Often Q is chosen to be equal to M. The graph of against can then be plotted and examined. An example is given later in Figure 7.5, for the data plotted in Figure 1.2, using the Tukey window with M=24. 7.4.2 Hanning This procedure, named after Julius Von Hann, is equivalent to the use of the Tukey window as described in Section 7.4.1, but adopts a different computational procedure. The estimated spectrum is calculated in two stages. First, a truncated unweighted cosine transform of the acv.f. of the data is taken to give

(7.22) This is the same as Equation (7.21) except that the lag window is taken to be unity (i.e. λk=1). The estimates given by Equation (7.22) are calculated at smoothed using the weights

for j=0, 1,…, M. These estimates are then

to give the Hanning estimates (7.23)

at

for j=1, 2 ,…, (M−1). At zero frequency, and at the Nyquist frequency π, we take

It can easily be shown algebraically that this procedure is equivalent to the use of the Tukey window. Substituting Equation (7.22) into (7.23) we find

< previous page

file:///C:/Documents and Settings/Yang/

page_133

next page >

面/The analysis of time series an introduction/files/page_133.html [5/24/2009 16:52:20]

page_134

< previous page

page_134

next page >

Page 134 and, using , a comparison with Equation (7.21) shows that the lag window is indeed the Tukey window. There is relatively little difference in the computational efficiency of Hanning and the straightforward use of the Tukey window. Both methods should yield the same estimates and so it does not matter which of the two procedures is used in practice. 7.4.3 Hamming This technique is very similar to Hanning and has a very similar title, which sometimes leads to confusion. In fact Hamming is named after a quite different person, namely R.W.Hamming. The technique is nearly identical to Hanning except that the weights in Equation (7.23) are changed to (0.23, 0.54, 0.23). At the frequencies =0 and =π, the weights are 0.54 (at the ‘end’ frequency) and 0.46. The procedure gives similar estimates to those produced by Hanning. 7.4.4 Smoothing the periodogram The methods of Sections 7.4.1–7.4.3 are based on transforming the truncated sample acv.f. An alternative type of approach is to smooth the periodogram ordinates in some way, the simplest approach being to group the periodogram ordinates in sets of size m and find their average value. The latter approach is based on a suggestion made by P.J.Daniell as long ago as 1946. However, the use of lag window estimators was standard for many years because less computation was involved. Nowadays, some form of smoothed periodogram is used much more widely, particularly with the advent of the fast Fourier transform—see Section 7.4.5. The basic idea of the simple smoothed periodogram can be expressed in the following formula:

(7.24) where

and j varies over m consecutive integers so that the

are symmetric about the

at the end-points =0 and =π, Equation frequency of interest, namely, . In order to estimate (7.24) has to be modified in an obvious way, treating the periodogram as being symmetric about 0 and π. Then, taking m to be odd with m*=(m−1)/2, we have

< previous page

file:///C:/Documents and Settings/Yang/

page_134

next page >

面/The analysis of time series an introduction/files/page_134.html [5/24/2009 16:52:21]

page_135

page_135

< previous page

next page >

Page 135 The expression for (0) can be simplified as the first term I(0) is zero. Now we know that the periodogram is asymptotically unbiased but inconsistent for the true spectrum. Since neighbouring periodogram ordinates are asymptotically uncorrelated, it is clear that the variance of Equation (7.24) will be of order 1/m. It is also clear that the estimator in Equation (7.24) may be biased since

which is only equal to

if the spectrum is linear over the relevant interval. However, the bias will be

is a reasonably smooth function and m is not too large compared with N. ‘small’ provided that The consequence of the above remarks is that the choice of group size m is rather like the choice of the truncation point M in Section 7.4.1 in that it has to be chosen so as to balance resolution against variance. However, the choice is different in that changes in m and in M act in opposite directions. An increase in m has a similar effect to a reduction in M. The larger the value of m the smaller will be the variance of the

, such as resulting estimate but the larger will be the bias. If m is too large, then interesting features of peaks, may be smoothed out. Of course, as N increases, we can in turn allow m to increase, just as we allowed M to increase with N in Section 7.4.1. There is relatively little advice in the literature on the choice of m. As in Section 7.4.1, it seems advisable to exist, but try several values for m. A ‘high’ value should give some idea as to whether large peaks in the curve is likely to be too smooth and some real peaks may be hidden. A ‘low’ value is likely to produce a much more uneven curve showing many peaks, some of which will be spurious. A compromise between the effects of bias and variance can then be made. In earlier editions, I suggested trying values near N/40, but I is probably a better guideline. now think that Although the procedure described in this section is computationally quite different to that of Section 7.4.1, there are in fact close theoretical links between the two procedures. In Section 7.3.1 we derived the relationship between the periodogram and the sample acv.f., and, if we substitute Equation (7.18) into (7.24), we can express the smoothed periodogram estimate of the spectrum in terms of the sample acv.f. in a similar form to Equation (7.21). After some algebra (Exercise 7.5), it can be shown that the truncation point is (N−1) and the lag window is given by

Thus, the formula uses values of ck right up to k=(N−1), rather than having a truncation point much lower than N. Moreover, the lag window has the undesirable property that it does not tend to zero as k tends to N. The smoothed periodogram effectively uses a rectangular window in the frequency domain and the resulting lag window shows that a sudden cut-off in the frequency domain can give rise to ‘nasty’ effects in the time domain

< previous page

file:///C:/Documents and Settings/Yang/

page_135

next page >

面/The analysis of time series an introduction/files/page_135.html [5/24/2009 16:52:22]

page_136

< previous page

page_136

next page >

Page 136 (and vice versa). The smoothed periodogram often works reasonably well, but its window properties suggest that it may be possible to find a way of smoothing the periodogram, using a non-uniform averaging procedure, that has better time-domain properties. In fact, the simple smoothed periodogram is rarely used today, but rather a windowed form of averaging is used instead. Various alternative smoothing procedures have been suggested, with the idea of giving more weight to the periodogram ordinate at the particular frequency of interest and progressively less weight to periodogram ordinates further away. The analyst can think of this as applying a window in the frequency domain rather than in the time domain, but in a way that corresponds to the use of a lag window as in Section 7.4.1. It is possible to use a triangular (Bartlett window) or aim for a bell-shaped curve, perhaps by applying a simple smoother, such as Hanning, more than once. These approaches will not be considered here and the reader is referred, for example, to Hayes (1996, Chapter 8) or Bloomfield (2000, Chapter 8). Historically, the smoothed periodogram was not much used until the 1990s because it apparently requires much more computational effort than transforming the truncated acv.f. Calculating the periodogram using Equation (7.17) at for p=1, 2,…, N/2 would require about N2 arithmetic operations (each one a multiplication and an addition), whereas using Equation (7.21) fewer than MN operations are required to calculate the {ck} so that the total number of operations is only of order M(N+M) if the spectrum is evaluated at M frequencies. Two factors have led to the increasing use of the smoothed periodogram. First, the advent of high-speed computers means that it is unnecessary to restrict attention to the method requiring fewest calculations. The second factor has been the widespread use of an algorithm called the fast Fourier transform, which makes it much quicker to compute the periodogram. This procedure will now be described. 7.4.5 The fast Fourier transform (FFT) The computational procedure described in this section is usually abbreviated to FFT5 and we adopt this abbreviation. For long series, the technique can substantially reduce the time required to perform a Fourier analysis of a set of data on a computer, and can also give more accurate results. The history of the FFT dates back to the early 1900s. However, it was the work of J.W.Cooley, J.W.Tukey and G.Sande in about 1965 coupled with the arrival of faster computers that stimulated the application of the technique to time-series analysis. Much of the early work was published in the various Transactions of the IEEE, but more recent coverage is given, for example, by Bendat and Piersol (2000), Bloomfield (2000) and Priestley (1981). We will only give a broad outline of the technique here. The FFT requires that the value of N should be composite, meaning that 5 Some authors have used this abbreviation to denote the finite Fourier transform.

< previous page

file:///C:/Documents and Settings/Yang/

page_136

next page >

面/The analysis of time series an introduction/files/page_136.html [5/24/2009 16:52:23]

page_137

< previous page

page_137

next page >

Page 137 N is not a prime number and so can be factorized. The basic idea of the FFT will be illustrated for the case when N can be factorized in the form N=rs, where r and s are integers. If we assume that N is even, then at least one of the factors, say r, will be even. Using complex numbers for mathematical simplicity, the Fourier coefficients from Equation (7.10) can be expressed in the form (7.25) for p=0, 1, 2 ,…, (N/2)−1. For mathematical convenience, we denote the observations by x0, x1,…, xN−1, so that the summation in Equation (7.25) is from t=0 to N−1. Now we can write t in the form where t1=0, 1,…, s−1, and t0=0, 1,…, r−1, as t goes from 0 to N−1, in view of the fact that N=rs. Similarly we can decompose p in the form where p1=0, 1,…, (r/2)−1, and P0=0, 1,…, s−1, as p goes from 0 to (N/2)−1. Then the summation in Equation (7.25) may be written

However,

since . Thus does not depend on p1 and is therefore a function of t0 and P0 only, say A(p0, t0). Then Equation (7.25) may be written

Now there are N=rs functions of type A(p0, t0) to be calculated, each requiring s complex multiplications and additions. There are N/2 values of (ap+ibp) to be calculated, each requiring r further complex multiplications and additions. This gives a grand total of calculations instead of the N×N/2=N2/2 calculations required to use Equation (7.25) directly. By a suitable choice of s and r, we can usually arrange for (s+r/2) to be (much) less than N/2. Much bigger reductions in computing can be made by an extension of the above procedure when N is highly composite (i.e. has many small factors). In particular, if N is of the form 2k, then we find that the number of operations is of order Nk (or N log2 N) instead of N2/2. Substantial gains can also be made when N has several factors (e.g. N=2p3q5r…). In practice it is unlikely that N will naturally be of a simple form such as 2k, unless the value of N can be chosen before measurement starts. However, there

< previous page

file:///C:/Documents and Settings/Yang/

page_137

next page >

面/The analysis of time series an introduction/files/page_137.html [5/24/2009 16:52:23]

page_138

page_138

< previous page

next page >

Page 138 are other things we can do. It may be possible to make N highly composite by the simple expedient of omitting a few observations from the beginning or end of the series. For example, with 270 observations, we can omit the last 14 to make N=256=28. More generally we can increase the length of the series by adding zeros to the (mean-corrected) sample record until the value of the revised N becomes a suitable integer. Then a procedure called tapering or data windowing (e.g. Percival and Walden, 1993; Priestley, 1981) is often recommended6 to avoid a discontinuity at the end of the data. Suppose, for example, that we happen to have 382 observations. This value of N is not highly composite and we might proceed as follows: • Remove any linear trend from the data, and keep the residuals (which should have mean zero) for subsequent analysis. If there is no trend, simply subtract the overall mean from each observation. • Apply a linear taper to about 5% of the data at each end. In this example, if we denote the detrended mean-corrected data by x0, x1,…, x381, then the tapered series is given by

• Add 512−382=130 zeros at one end of the tapered series, so that N=512=29. • Carry out an FFT on the data, calculate the Fourier coefficients ap+ibp and average the values of in groups of about 10. In fact with N as low as 382, the computational advantage of the FFT is limited and we could equally well calculate the periodogram directly, which avoids the need for tapering and adding zeros. The FFT really comes into its own when there are several thousand observations. It is also worth explaining that the FFT is still useful when the analyst prefers to look at the autocorrelation function (ac.f.) before carrying out a spectral analysis, either because inspecting the ac.f. is thought to be an invaluable preliminary exercise or because the analyst prefers to transform the truncated weighted acv.f. rather than smooth the periodogram. It can be quicker to calculate the sample acv.f. by performing two FFTs (e.g. Priestley, 1981, Section 7.6), rather than directly as a sum of lagged products. The procedure is as follows. Compute the Fourier coefficients (ap, bp) with an FFT of the mean-corrected data at for p=0, 1,…, N−1 rather than for p=0, 1,…, N/2 as we usually do. The extra coefficients are normally redundant for real-valued processes since aN−k=ak and bN−k=−bk. However, for calculating the autocovariances, we can compute

at these values of p and then fast Fourier retransform

to the sequence 6 Note that some researchers view tapering with suspicion as the data are modified—see, for example, the discussion in Percival and Walden (1993, p. 215).

< previous page

file:///C:/Documents and Settings/Yang/

page_138

next page >

面/The analysis of time series an introduction/files/page_138.html [5/24/2009 16:52:24]

page_139

page_139

< previous page

next page >

Page 139 get the mean lagged products. We will not give the algebra here. For several thousand observations, this can be much faster than calculating them directly. However, when using the FFT in this way, the analyst should take care to add enough zeros to the data (without tapering) to make sure that non-circular sums of lagged products are calculated, as defined by Equation (4.1) and used throughout this book. Circular coefficients result if zeros are not added where, for example, the circular autocovariance coefficient at lag 1 is

where xN+1 is taken to be equal to x1 to make the series ‘circular’. Note that, if x1= , then the circular and non-circular coefficients at lag 1 are the same. If we use mean-corrected data, which will have mean zero, then adding zeros will make circular and non-circular coefficients be the same. In order to calculate all the non-circular autocovariance coefficients of a set of N mean-corrected observations, the analyst should add N zeros, to make 2N ‘observations’ in all. 7.5 Confidence Intervals for the Spectrum The methods of Section 7.4 all produce point estimates of the spectral density function, and hence give no indication of their likely accuracy. This section shows how to find appropriate confidence intervals. In Section 7.3.2, we showed that data from a white noise process, with constant spectrum

, yields a periodogram ordinate distributed as

at frequency

, which is such that

is

. Note that this distibution does not depend on N, which explains why

. Wide confidence intervals would result if consistent estimator for Suppose instead that we use the estimator of Section 7.4.1, namely

Then, it can be shown (Jenkins and Watts, 1968, Section 6.4.2) that distributed as an approximate

so that a 100(1−α)% confidence interval for

file:///C:/Documents and Settings/Yang/

was used as an estimator.

is asymptotically

random variable, where

is called the number of degrees of freedom of the lag window. It follows that

< previous page

is not a

(7.26)

is given by

page_139

next page >

面/The analysis of time series an introduction/files/page_139.html [5/24/2009 16:52:25]

page_140

< previous page

page_140

next page >

Page 140 Some simple algebra shows that the degrees of freedom for the Tukey and Parzen windows turn out to be 2.67N/M and 3.71N/M, respectively. Although relying on asymptotic results, Neave (1972a) has shown that the above formulae are also quite accurate for short series. For the smoothed periodogram estimator of Section 7.4.4, there is no need to apply Equation (7.26), because smoothing the periodogram in groups of size m is effectively the same as averaging independent random variables. Thus, it is clear that the smoothed periodogram will have v=2m degrees of freedom, and we can then apply the same formula for the confidence interval as given above. 7.6 Comparison of Different Estimation Procedures Several factors need to be considered when comparing the different estimation procedures that were introduced in Section 7.4. Although we concentrate on the theoretical properties of the different procedures, the analyst will also need to consider practical questions such as computing time and the availability of suitable computer software. Alternative comparative discussions are given by Jenkins and Watts (1968), Neave (1972b), Priestley (1981, Section 7.5) and Bloomfield (2000). It is helpful to introduce a function called the spectral window or kernel, which is defined to be the Fourier transform of the lag window { k} introduced in Equation (7.21). Assuming that k is zero for k>M, and symmetric, so that then the spectral window is defined by

for (−π<

面/The analysis of time series an introduction/files/page_140.html [5/24/2009 16:52:26]

page_141

< previous page

page_141

next page >

Page 141 (7.29) using Equation (7.19). Equation (7.29) shows that all the estimation procedures are essentially smoothing the . The value of the lag window at lag zero is usually specified to periodogram using the weight function be one, so that from Equation (7.28) we have

which is a desirable property for a smoothing function. Taking expectations in Equation (7.29) we have asymptotically that (7.30) Thus the spectral window is a weight function expressing the contribution of the spectral density function at . The name ‘window’ arises from the fact that each frequency to the expectation of the part of the periodogram that is ‘seen’ by the estimator.

determines

Figure 7.2 The spectral windows for three common methods of spectral analysis: A, smoothed periodogram (m=20); B, Parzen (M=93); C, Tukey (M=67); all with N=1000.

< previous page

file:///C:/Documents and Settings/Yang/

page_141

next page >

面/The analysis of time series an introduction/files/page_141.html [5/24/2009 16:52:27]

page_142

< previous page

page_142

next page >

Page 142 Examples of the spectral windows for three common methods of spectral analysis are shown in Figure 7.2. Taking N=1000, the spectral window for the smoothed periodogram with m=20 is shown as line A. The other two windows are the Parzen and Tukey windows, denoted by lines B and C. The values of the truncation point M were chosen to be 93 for the Parzen window and 67 for the Tukey window. These values of M were chosen so that all three windows gave estimators with equal variance. Formulae for variances will be given later in this section. Inspecting Figure 7.2, we see that the Parzen and Tukey windows look very similar, although the Parzen window has the advantage of being non-negative and of having smaller side lobes. The shape of the periodogram window is quite different. It is approximately rectangular with a sharp cut-off and is close to the ‘ideal’ band-pass filter, which would be exactly rectangular but which is unattainable in practice. The periodogram window also has the advantage of being non-negative. In comparing different windows, we should consider both the bias and the variance of the estimator. This is sometimes called the variance-bias trade-off question, as well as balancing resolution against variance. By taking a wider window, we generally get a lower variance but a larger bias and some sort of compromise has to be made in practice. This is often achieved by using trial and error, as, for example, in the choice of the truncation point for a lag window as discussed earlier in Section 7.4.1. It is not easy to get general formulae for the bias produced by the different procedures. However, it is intuitively clear from Equation (7.30) and from earlier remarks that the wider the window, the larger will be the bias. In particular, it is clear that all the smoothing procedures will tend to lower peaks and raise troughs. is approximately distributed as , where As regards variance, we noted in Section 7.5 that v=2m, for the smoothed periodogram, and, using Equation (7.26), 3.71N/M and 8N/3M for the Parzen and Tukey windows, respectively. Since and

turns out to be 1/m, 2M/3.71N, and 3M/4N, respectively, for the three windows. we find Equating these expressions gives the values of M chosen for Figure 7.2. When comparing the different estimators, the notion of a bandwidth may be helpful. Roughly speaking, the bandwidth is the width of the spectral window, as might be expected. Various formal definitions are given in the literature, but we adopt the one given by Jenkins and Watts (1968), namely, the width of the ‘ideal’ rectangular window that would give an estimator with the same variance. The window of the smoothed periodogram is so close to being rectangular for m ‘large’ that it is clear from Figure 7.2 that the bandwidth will be approximately 2mπ/N (as area must be unity and height

< previous page

file:///C:/Documents and Settings/Yang/

page_142

next page >

面/The analysis of time series an introduction/files/page_142.html [5/24/2009 16:52:27]

page_143

< previous page

page_143

next page >

Page 143 is N/2mπ). The bandwidths for the Bartlett, Parzen and Tukey windows turn out to be 3/2M, 2π(1.86/M) and 8π/3M, respectively. When plotting a graph of an estimated spectrum, it is a good idea to indicate the bandwidth that has been used. The choice of bandwidth is equivalent to the choice of m or M, depending on the method used. This choice is an important step in spectral analysis, though it is important to remember that the effects of changing m and M act in opposite directions. For the Bartlett, Parzen and Tukey windows, the bandwidth is inversely proportional to M. Figure 7.3 shows how the window changes as M varies, using the Bartlett window as a representative example. As M gets larger, the window gets narrower, the bias gets smaller but the variance of the resulting estimator gets larger. For the smoothed periodogram, the reverse happens. The bandwidth is directly proportional to m, and as m gets larger, the window gets wider, the bias increases but the variance reduces. For the unsmoothed periodogram, with m=1, the window is very tall and narrow giving an estimator with large variance as we have already shown. All in all, the choice of bandwidth is rather like the choice of class interval when constructing a histogram.

Figure 7.3 The Bartlett spectral window for different values of M. We are now in a position to give guidance on the relative merits of the different estimation procedures. As regards theoretical properties, it is arguable that the smoothed periodogram has the better-shaped spectral window in that it is approximately rectangular, although there are some side lobes. For the transformed acv. f., the Parzen and Tukey windows are preferred to the Bartlett window. Computationally, the smoothed periodogram can be much slower for large N unless the FFT is used. However, if the FFT is used, then the smoothed periodogram can be faster. Moreover it is

< previous page

file:///C:/Documents and Settings/Yang/

page_143

next page >

面/The analysis of time series an introduction/files/page_143.html [5/24/2009 16:52:28]

page_144

< previous page

page_144

next page >

Page 144 possible to calculate the ac.f. quickly using two FFTs. One drawback to the use of the FFT is that it may require data-tapering, whose use is still somewhat controversial. Of course, for small N, computing time is a relatively unimportant consideration. As regards computer software, it is much easier to write a program for the Parzen or Tukey windows, but programs and algorithms for the FFT are becoming readily available. Thus the use of the smoothed periodogram has become more general, either with equal weights as discussed in Section 7.4.4, or with a more ‘bell-shaped’ set of weights. All the above assumes that a non-parametric approach is used in that no model fitting is attempted prior to carrying out a spectral analysis. As noted earlier, an alternative approach gaining ground is to use a parametric approach, fitting an autoregressive (AR) or ARMA model to the data. The spectrum of the fitted model is then used to estimate the spectrum. This approach will be described later in Section 13.7.1. 7.7 Analysing a Continuous Time Series Up to now, we have been concerned with the spectral analysis of time series recorded at discrete time intervals. However, time series are sometimes recorded as a continuous trace. For example, variables such as air temperature, humidity and the moisture content of tobacco emerging from a processing plant are often recorded by machines that give continuous-time readings. For series that contain components at very high frequencies, such as those arising in acoustics and speech processing, it may be possible to analyse such records mechanically using tuned filters, but the more usual procedure is to digitize the series by reading off the values of the trace at discrete intervals. If values are taken at equal time intervals of length Δt, we have converted a continuous time series into a standard discrete-time time series and can use the methods already described. In sampling a continuous time series, the main question is how to choose the sampling interval Δt. It is clear that sampling leads to some loss of information and that this loss gets worse as Δt increases. However, sampling costs increase as Δt gets smaller and so a compromise value must be sought. For the sampled series, the Nyquist frequency is π/Δt radians per unit time, and we can get no information about variation at higher frequencies. Thus we clearly want to choose Δt so that variation in the continuous series is negligible at frequencies higher than π/Δt. In fact most measuring instruments are bandlimited in that they do not respond to frequencies higher than a certain maximum frequency. If this maximum , is known or can be guessed, then the choice of Δt straightforward in that it should be frequency, say . However, if Δt is chosen to be too large, then a phenomenon called aliasing may occur. less than This can be illustrated by the following theorem.

< previous page

file:///C:/Documents and Settings/Yang/

page_144

next page >

面/The analysis of time series an introduction/files/page_144.html [5/24/2009 16:52:29]

page_145

page_145

< previous page

next page >

Page 145

Theorem 7.1 Suppose that a continuous time series, with spectrum for 0<

面/The analysis of time series an introduction/files/page_145.html [5/24/2009 16:52:30]

page_146

page_146

< previous page

next page >

Page 146 The implications of this theorem may now be considered. First, as noted earlier, if the continuous series contains no variation at frequencies above the Nyquist frequency, so that . In this case no information is lost by sampling. However, the more general result is then that sampling will have an effect in that variation at frequencies above the Nyquist frequency in the continuous series will be ‘folded back’ to produce apparent variation in the sampled series at a frequency , then the frequencies lower than the Nyquist frequency. If we denote the Nyquist frequency π/Δt by are called aliases of one another. Variation at all these frequencies in the continuous series will appear as variation at frequency in the sampled series. From a practical point of view, aliasing will cause trouble unless Δt is chosen to be sufficiently small so that . If we have no advance knowledge about guesstimate a value for Δt. If the resulting estimate of

, then we have to

approaches zero near the Nyquist frequency π/

does not approach zero Δt, then our choice of Δt is almost certainly sufficiently small. However, if near the Nyquist frequency, then it is probably wise to try a smaller value of Δt. Alternatively, if the analyst is only interested in the low-frequency components, then it may be easier to filter the continuous series so as to remove the high-frequency components and remove the need for selecting a small value of Δt. 7.8 Examples and Discussion Spectral analysis can be a useful exploratory diagnostic tool in the analysis of many types of time series. With the aid of examples, this section discusses how to interpret an estimated spectrum, and tries to indicate when spectral analysis is likely to be most useful and when it is likely to be unhelpful. We also discuss some of the practical problems arising in spectral analysis. We begin with an example to give the reader some ‘feel’ for the sorts of spectrum shapes that may arise. Figure 7.4 shows four sections of trace, labelled A, B, C and D, which were produced by four different processes (generated in a control engineering laboratory). Figure 7.4 also shows the corresponding spectra calculated from longer series than the short sections shown here. The spectra are labelled J, K, L and M, but are given in random order. Note that the four traces use the same scale, the length produced in one second being shown on trace D. The four spectra are also plotted using the same scales (linear in both directions). The peak in spectrum L is at 15 cycles per second (or 15 Hz). Before reading on, the reader is invited to guess which series goes with which spectrum. This is not easy, especially for the novice time-series analyst, but even experienced analysts may have difficulty. Try to assess which series are smoother (and hence have less high frequency variation) and which oscillate quicker. Spotting a deterministic sinusoidal perturbation in the presence of noise is more difficult than you might expect.

< previous page

file:///C:/Documents and Settings/Yang/

page_146

next page >

面/The analysis of time series an introduction/files/page_146.html [5/24/2009 16:52:31]

page_147

< previous page

page_147

next page >

Page 147

Figure 7.4 Four time series and their spectra. The spectra are given in random order.

< previous page

file:///C:/Documents and Settings/Yang/

page_147

next page >

面/The analysis of time series an introduction/files/page_147.html [5/24/2009 16:52:31]

page_148

< previous page

page_148

next page >

Page 148 The easiest series to assess is trace A, which is much smoother than the other three traces. This means that its spectrum is concentrated at low frequency. Of the four spectra, spectrum M cuts off at the lowest frequency and is largest at zero frequency. This is the spectrum for trace A. The other three spectra are harder to distinguish. Trace B is somewhat smoother than C or D and corresponds to spectrum K, which ‘cuts off’ at a lower frequency than J or L. Trace C corresponds to spectrum J, while trace D contains a deterministic sinusoidal component at 15 cycles per second, which contributes 20% of the total power. Thus D corresponds to spectrum L. From a visual inspection of traces C and D, it is difficult or impossible to decide which goes with spectrum J and which with spectrum L. For this type of data, spectral analysis is invaluable in assessing the frequency properties. The reader may find it surprising that the deterministic component in trace D is so hard to see, but remember that 80% of the power is some sort of noise spread over a wide frequency range and this makes the deterministic component, constituting only 20% of the power, hard to see. The above example contrasts with the air temperature series at Recife, plotted in Figure 1.2. There the regular seasonal variation is quite obvious from a visual inspection of the time plot, but in this case the deterministic component accounts for about 85% of the total variation. If we nevertheless carry out a spectral analysis of the air temperature series, we get the spectrum shown in Figure 7.5(a) with a large peak at a frequency of one cycle per year. However, it is arguable that the spectral analysis is not really necessary here, as the seasonal effect is so obvious anyway. In fact, if the analyst has a series containing an obvious trend or seasonal component, then it is advisable to remove such variation from the data before carrying out a spectral analysis, as any other effects will be relatively small and may not be visible in the spectrum of the raw data. Figure 7.5(b) shows the spectrum of the Recife air temperature data when the seasonal variation has been removed. The variance is concentrated at low frequencies, indicating either a trend, which is not apparent in Figure 1.2, or short-term correlation as in a first-order AR process with a positive coefficient (cf. Figure 6.4 (a)). The latter seems the more likely explanation here, given that there is no contextual reason to expect a trend in temperature (other than global warming, which is relatively small compared with other effects). As noted earlier, the corresponding periodogram in Figure 7.5(c) shows a graph that oscillates up and down very quickly and is not helpful for interpreting the properties of the data. This demonstrates again that the periodogram has to be smoothed to get a consistent estimate of the underlying spectrum.

< previous page

file:///C:/Documents and Settings/Yang/

page_148

next page >

面/The analysis of time series an introduction/files/page_148.html [5/24/2009 16:52:32]

page_149

< previous page

page_149

next page >

Page 149

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_149.html (1 of 2) [5/24/2009 16:52:33]

page_149

Figure 7.5 Spectra for average monthly air temperature readings at Recife, (a) for the raw data; (b) for the seasonally adjusted data using the Tukey window with M=24; (c) the periodogram of the seasonally adjusted data is shown for comparison.

< previous page

file:///C:/Documents and Settings/Yang/

page_149

next page >

面/The analysis of time series an introduction/files/page_149.html (2 of 2) [5/24/2009 16:52:33]

page_150

< previous page

page_150

next page >

Page 150 Removing trend and seasonality is a simple form of the general procedure usually called prewhitening. As the name suggests, this aims to construct a series having properties which are closer to those of white noise. In spectral analysis, this is useful because it is easier to estimate the spectrum of a series having a relatively flat spectrum, than one with sharp peaks and troughs. Prewhitening is often carried out by making a linear transformation of the raw data. Then the spectrum of the transformed series can be found, after which the spectrum of the original series can be found, if desired, by using the properties of the linear transformation8 used to carry out the prewhitening. In spectral analysis, this procedure is often limited to removing trend and seasonality, though in other applications more sophisticated model fitting is often used9. Having estimated the spectrum of a given time series, how do we interpret the results? There are various features to look for. First, are there any peaks in the spectrum and, if so, at what frequency? Can we find a contextual reason for a peak at this frequency? Second, what is the general shape of the spectrum? In particular, does the spectrum get larger as the frequency tends to zero? This often happens with economic variables and indicates a business cycle with a very long period or an underlying long-term non-stationarity in the mean that has not been removed by prior filtering. Economists, who expect to find a clear peak at low frequency, will usually be disappointed, especially if looking for business cycles with a period around 5–7 years. There is usually little evidence of anything so clear-cut. Rather the low-frequency variation is typically spread over a range of frequencies. The general shape of the spectrum could in principle be helpful in indicating an appropriate parametric model. For example, the shape of the spectrum of various ARMA models could be found and listed in a similar way to that used for specifying ac.f.s for different ARMA models. The use of the correlogram is a standard diagnostic tool in the Box-Jenkins procedure for identifying an appropriate ARIMA process, but the observed spectrum has rarely been used in this way. Why is this? Spectral analysis, as described in this chapter, is essentially a non-parametric procedure in which a finite set of observations is used to estimate a function defined over the whole range from (0, π). The function is not constrained to any particular functional form and so one is effectively trying to estimate more items than in a correlogram analysis, where the analyst may only look at values for a few low lags. Being non-parametric, spectral analysis is in one sense more general than inference based on a particular parametric class of models, but the downside is that it is likely to be less accurate if a parametric model really is appropriate. In my experience, spectral analysis is typically used when there is a suspicion that 8 The frequency response function of the linear transformation is defined later in Chapter 9 and leads directly to the required spectrum. 9 For example, with two time series it is advisable to prewhiten the series by removing as much autocorrelation as possible before calculating quantities called cross-correlations—see Chapter 8.

< previous page

file:///C:/Documents and Settings/Yang/

page_150

next page >

面/The analysis of time series an introduction/files/page_150.html [5/24/2009 16:52:33]

page_151

< previous page

page_151

next page >

Page 151 cyclic variation may be present at some unknown frequency, and the spectrum shape is rarely used for diagnosing a parametric model. Spectral analysis is arguably at its most useful for series of the type shown in Figure 7.4, where there is no obvious trend or ‘seasonal’ variation. Such series arise mostly in the physical sciences. In economics, spectral techniques have perhaps not proved as useful as was first hoped, although there have been a few successes. Attempts have also been made to apply spectral analysis to marketing data, but it can be argued (Chatfield, 1974) that marketing series are usually too short and the seasonal variation too large for spectral analysis to give useful results. In meteorology and oceanography, spectral analysis can be very useful (e.g. Craddock, 1965; Snodgrass et al., 1966) but, even in these sciences, spectral analysis may produce no worthwhile results, other than those that are obvious anyway. It is often the case that, once obvious cyclic effects have been removed (e.g. annual variation from monthly rainfall data; daily variation from hourly temperature data), the spectrum will show no clear peaks, but rather a tendency to get larger as the frequency tends to zero. The spectrum in Figure 7.5(b) is a case in point. The two examples in Percival and Walden (1993, Chapter 6), featuring ocean wave data and ice profile data, yield similar results. Sometimes a small peak is observed but tests usually show that this has dubious significance. We conclude this section by commenting on some practical aspects of spectral analysis. Most aspects, such as the choice of truncation point, have already been discussed and will be further clarified in Example 7.1 below. One problem that has not been discussed, is whether to plot the estimated spectrum on a linear or logarithm scale. An advantage of using a logarithmic scale is that the asymptotic variance of the estimated spectrum is then independent of the level of the spectrum, and so confidence intervals for the spectrum are of constant width on a logarithmic scale. For spectra showing large variations in power, a logarithmic scale also makes it possible to show more detail over a wide range. A similar idea is used by engineers when measuring sound in decibels, as the latter take values on a logarithmic scale. Jenkins and Watts (1968, p. 266) suggest that spectrum estimates should always be plotted on a logarithmic scale. However, Anderson (1971, p. 547) points out that this exaggerates the visual effects of variations where the spectrum is small. It may be easier to interpret a spectrum plotted on an arithmetic scale, as the area under the graph corresponds to power and this makes it easier to assess the relative importance of different peaks. Thus, while it is often useful to plot on a logarithmic scale in the initial stages of a spectral analysis, especially when trying different truncation points and testing the significance of peaks, it is often better to plot the final version of the estimated spectrum on a linear scale in order to get a clearer interpretation of the final result. It is also generally easier to interpret a spectrum if the frequency scale is measured in cycles per unit time (f) rather than radians per unit time . This has been done in Figures 7.4 and 7.5. A linear transformation of frequency

< previous page

file:///C:/Documents and Settings/Yang/

page_151

next page >

面/The analysis of time series an introduction/files/page_151.html [5/24/2009 16:52:34]

page_152

< previous page

page_152

next page >

Page 152 does not affect the relative heights of the spectrum at different frequencies, though it does change the absolute heights by a constant multiple. Another point worth mentioning is the possible presence in estimated spectra of harmonics. As noted earlier, when a series has a strong cyclic component at some frequency , then the estimated spectrum may additionally show related peaks at , ,…. These multiples of the fundamental frequency are called harmonics and generally speaking simply indicate that the main cyclical component is not exactly sinusoidal in character. Finally, a question that is often asked is how large a value of N is required to get a reasonable estimate of the spectrum. It is often recommended that between 100 and 200 observations is the minimum. With smaller values of N, only very large peaks can be found. However, if the data are prewhitened to make the spectrum fairly flat, then reasonable estimates may be obtained even with values of N around 100, as we have shown in Figure 7.5(b). However, much longer series are to be preferred and are the norm when spectral analysis is contemplated. Example 7.1 As an example, we analyse part of trace D of Figure 7.4. Although a fairly long trace was available, I decided just to analyse a section lasting for 1 second to illustrate the problems of analysing a fairly short series. This set of data will also illustrate the problems of analysing a continuous trace as opposed to a discrete time series. The first problem was to digitize the data, and this required the choice of a suitable sampling interval. Inspection of the original trace showed that variation seemed to be ‘fairly smooth’ over a length of 1 mm, corresponding to 1/100 second, but to ensure that there was no aliasing a sampling interval of 1/200 second was chosen giving N=200 observations. For such a short series, there is little to be gained by using the FFT. I therefore decided to transform the truncated acv.f. using Equation (7.21), with the Tukey window. Several truncation points were tried, and the results for M=20, 40 and 80 are shown in Figure 7.6. Equation (7.21) was evaluated at 51 points at , for j=0, 1,…, 50, where is measured in radians per unit time. Now in this example ‘unit time’ is 1/200 second and so the values of in radians per second are =200πj/50, for j=0, 1,…, 50. If we now convert the frequencies into cycles per second using , we find that the spectrum is evaluated at f=2j, for j=0, 1,…, 50. The Nyquist frequency is given by ƒN=100 cycles per second, which completes one cycle every two observations.

< previous page

file:///C:/Documents and Settings/Yang/

page_152

next page >

面/The analysis of time series an introduction/files/page_152.html [5/24/2009 16:52:35]

page_153

< previous page

page_153

next page >

Page 153

Figure 7.6 Estimated spectra for graph D of Figure 7.4 using the Tukey window with (a) M=80; (b) M=40; (c) M=20. Looking at Figure 7.6, it can be seen that above about 70 cycles per second, the estimates produced by the three values of M are all very small and cannot be distinguished on the graph. As the estimated spectrum approaches zero as the frequency approaches the Nyquist frequency, it seems clear that no information has been lost by aliasing so that our choice of sampling interval is sufficiently small. Indeed we could have made the sampling interval somewhat larger without losing much information. For lower frequencies, the estimated spectrum is judged rather too smooth with M=20, and much too erratic when M=80. The value M=40 looks about right, although M=30 might be even better. The subjective nature of this choice is clear. Using M=40 or 80, there is a clear peak in the spectrum at about 15 cycles per second (15 Hz). This matches the peak in spectrum L of Figure 7.4. However, Figure 7.6 also reveals a smaller unexpected peak at around 30 cycles per second. This looks like a harmonic of the deterministic sinusoidal component at 15 cycles per second, and may reduce in size if a longer series of observations were to be analysed. We also estimated the spectrum using a Parzen window with a truncation point of M=56. This value was chosen so that the degrees of freedom of the window, namely, 13.3, were almost the same as for the Tukey window with M=40. The results were so close to those produced by the Tukey window that there was no point in plotting them. The largest difference in the spectrum estimates was 0.33 at 12 cycles per second, but most of the estimates differed only in the second decimal place. Thus the Tukey and Parzen windows give much the same estimates when equivalent values of M are used.

< previous page

file:///C:/Documents and Settings/Yang/

page_153

next page >

面/The analysis of time series an introduction/files/page_153.html [5/24/2009 16:52:35]

page_154

< previous page

page_154

next page >

Page 154 The reader should note that the bandwidths for different values of M are indicated in Figure 7.6. The bandwidth for the Tukey window is 8π/3M in radians per unit time. As ‘unit time’ is 1/200 second, the bandwidth is 1600π/3M in radians per second or 800/3M in cycles per second. Confidence intervals can be calculated as described in Section 7.5. For a sample of only 200 observations, they are disturbingly wide. For example, when M=40, the degrees of freedom are 2.67N/M=13.3. For convenience this is rounded off to the nearest integer, namely, v=13. The peak in the estimated spectrum is =7.5. Here the 95% confidence interval is (3.9 to 19.5). Clearly a at 14 cycles per second, where longer series is desirable to make the confidence intervals acceptably narrow. Exercises 7.1 Revision of Fourier series. Show that the Fourier series, which represents the function is given by

7.2 Derive Equations (7.6) and (7.8). 7.3 Derive Parseval’s theorem, given by Equation (7.14). 7.4 If X1,…, XN are independent N(µ, σ2) variates show that is N(0, 2σ2/N) for p=1, 2,…, (N/2)−1. 7.5 Derive the lag window for smoothing the periodogram in sets of size m. For algebraic simplicity take m odd, with , so that

(Hint: The answer is given in Section 7.4.4. The algebra is rather messy. Use Equation (7.18) and the following two trigonometric results:

< previous page

file:///C:/Documents and Settings/Yang/

page_154

next page >

面/The analysis of time series an introduction/files/page_154.html [5/24/2009 16:52:36]

page_155

< previous page

page_155

next page >

Page 155 CHAPTER 8 Bivariate processes Thus far, we have been concerned with analysing a single time series. We now turn our attention to the situation where we have observations on two time series and we are interested in the relationship between them. We may distinguish two types of situations. First, we may have two series that arise ‘on an equal footing’. For example, it is often of interest to analyse seismic signals received at two recording sites. Here, we are not usually interested in trying to predict one variable from the other, but rather are primarily interested in measuring the correlations between the two series. In the second type of situation, the two series are thought to be ‘causally related’, in that one series is regarded as being the input to some sort of processor or system, while the second series is regarded as the output; we are then interested in finding the properties of the system that converts the input into the output. The two types of situations are roughly speaking the time-series analogues of correlation and regression. The first type of situation is considered in this chapter, where the cross-correlation function and the crossspectrum are introduced, and again in Chapter 12, where vector ARMA models are introduced. The second type of situation is discussed in Chapter 9 where it is assumed that the system can be described as a linear system. 8.1 Cross-Covariance and Cross-Correlation Suppose we make N observations on two variables at unit time intervals over the same period and denote the observations by (x1, y1),…, (xN, yN). We assume that these observations may be regarded as a finite realization of a discrete-time bivariate stochastic process (Xt, Yt). In order to describe a bivariate process it is useful to know the first- and second-order moments. For a univariate process, the first-order moment is the mean while the second-order moment is the autocovariance function (acv.f.), which includes the variance as a special case at lag zero. For a bivariate process, the moments up to second order consist of the mean and acv.f.s for each of the two components plus a new function, called the cross-covariance function. We will only consider bivariate processes that are second-order stationary, meaning that all moments up to second order do not change with time (as in the univariate case). We use the following notation:

< previous page

file:///C:/Documents and Settings/Yang/

page_155

next page >

面/The analysis of time series an introduction/files/page_155.html [5/24/2009 16:52:37]

page_156

page_156

< previous page

next page >

Page 156

Means. Autocovariances. Then the cross-covariance function is defined by (8.1)

and is a function of the lag only, because the processes are assumed to be stationary. Note that some authors define the cross-covariance function in the ‘opposite direction’ by Comparing with Equation (8.1) we see that It doesn’t matter which definition is used as long as it is stated clearly and used consistently. The cross-covariance function differs from the acv.f. in that it is not an even function, since in general Instead we have the relationship

where the subscripts are reversed. The size of the cross-covariance coefficients depends on the units in which Xt and Yt are measured. Thus for interpretative purposes, it is useful to standardize the cross-covariance function to produce a function called the cross-correlation function, ρXY(k), which is defined by (8.2) denotes the standard deviation of the X-process, and similarly for σY. This where function measures the correlation between Xt and Yt+k and has these two properties: (1)

(Note the subscripts reverse in order.)

(See Exercise 8.2.) (2) Whereas ρX(0), ρY(0) are both equal to one, the value of ρXY(0) is usually not equal to one, a fact that is sometimes overlooked. 8.1.1 Examples Before discussing the estimation of cross-covariance and cross-correlation functions, we will derive the theoretical functions for two examples of bivariate processes. The first example is rather ‘artificial’, but the model in Example 8.2 can be useful in practice.

< previous page

file:///C:/Documents and Settings/Yang/

page_156

next page >

面/The analysis of time series an introduction/files/page_156.html [5/24/2009 16:52:37]

page_157

< previous page

page_157

next page >

Page 157 Example 8.1 Suppose that {Xt}, {Yt} are both formed from the same purely random process {Zt}, which has mean zero, variance

, by

Then, using (8.1), we have

Now the variances of the two components are given by

so that, using (8.2), we have

Example 8.2 Suppose that (8.3) where {Z1,t}, {Z2,t} are uncorrelated purely random processes with mean zero and variance d is a positive integer. Then we find

, and where

. since σX=σZ and In Chapter 9 we will see that Equation (8.3) corresponds to putting noise into a linear system, which consists of a simple delay of lag d and then adding more noise. The cross-correlation function has a peak at lag d corresponding to the delay in the system, a result that the reader should find intuitively reasonable. 8.1.2 Estimation The ‘obvious’ way of estimating the cross-covariance and cross-correlation functions is by means of the corresponding sample functions. Suppose we have N pairs of observations {(xi, yi); i=1 to N}, on two series labelled x

< previous page

file:///C:/Documents and Settings/Yang/

page_157

next page >

面/The analysis of time series an introduction/files/page_157.html [5/24/2009 16:52:38]

page_158

page_158

< previous page

next page >

Page 158 and y. Then the sample cross-covariance function is

(8.4)

and the sample cross-correlation function is

(8.5) where SX, SY are the sample standard deviations of observations on xt and yt, respectively. It can be shown that these estimators are asymptotically unbiased and consistent. However, it can also be shown that estimators at neighbouring lags are themselves autocorrelated. Furthermore, it can be shown that the variances of sample cross-correlations depend on the autocorrelation functions of the two components. In general, the variances will be inflated. Thus, even for moderately large values of N up to about 200, or even higher, it is possible for two series, which are actually unrelated, to give rise to apparently ‘large’ cross-correlation coefficients, which are spurious, in that they arise solely from autocorrelations within the two series. Thus, if a test is required for non-zero correlation between two time series, then (at least) one of the series should first be filtered to convert it to (approximate) white noise. The same filter should then be applied to the second series before computing the cross-correlation function—see also Section 9.4.2. For example, suppose that one series appears to be a first-order autoregressive process with sample mean and estimated parameter . Then the filtered series is given by In other words, the new filtered series consists of the residuals from the fitted model. Applying the same filter to the second series, we get If the two series are actually uncorrelated, the above procedure will not affect the expected value of the sample cross-correlation coefficients, but should reduce their variance. This makes it much easier to interpret the cross-correlations, or indeed to use them to fit a linear model as in Section 9.4.2. Note that some authors (e.g. Brockwell and Davis, 1991, Chapter 11) recommend prewhitening both series, sometimes called double prewhitening, before calculating cross-correlations. For two uncorrelated series, of which one is white noise, it can be shown that

< previous page

file:///C:/Documents and Settings/Yang/

page_158

next page >

面/The analysis of time series an introduction/files/page_158.html [5/24/2009 16:52:39]

page_159

< previous page

page_159

next page >

Page 159 are significantly different from zero, and this result can be used Thus values outside the interval to test for zero cross-correlation. 8.1.3 Interpretation I have sometimes found it rather difficult to interpret a sample cross-correlation function. If the series are properly prewhitened, we have seen that it is easy to test whether any of the cross-correlation coefficients are significantly different from zero. Example 8.2 suggests that a significant peak in the estimated crosscorrelation function at lag d may indicate that one series is related to the other when delayed by time d. However, you are more likely to find a series of significant coefficients at neighbouring lags, and they are more difficult to interpret. The interpretation is even more difficult, and indeed fraught with danger, if the prefiltering procedure, described in Section 8.1.2, is not used. For example, Coen et al. (1969) calculated cross-correlation functions between variables such as the (detrended) Financial Times (FT) share index and (detrended) U.K. car production, and this resulted in a fairly smooth, roughly sinusoidal function with ‘large’ coefficients at lags 5 and 6 months. Coen et al. used this information to set up a regression model to ‘explain’ the variation in the FT share index in terms of car production 6 months earlier. However, Box and Newbold (1971) have shown that the ‘large’ cross-correlation coefficients are spurious as the two series had not been properly filtered. Rather than having models with independent error terms, the appropriate models for the given series were close to being random walks. This meant there were very high autocorrelations within each series and this inflated the variances of the cross-correlations. When the series were properly modelled, it was found that cross-correlations were negligible. Box and Newbold (1971) also presented some interesting simulations. They constructed two independent random walks and then computed the cross-correlations. The latter should have an expectation close to zero, but the high autocorrelations inflated the variance so that several spuriously high coefficients were observed. The general sinusoidal shape was similar to that found for the real data. The best general advice is ‘Beware’! 8.2 The Cross-Spectrum The cross-correlation function is the natural tool for examining the relationship between two time series in the time domain. This section introduces a complementary function, called the cross-spectral density function or cross-spectrum, which is the natural tool in the frequency domain. By analogy with Equation (6.11), we will define the cross-spectrum of a discrete-time bivariate stationary process, measured at unit intervals of time,

< previous page

file:///C:/Documents and Settings/Yang/

page_159

next page >

面/The analysis of time series an introduction/files/page_159.html [5/24/2009 16:52:40]

page_160

< previous page

page_160

next page >

Page 160 as the Fourier transform of the cross-covariance function, namely

(8.6) over the range 0<

面/The analysis of time series an introduction/files/page_160.html [5/24/2009 16:52:40]

page_161

page_161

< previous page

next page >

Page 161 so that An alternative way of expressing the cross-spectrum is in the form

(8.11) (8.12)

where

(8.13)

is the cross-amplitude spectrum, and

(8.14) is the phase spectrum. From Equation (8.14), it appears that XY( ) is undetermined by a multiple of π. However, if the cross-amplitude spectrum is required to be positive so that we take the positive square root in Equation (8.13), then the phase is actually undetermined by a multiple of 2π using the equality of Equations (8.11) and (8.12). This apparent non-uniqueness makes it difficult to evaluate the phase. However, when we consider linear systems in Chapter 9, we will see that there are physical reasons why the phase is often uniquely determined and does not need to be confined to the range ±π. The phase is usually zero at =0 and it makes sense to treat it as a continuous function of , as goes from 0 to π. Thus, if the phase grows to +π say, then it can be allowed to continue to grow rather than revert to −π. Another useful function derived from the cross-spectrum is the (squared) coherency, which is given by

where fX(

(8.15) ), fY( ) are the power spectra of the individual processes, {Xt} and {Yt}. It can be shown that

This quantity measures the square of the linear correlation between the two components of the bivariate process at frequency and is analogous to the square of the usual correlation coefficient. The closer C( ) is to one, the more closely related are the two processes at frequency . Finally, we will define a function called the gain spectrum, which is given by

which is essentially the regression coefficient of the process Yt on the process Xt at frequency

(8.16) . A second

where we divide by ƒY rather than ƒX. gain function can also be defined by In the terminology of linear systems—see Chapter 9—this definition corresponds to regarding Yt as the input and Xt as the output.

< previous page

file:///C:/Documents and Settings/Yang/

page_161

next page >

面/The analysis of time series an introduction/files/page_161.html [5/24/2009 16:52:41]

page_162

page_162

< previous page

next page >

Page 162 By this point, the reader will probably be rather confused by all the different functions that have been introduced in relation to the cross-spectrum. Whereas the cross-correlation function is a relatively straightforward development from the autocorrelation function (in theory at least, if not always in practice), statisticians often find the cross-spectrum much harder to understand than the autospectrum. It is no longer possible to interpret the results as some sort of contribution to power (variance) at different frequencies. Usually three functions have to be plotted against frequency to describe the relationship between two series in the frequency domain. Sometimes the co-, quadrature and coherency spectra are most suitable. Sometimes the coherency, phase and cross-amplitude are more appropriate, while another possible trio is coherency, phase and gain. Each trio can be determined from any other trio. The physical interpretation of these functions will probably not become clear until we have studied linear systems in Chapter 9. 8.2.1 Examples This subsection derives the cross-spectrum and related functions for the two examples discussed in Section 8.1.1. Example 8.3 For Example 8.1, we may use Equation (8.6) to derive the cross-spectrum from the crosscovariances by for 0<

面/The analysis of time series an introduction/files/page_162.html [5/24/2009 16:52:42]

page_163

page_163

< previous page

next page >

Page 163 Thus, using Equation (8.15), the coherency spectrum is given by This latter result may appear surprising at first sight. However, both Xt and Yt are generated from the same noise process and this explains why there is perfect correlation between the components of the two processes at any given frequency. Finally, using Equation (8.16), the gain spectrum is given by since the coherency is unity. Example 8.4 For the bivariate process of Example 8.2, Equation (8.6) gives the cross-spectrum as

, and this leads in turn to the following functions:

(8.17)

Then, as the two autospectra are given by

we find

Figure 8.1 The phase spectrum for Example 8.4 with d=4, with (a) phase unconstrained; (b) phase constrained. Note that all the above functions are defined on (0, π). The function of particular interest in this example is the phase, which, from Equation (8.17),

< previous page

file:///C:/Documents and Settings/Yang/

page_163

next page >

面/The analysis of time series an introduction/files/page_163.html [5/24/2009 16:52:43]

page_164

page_164

< previous page

next page >

Page 164 is a straight line with slope −d when XY( ) is unconstrained and is plotted against as a continuous function starting with zero phase at zero frequency (see Figure 8.1(a)). If, however, the phase is constrained to lie within the interval (−π,π) then a graph like Figure 8.1(b) will result, where the slope of each line is −d. This result is often used to help in the identification of relationships between time series. If the estimated phase approximates a straight line through the origin, then this indicates a delay between the two series equal to the slope of the line. More generally, the time delay between two recording sites will change with frequency, due, for example, to varying speeds of propagation. This is called the dispersive case, and can be recognized by changes in the slope of the phase function. 8.2.2 Estimation As in univariate spectral analysis, there are two basic approaches to estimating a cross-spectrum. First, we can take a Fourier transform of the truncated sample cross-covariance function (or of the cross-correlation function to get a normalized cross-spectrum). This is analogous to Equation (7.21) for the univariate case. The estimated co-spectrum is given by

(8.18) where M is the truncation point, and {λk} is the lag window. The estimated quadrature spectrum is given by

(8.19)

Equations (8.18) and (8.19) are often used in the equivalent forms

The truncation point M and the lag window {λk} are chosen in a similar way to that used in spectral analysis for a single series, with the Tukey and Parzen windows being most popular. Having estimated the co- and quadrature spectra, estimates of the cross-amplitude spectrum, phase and coherency follow, in an obvious way, from Equations (8.13), (8.14) and (8.15). Estimates of the power

, are needed in the latter case. We find

spectra of the two individual series, namely,

and

< previous page

page_164

file:///C:/Documents and Settings/Yang/

next page >

面/The analysis of time series an introduction/files/page_164.html [5/24/2009 16:52:44]

page_165

< previous page

page_165

next page >

Page 165

When plotting the estimated phase spectrum, similar remarks apply as to the (theoretical) phase. Phase estimates are undetermined by a multiple of 2π, but can usually be plotted as a continuous function, which is zero at zero frequency. Before estimating the coherency, it may be advisable to align the two series. If this is not done, Jenkins and Watts (1968) have demonstrated that estimates of coherency will be biased if the phase changes rapidly. If the sample cross-correlation function has its largest value at lag s say, then the two series are aligned by translating one series a distance s so that the peak in the cross-correlation function of the aligned series is at zero lag. The second approach to cross-spectral analysis is to smooth a function called the cross-periodogram. The univariate periodogram of a series {xt} can be written in the form

(8.20) using Equations (7.17) and (7.10), where {ap}, {bp} are the coefficients in the Fourier series representation of {xt}. By analogy with Equation (8.20), we may define the cross-periodogram of two series {xt} and {yt} as (8.21) After some algebra, it can be shown that the real and imaginary parts of IXY(

) are given by

where (apx, bpx), (apy, bpy) are the Fourier coefficients of {xt}, {yt}, respectively at . These real and imaginary parts may then be smoothed to get consistent estimates of the co- and quadrature spectral density functions by

where is chosen, as in the univariate case, so as to balance variance and resolution. These two equations are analogous to Equation (7.24) in the univariate case. The above estimates may then be used to estimate the cross-amplitude spectrum, phase and coherency as before. The computational advantages of the second type of approach are clear. Once a periodogram analysis has been made of the two individual processes,

< previous page

file:///C:/Documents and Settings/Yang/

page_165

next page >

面/The analysis of time series an introduction/files/page_165.html [5/24/2009 16:52:44]

page_166

< previous page

page_166

next page >

Page 166 and only involve the Fourier coefficients of nearly all the work has been done as the estimates of the two series. The disadvantage of the approach is that alignment is only possible if the cross-correlation function is calculated separately. This can be done directly or by the use of two (fast) Fourier transforms by an analogous procedure to that described in Section 7.4.5. The properties of cross-spectral estimators are discussed, for example, by Jenkins and Watts (1968), Priestley (1981) and Bloomfield (2000). The following points are worth noting. Estimates of phase and crossamplitude are imprecise when the coherency is relatively small. Estimates of coherency are constrained to lie between 0 and 1, and there may be a bias towards 1/2, which may be serious with short series. Finally, we note that rapid changes in phase may bias coherency estimates, which is another reason why alignment is generally a good idea. 8.2.3 Interpretation Cross-spectral analysis is a technique for examining the relationship between two series in the frequency domain. The technique may be used for two time series that ‘arise on a similar footing’ and then the coherency spectrum is perhaps the most useful function. It measures the linear correlation between two series at each frequency and is analogous to the square of the ordinary product-moment correlation coefficient. The other functions introduced in this chapter, such as the phase spectrum, are most readily understood in the context of linear systems, which will be discussed later in Chapter 9. We will therefore defer further discussion of how to interpret cross-spectral estimates until Section 9.3. Exercises 8.1 Show that the cross-covariance function of the stationary bivariate process {Xt, Yt} where

and {Z1, t}, {Z2, t} are independent purely random processes with zero mean and variance

, is given by

Hence evaluate the cross-spectrum. 8.2 Define the cross-correlation function ρXY( ) of a bivariate stationary process and show that | ρXY( ) | ≤1 for all .

< previous page

file:///C:/Documents and Settings/Yang/

page_166

next page >

面/The analysis of time series an introduction/files/page_166.html [5/24/2009 16:52:45]

page_167

page_167

< previous page

next page >

Page 167 Two first-order moving average processes

are formed from a purely random process {Zt}, which has mean zero and variance . Find the crosscovariance and cross-correlation functions of the bivariate process {Xt, Yt} and hence show that the crossspectrum is given by Evaluate the co-, quadrature, cross-amplitude, phase and coherency spectra.

< previous page

file:///C:/Documents and Settings/Yang/

page_167

next page >

面/The analysis of time series an introduction/files/page_167.html [5/24/2009 16:52:46]

page_168

< previous page

page_168

next page >

page_168

next page >

Page 168 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_168.html [5/24/2009 16:52:46]

page_169

< previous page

page_169

next page >

Page 169 CHAPTER 9 Linear Systems 9.1 Introduction An important problem in engineering and the physical sciences is that of identifying a model for a physical system (or process) given observations on the input and output to the system. For example, the yield from a chemical reactor (the output) depends in part on the temperature at which the reactor is kept (the input). Much of the literature assumes that the system can be adequately approximated over the range of interest by a linear model whose parameters do not change with time, although recently there has been increased interest in time-varying and non-linear systems. It turns out that the study of linear systems is useful, not only for examining the relationship between different time series, but also for examining the properties of linear filtering procedures such as many of the formulae for removing trend and seasonality. This chapter confines attention to time-invariant linear systems. After defining such a system, Sections 9.2 and 9.3 look at their properties and show how to describe a linear system in the time and frequency domains, respectively. Then Section 9.4 discusses how to identify the structure of a linear system from observed data. Much of the literature on linear systems (e.g. Bendat and Piersol, 2000) is written from an engineering viewpoint, and looks especially at such topics as control theory and digital communications (e.g. Glover and Grant, 1998), as well as the identification of input/output systems. We naturally concentrate on more statistical issues and acknowledge, in particular, the major contribution of Box et al. (1994). We generally denote the input and output series by {xt}, {yt}, respectively in discrete time, and by x(t), y(t), respectively in continuous time, though we sometimes use the latter notation for either type of series, as in the following definition. Definition of a Linear System. Suppose y1(t), y2(t) are the outputs corresponding to inputs x1(t), x2(t), respectively. Then the system is said to be linear if, and only if, any linear combination of the inputs, say 1x1(t)+ 2x2(t), produces the same linear combination of the outputs, namely, 1y1(t)+ 2y2(t), where 1, 2 are any constants1. 1 An alternative, neater way of writing this, is to denote the transformation effected by the system by say. and , the operator is Mathematicians would call this an operator. Then if linear if

< previous page

file:///C:/Documents and Settings/Yang/

page_169

next page >

面/The analysis of time series an introduction/files/page_169.html [5/24/2009 16:52:47]

page_170

< previous page

page_170

next page >

Page 170 As two special cases of the above definition, we note that a linear system preserves addition and scalar multiplication, and some writers define linearity by specifying two conditions that correspond to these two special cases. For addition, we set , and get a condition sometimes called the Principle of Superposition, which says that the sum of the inputs gives rise to the sum of the outputs. For scalar multiplication, we set , and get a condition sometimes called the homageneity, or proportionality or scale-invariant condition. Note that the latter condition means that if, for example, you double the input, then the output will also be doubled. We emphasize again that both these conditions are special cases of the more general definition given above. We further confine attention to linear systems that are time-invariant. This term is defined as follows. If input x(t) produces output y(t), then the system is said to be time-invariant if a delay of time in the input produces the same delay in the output. In other words, x(t− ) produces output y(t− ), so that the inputoutput relation does not change with time. Generally speaking, any equation with constant coefficients defines a time-invariant system (though it need not be linear). We only consider systems having one input and one output. The extension to several inputs and outputs is straightforward in principle, though more difficult in practice. The reader may readily verify that the following equations both define a time-invariant linear system:

where we note that (2) involves lagged values of the output. Differencing (or differentiation in continuous time) also gives a linear system. However, the reader should check that the equation which looks like a linear equation, does not in fact define a linear system2, although it is time-invariant. Of course, an equation involving a non-linear function, such as , is not linear either. Further note that an equation with time-varying coefficients, such as yt=0.5txt, will not be time-invariant, even if it is linear. 2 Hint: You can apply the definition of linearity to two carefully chosen known input series such as x1, t=K for all t, where K is a constant, and see what you get. Alternatively, write the equation in operator form as and then and so does not satisfy the Principle of Superposition. Mathematicians would describe a non-linear transformation of this type as an affine transformation, which can readily be made linear by a suitable linear transformation, namely, zt=yt−2.

< previous page

file:///C:/Documents and Settings/Yang/

page_170

next page >

面/The analysis of time series an introduction/files/page_170.html [5/24/2009 16:52:48]

page_171

< previous page

page_171

next page >

Page 171 9.2 Linear Systems in the Time Domain A time-invariant linear system may generally be written in the form (9.1)

in continuous time, or

(9.2) in discrete time. The weight function, h(u) in continuous time or {hk} in discrete time, provides a description of the system in the time domain. This function is called the impulse response function of the system, for reasons that will become clear in Section 9.2.2 below. It is obvious that Equations (9.1) and (9.2) define a linear system. Moreover, the fact that the impulse response functions do not depend on t, ensures that the systems are time invariant. A linear system is said to be physically realizable or causal if in continuous time, or in discrete time. We further restrict attention to stable systems for which any bounded input produces a bounded output, although control engineers are often concerned with controlling unstable systems. In discrete time, a sufficient condition for stability is that the impulse response function should satisfy

where C is a finite constant. In continuous time, the above sum is replaced by an appropriate integral. Engineers have been mainly concerned with continuous-time systems but are increasingly studying sampleddata control problems. Statisticians generally work with discrete data and so the subsequent discussion concentrates on the discrete-time case. 9.2.1 Some types of linear systems The linear filters introduced in Section 2.5.2 are examples of linear systems. For example, the simple moving average given by has impulse response function

< previous page

file:///C:/Documents and Settings/Yang/

page_171

next page >

面/The analysis of time series an introduction/files/page_171.html [5/24/2009 16:52:49]

page_172

page_172

< previous page

next page >

Page 172 Note that this filter is not ‘physically realizable’, as defined above, although it can of course be used in practice as a mathematical smoothing device. The filter can be made physically realizable by defining a new output variable, zt say, where zt=yt−1. Another general class of linear systems are those expressed as linear differential equations with constant coefficients in continuous time. For example, the equation

is a description of a linear system, where K>0 for stability. In discrete time, the analogue of differential equations are difference equations given by (9.3) where , and {αi}, {βj} are suitably chosen constants so as to make the system stable. Equation (9.3) can be rewritten as It is clear that Equation (9.4) can be rewritten in the form (9.2) by successive substitution, or, more elegantly, by using polynomial equations in the backward shift operator B. For example, if

then we can write the equation as

(9.4)

so that we find

Thus the impulse response function is given by

The reader will notice that hk is defined for all positive k giving an impulse response function of infinite order. Engineers call such a system an infinite impulse response (IIR) system. More generally, whenever the equation for the system includes lagged values of the output, as in Equation (9.4), the system will generally be IIR. However, if there are no lagged values of the output and a finite sum of input values at different lags, then engineers call the system a finite impulse response (FIR) system. Two very simple FIR linear systems are given by (9.5) called simple delay, where the integer d denotes the delay time, and (9.6) called simple gain, where g is a constant called the gain. The impulse

< previous page

file:///C:/Documents and Settings/Yang/

page_172

next page >

面/The analysis of time series an introduction/files/page_172.html [5/24/2009 16:52:49]

page_173

< previous page

page_173

next page >

Page 173 response functions of Equations (9.5) and (9.6) are

and

respectively. In continuous time, the impulse response functions of the corresponding equations for simple delay and and y(t)=gx(t) can only be represented in terms of a function simple gain, namely, called the Dirac delta function, denoted by δ(u), which is mathematically tricky to handle—see Appendix B. The resulting impulse response functions are δ(u− ) and gδ(u), respectively.

Figure 9.1 A delayed exponential response system showing graphs of: (a) the impulse response function; (b) an input with a unit step change at time zero; (c) the corresponding output.

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_173.html (1 of 2) [5/24/2009 16:52:50]

page_173

< previous page

file:///C:/Documents and Settings/Yang/

page_173

next page >

面/The analysis of time series an introduction/files/page_173.html (2 of 2) [5/24/2009 16:52:50]

page_174

page_174

< previous page

next page >

Page 174 An important class of impulse response functions, which often provides a reasonable approximation to physically realizable systems, is given by

A function of this type is called a delayed exponential, and depends on three constants, denoted by g, T and . The constant is called the delay. When =0, we have simple exponential response. The constant g is called the gain, and represents the eventual change in output when a step change of unit size is made to the input. The constant T governs the rate at which the output changes. Figure 9.1(a) presents a graph of the impulse response function for a delayed exponential system together with an input showing a step change of unity at time zero and the corresponding output—see Section 9.2.3 below. 9.2.2 The impulse response function: An explanation The impulse response function of a linear system describes how the output is related to the input as defined in Equations (9.1) or (9.2). The name ‘impulse response’ arises from the fact that the function describes the response of the system to an impulse input of unit size. For example, in discrete time, suppose that the input xt is zero for all t except at time zero when it takes the value unity. Thus

Then the output at time t is given by

Thus the output resulting from the unit impulse input is the same as the impulse response function, and this explains why engineers often prefer the description ‘unit impulse response function’. 9.2.3 The step response function An alternative, equivalent, way of describing a linear system in the time domain is by means of a function called the step response function, which is defined by

in continuous time, and

in discrete time.

< previous page

file:///C:/Documents and Settings/Yang/

page_174

next page >

面/The analysis of time series an introduction/files/page_174.html [5/24/2009 16:52:51]

page_175

page_175

< previous page

next page >

Page 175 The name ‘step response’ arises from the fact that the function describes the response of the system to a unit step change in the input at time zero. For example, in discrete time, consider the input which is zero before time zero but unity thereafter. This can be represented mathematically by

Then the corresponding output is given by

so that the output is equal to the step response function. Engineers sometimes use this relationship to measure the properties of a physically realizable system. The input is held steady for some time and then a unit step change is made to the input. The output is then observed and this provides an estimate of the step response function, and hence of its derivative, the impulse response function. A step change in the input may be easier to arrange in practice than an impulse. The step response function for a delayed exponential system is given by and the graph of y(t) in Figure 9.1(c) is also a graph of S(t). 9.3 Linear Systems in the Frequency Domain 9.3.1 The frequency response function An alternative way of describing a time-invariant linear system is by means of a function, called the frequency response function, which is the Fourier transform of the impulse response function. It is defined by (9.7)

in continuous time, and

(9.8) in discrete time. The frequency response function is sometimes given the alternative description of transfer function, but the former term is arguably more descriptive while the latter term is sometimes used in a different way—see below. The frequency response and impulse response functions are equivalent ways of describing a linear system, in a somewhat similar way that the autocovariance and power spectral density functions are equivalent ways of describing a stationary stochastic process, one function being the Fourier transform of the other. We shall see that, for some purposes,

< previous page

file:///C:/Documents and Settings/Yang/

is much

page_175

next page >

面/The analysis of time series an introduction/files/page_175.html [5/24/2009 16:52:52]

page_176

page_176

< previous page

next page >

Page 176 more useful than h(u) or hk. First, we prove the following theorem. Theorem 9.1 A sinusoidal input to a linear system gives rise, in the steady state, to a sinusoidal output at the same frequency. The amplitude of the sinusoid may change and there may also be a phase shift. Proof The proof is given for continuous time, the extension to discrete time being straightforward. Suppose that the input to a linear system, with impulse response function h(u), is given by Then the output is given by

Now cos(A−B)=cos A cos B+sin A sin B, so we may rewrite Equation (9.9) as

(9.9)

(9.10) As the two integrals do not depend on t, it is now obvious that y(t) is a mixture of sine and cosine terms at the same frequency . Thus the output is a sinusoidal perturbation at the same frequency as the input. If we write (9.11) (9.12) (9.13) (9.14)

then Equation (9.10) may be rewritten as

Equation (9.15) shows that a cosine wave is amplified by a factor G(

(9.15) ), which is called the gain of the

system. The equation also shows that the cosine wave is shifted by an angle ( ), which is called the phase shift. The above results have been derived for a particular frequency, , but hold true for any fixed such that >0. However, note that both the gain and phase shift may vary with frequency. Looking at Equation (9.14), the reader may note that the phase shift is not uniquely determined, because tan x=tan(x+2π). If we take the positive square root in Equation (9.13), so that the gain is required to be positive, then

< previous page

file:///C:/Documents and Settings/Yang/

page_176

next page >

面/The analysis of time series an introduction/files/page_176.html [5/24/2009 16:52:52]

page_177

< previous page

page_177

next page >

Page 177 the phase shift is undetermined by a multiple of 2π (see also Sections 8.2 and 9.3.2). This is not usually a problem in practice as the analyst may choose to constrain the phase to lie within a range, such as (−π, π), or allow it to be a continuous function. We have so far considered an input cosine wave. By a similar argument it can be shown that an input sine wave, , gives an output and phase shift. More generally if we consider an input given by

, so that there is the same gain

then the output is given by

(9.16) Thus we still have gain G( ) and phase shift ( ) We can now link these results back to the frequency response function, H( (9.12) give us

). Equations (9.7), (9.11) and

This may be further rewritten, using Equations (9.13) and (9.14), in the form (9.17) Thus, when the input in Equation (9.16) is of the form , the output is obtained simply by multiplying by the frequency response function, and we have (in the steady-state situation) that (9.18) This completes the proof of Theorem 9.1. Transients The reader should note that Theorem 9.1 only applies in the steady state where it is assumed that the input sinusoid was applied at t=−∞. If, in fact, the sinusoid is applied starting at say t=0, then the output will take some time to settle to the steady-state form given by the theorem. The difference between the observed output and the steady-state output is called the transient component. If the system is stable, then the transient component tends to zero as t→∞. If the relationship between input and output is expressed as a differential equation (or a difference equation in discrete time), then the reader who knows how to solve such equations will recognize that the steady-state behaviour of the system corresponds to the

< previous page

file:///C:/Documents and Settings/Yang/

page_177

next page >

面/The analysis of time series an introduction/files/page_177.html [5/24/2009 16:52:53]

page_178

< previous page

page_178

next page >

Page 178 particular integral of the equation, while the transient component corresponds to the complementary function. It is easier to describe the transient behaviour of a linear system by using the Laplace transform3 of the impulse response function. The Laplace transform also has the advantage of being defined for some unstable systems and so is sometimes preferred by engineers. However, statisticians customarily deal with steadystate behaviour for stable systems and so typically use Fourier transforms. We will continue this custom. Discussion of Theorem 9.1 Theorem 9.1 helps to explain the importance of the frequency response function. For inputs consisting of an impulse or step change it is easy to calculate the output using the (time-domain) impulse response function. However, for a sinusoidal input, it is much easier to calculate the output using the frequency response function. (Compare the convolution in the time domain as in Equation (9.9) with the simple multiplication in the frequency domain using Equation (9.18).) More generally for an input containing several sinusoidal perturbations, namely

it is easy to calculate the output using the frequency response function as

and this is one type of situation where linear systems are easier to study in the frequency domain. Further comments on the frequency response function Returning to the definition of the frequency response function as given by Equations (9.7) and (9.8), note that some authors define H( ) for negative as well as positive frequencies. However, for real-valued processes we need only consider H( ) for >0. Note that, in discrete time, H( ) is only defined for frequencies up to the Nyquist frequency π (or π/Δt if there is an interval Δt between successive observations). We have already introduced the Nyquist frequency in Section 7.2.1 and can apply similar ideas to a linear system measured at unit intervals of time. Given any sinusoidal input, with a frequency higher than π, we can find a corresponding sinusoid at a lower frequency in the range (0, π), which gives identical readings at unit intervals of time. This alternative sinusoid will therefore give rise to an identical output when measured only at unit intervals of time. We have already noted that H( ) is sometimes called the frequency response function and sometimes the transfer function. We prefer the former term, as it is more descriptive, indicating that the function shows how a linear system 3 The Laplace transform is defined in Appendix A.

< previous page

file:///C:/Documents and Settings/Yang/

page_178

next page >

面/The analysis of time series an introduction/files/page_178.html [5/24/2009 16:52:54]

page_179

< previous page

page_179

next page >

Page 179 responds to sinusoids at different frequencies. In any case the term ‘transfer function’ is used by some authors in a different way. Engineers sometimes use the term to denote the Laplace transform of the impulse response function—see, for example, Bendat and Piersol (2000, p. 30). For a physically realizable stable system, the Fourier transform of the impulse response function may be regarded as a special case of the Laplace transform. A necessary and sufficient condition for a linear system to be stable is that the Laplace transform of the impulse response function should have no poles in the right half-plane or on the imaginary axis. For an unstable system, the Fourier transform does not exist, but the Laplace transform does. However, we only consider stable systems, in which case the Fourier transform is adequate. Further confusion can arise because Jenkins and Watts (1968) use the term ‘transfer function’ to denote the z-transform4 of the impulse response function in the discrete-time case, while Box et al. (1994) use the term for a similar expression in connection with what they call transfer-function models—see Section 9.4.2. The ztransform is also used more generally by engineers for analysing discrete-time systems (e.g. Hayes, 1996, Chapter 2) but the Fourier transform is adequate for our purposes. 9.3.2 Gain and phase diagrams The frequency response function H( ) of a linear system is a complex function that may be written in the form where G(

),

( ) are the gain and phase, respectively—see Equation (9.17). In order to understand the

properties of a linear system, it is helpful to plot G( ) and ( ) against to obtain what are called the gain diagram and the phase diagram. The gain may be calculated via Equation (9.13) or equivalently as |H( )|, which may be obtained via where denotes the complex conjugate5 of H( ). If G ( ) is ‘large’ for low values of but ‘small’ for high values of as in Figure 9.2(a), then we have what is called a low-pass filter. This description is self-explanatory in that, if the input is a mixture of variation at several different frequencies, only those components with a low frequency will ‘get through’ the filter. Conversely, if G( ) is ‘small’ for low values of , but ‘large’ for high values of , then we have a highpass filter as in Figure 9.2(b). 4 The z-transform is defined in Appendix A. 5 If z=a+ib, then its complex conjugate is .

< previous page

file:///C:/Documents and Settings/Yang/

page_179

next page >

面/The analysis of time series an introduction/files/page_179.html [5/24/2009 16:52:55]

page_180

< previous page

page_180

next page >

Page 180

Figure 9.2 Gain diagrams for, (a) a low-pass filter; (b) a high-pass filter; (c) a simple moving average of three successive observations. The phase shift ( ) may be calculated from the real and imaginary parts of H( ) using Equation (9.14). Effectively this means plotting the value of H( ) in the complex plane and examining the result to see which quadrant it is in. The phase can be written directly as or equivalently as arctan[−B ( )/A( )]. However, as noted earlier, plotting the phase diagram is complicated by the fact that the phase is not uniquely determined. If the gain is always taken to be positive, then the phase is undetermined by a multiple of 2π and is often constrained to the range (−π, π). Unfortunately this may result in spurious discontinuities when the phase reaches the upper or lower boundary and engineers often prefer to plot the phase as a continuous unconstrained function, using the fact that (0)=0 provided G(0) is finite. Even then, the phase may have a discontinuity when the gain becomes zero as in Example 9.1 below, and this can only be avoided by allowing the gain to go negative.

< previous page

file:///C:/Documents and Settings/Yang/

page_180

next page >

面/The analysis of time series an introduction/files/page_180.html [5/24/2009 16:52:56]

page_181

< previous page

page_181

next page >

Page 181 9.3.3 Some examples Example 9.1 Consider the simple moving average which is a linear system with impulse response function

The frequency response function of this filter is (using Equation (9.8))

This function happens to be real, not complex, and so the phase appears to be given by However, H( ) is negative for then we have

>2π/3, and so if we adopt the convention that the gain should be positive,

and

The gain is plotted in Figure 9.2(c) and is of low-pass type. This is to be expected as a moving average smooths out local fluctuations (high-frequency variation) and measures the trend (low-frequency variation). In fact it is arguably more sensible to allow the gain to go negative in (2π/3, π) so that the phase is zero for all in (0, π). Example 9.2 A linear system showing simple exponential response has impulse response function Using Equation (9.7), the frequency response function is Hence

< previous page

file:///C:/Documents and Settings/Yang/

page_181

next page >

面/The analysis of time series an introduction/files/page_181.html [5/24/2009 16:52:57]

page_182

page_182

< previous page

next page >

Page 182 As the frequency increases, G( ) decreases so that the system is of low-pass type. As regards the phase, if we take ( ) to be zero at zero frequency, then the phase becomes increasingly negative as increases until the output is completely out of phase with the input (see Figure 9.3).

Figure 9.3 Phase diagram for a simple exponential response system. Example 9.3 Consider the linear system consisting of pure delay, so that where

is a constant. The impulse response function is given by

where δ denotes the Dirac delta function—see Appendix B. Then the frequency response function is given by

In fact H( that input

) can be derived without using the rather difficult delta function by using Theorem 9.1. Suppose is applied to the system. Then the output is . Thus, by analogy with Equation (9.18), we have . Then, using Equation (9.17), it is easy to see that this linear system has a constant gain

equal to unity, namely, G(

)=1, while the phase is given by

< previous page

file:///C:/Documents and Settings/Yang/

page_182

.

next page >

面/The analysis of time series an introduction/files/page_182.html [5/24/2009 16:52:58]

page_183

< previous page

page_183

next page >

Page 183 9.3.4 General relation between input and output So far, we have only considered sinusoidal inputs in the frequency domain. This section considers any type of input and shows that it is generally easier to work with linear systems in the frequency domain than in the time domain. The general relation between input and output in continuous time is given by Equation (9.1), namely (9.19) When x(t) is not of a simple form, this integral may be hard to evaluate. Now consider the Fourier transform of the output, given by

However,

for all values of u, and is therefore the Fourier transform of x(t), which we will denote by X(

). Furthermore

so that (9.20) Thus the integral in Equation (9.19) corresponds to a multiplication in the frequency domain provided that the Fourier transforms exist. A similar result holds in discrete time. A more useful general relation between input and output, akin to Equation (9.20), can be obtained when the input x(t) is a stationary process with a continuous power spectrum. This result will be given as Theorem 9.2. Theorem 9.2 Consider a stable time-invariant linear system with gain function G( ). Suppose that the input X (t) is a stationary process with continuous power spectrum fX( ). Then the output Y(t) is also a stationary process, whose power spectrum fY( ) is given by (9.21) Proof The proof will be given for continuous time, but a similar proof yields the same result in discrete time. It is easy to show that a stationary input to

< previous page

file:///C:/Documents and Settings/Yang/

page_183

next page >

面/The analysis of time series an introduction/files/page_183.html [5/24/2009 16:52:58]

page_184

< previous page

page_184

next page >

Page 184 a stable linear system gives rise to a stationary output, and this will not be shown here. We denote the impulse response and frequency response functions of the system by h(u), H( ), respectively. Thus G( )=|H( )|. By definition, the output is related to the input by

For mathematical convenience, we assume that the input has mean zero, in which case the output also has mean zero. It is straightforward to extend the proof to an input with non-zero mean. Denote the autocovariance functions (acv.f.s) of X(t), Y(t) by γX( ), γY( ), respectively. Then

But Thus (9.22) The relationship between the acv.f.s of the input and the output in Equation (9.22) is not of a simple form. However, if we take Fourier transforms of both sides of Equation (9.22) by multiplying by and integrating with respect to from −∞ to +∞, we find, using Equation (6.17), that the left-hand side is the spectrum of the output, namely

The right-hand side of Equation (9.22) requires some work to simplify. We find

However,

< previous page

file:///C:/Documents and Settings/Yang/

page_184

next page >

面/The analysis of time series an introduction/files/page_184.html [5/24/2009 16:52:59]

page_185

page_185

< previous page

next page >

Page 185 for all u, u′, and (denoting the complex conjugate of

Thus

This completes the proof of Theorem 9.2. The relationship between the spectra of the input and the output of a linear system is very important and yet of a surprisingly simple form. Once again a result in the frequency domain—Equation (9.21)—is much simpler than the corresponding result in the time domain—Equation (9.22). Theorem 9.2 can be used in various ways and, in particular, can be used to evaluate the spectra of various classes of stationary processes in a simpler manner to that used in Chapter 6. There the procedure was to evaluate the acv.f. of the process and then find its Fourier transform. This can be algebraically tedious, and so we give three examples showing how to use Equation (9.21) instead. (1) Moving average (MA) processes An MA process of order q is given by where Zt denotes a purely random process with variance . Usually β0 is one, but it simplifies the algebra to include this extra coefficient. Comparing with Equation (9.2), we see that the MA equation may be regarded as specifying a linear system with {Zt} as input and {Xt} as output. This system is stable and timeinvariant and so, using Equation (9.8), has frequency response function

As {Zt} is a purely random process, its spectrum is constant, namely Thus, using Equation (9.21), the spectrum of {Xt} is given by

For example, for the first-order MA process (9.23)

< previous page

file:///C:/Documents and Settings/Yang/

page_185

next page >

面/The analysis of time series an introduction/files/page_185.html [5/24/2009 16:53:00]

page_186

page_186

< previous page

next page >

Page 186 we have and

so that as already derived in Section 6.5. This type of approach can also be used when {Zt} is not a purely random process. For example, suppose that the {Zt} process in Equation (9.23) is stationary with spectrum fz( ). Then the spectrum of {Xt} is given by (2) Autoregressive (AR) processes The stationary first-order AR process with |α|

面/The analysis of time series an introduction/files/page_186.html [5/24/2009 16:53:01]

page_187

page_187

< previous page

next page >

Page 187 It can readily be shown that this is a time-invariant linear system. A differentiator is of considerable mathematical interest, although in practice only approximations to it are physically realizable. If the input is sinusoidal, say

, then, by differentiating, we find the output is given by

so that, using Equation (9.18), the frequency response function is given by If the input is a stationary process, with spectrum fx(

), then it appears that the output has spectrum

(9.26) However, this result assumes that the linear system in Equation (9.25) is stable, when in fact it is only stable for certain types of input processes. For example, it can be shown that the response to a unit step change is an unbounded impulse, which means the system is not stable. In order for the system to be stable, the variance of the output must be finite. Now

However, using equation (6.18), we have

and

so that

Thus Y(t) has finite variance provided that γX(k) can be differentiated twice at k=0, and only then does Equation (9.26) hold. 9.3.5 Linear systems in series

Figure 9.4 Two linear systems in series.

< previous page

file:///C:/Documents and Settings/Yang/

page_187

next page >

面/The analysis of time series an introduction/files/page_187.html [5/24/2009 16:53:02]

page_188

< previous page

page_188

next page >

Page 188 The advantages of working in the frequency domain are also evident when we consider two or more linear systems in series (sometimes said to be in cascade). For example, Figure 9.4 shows two linear systems in series, where the input x(t) to system I produces output y(t), which in turn is the input to system II producing output z(t). It is often of interest to evaluate the properties of the overall combined system, having x(t) as input and z(t) as output. It can readily be shown that the combined system is also linear, and we now find its properties. Denote the impulse response and frequency response functions of systems I and II by h1 ( ), h2( ), H1( ) and H2( ), respectively. In the time domain, the relationship between x(t) and z(t) would be in the form of a double integral involving h1(u) and h2(u), which is rather complicated. However, in the frequency domain we can denote the Fourier transforms of x(t), y(t), z(t) by X( ), Y( ), Z( ), respectively, and use Equation (9.20). Then and

Thus it is easy to see that the overall frequency response function of the combined system is (9.27)

If

then (9.28) Thus the overall gain is the product of the component gains, while the overall phase is the sum of the component phases. This result may be immediately applied to the case where the input x(t) is a stationary process with power spectrum fX( ). Generalizing Equation (9.21), we find The above results are easily extended to the situation where there are m linear systems in series with respective frequency response functions H1( ),…, Hm( ). The overall frequency response function is the product of the individual functions, namely

< previous page

file:///C:/Documents and Settings/Yang/

page_188

next page >

面/The analysis of time series an introduction/files/page_188.html [5/24/2009 16:53:03]

page_189

< previous page

page_189

next page >

Page 189 9.3.6 Design of filters We are now in a position to reconsider in more depth the properties of the filters introduced in Sections 2.5.2 and 2.5.3. Given a time series {xt}, the filters for estimating or removing trend are of the general form

This equation clearly defines a time-invariant linear system and its frequency response function is given by

. with gain function How do we set about choosing an appropriate filter for a time series? The design of a filter involves a choice of {hk} and hence of H( ) and G( ). Two types of ‘ideal’ filters are shown in Figure 9.5. Both have sharp cut-offs, the low-pass filter completely eliminating high-frequency variation and the high-pass filter completely eliminating low-frequency variation.

Figure 9.5 Two types of ideal filters; (a) a low-pass filter or trend estimator; (b) a high-pass filter or trend eliminator. However, ideal filters of this type are impossible to achieve with a finite set of weights. Instead the smaller the number of weights used, the less sharp will generally be the cut-off property of the filter. For example, the gain diagram of a simple moving average of three successive observations is of low-pass type but has a much less sharp cut-off than the ideal low-pass filter (compare Figure 9.2(c) with Figure 9.5(a)). More sophisticated trend estimators, such as Spencer’s 15-point moving average, have better cut-off properties. As an example of a trend eliminator, consider first differencing, namely

< previous page

file:///C:/Documents and Settings/Yang/

page_189

next page >

面/The analysis of time series an introduction/files/page_189.html [5/24/2009 16:53:03]

page_190

page_190

< previous page

next page >

Page 190 This has frequency response function and gain function which is plotted in Figure 9.6. This is indeed of high-pass type, as required, but the shape is a long way from the ideal filter in Figure 9.5(b). This should be borne in mind when working with first differences.

Figure 9.6 The gain diagram for the difference operator. 9.4 Identification of Linear Systems We have so far assumed that the structure of the linear system under consideration is known. Given the impulse response function of a system, or equivalently the frequency response function, we can find the output corresponding to a given input. In particular, when considering the properties of filters for estimating or removing trend and seasonality, a formula for the ‘system’ is given when the filter is specified. However, many problems concerning linear systems are of a completely different type. The structure of the system is not known a priori and the problem is to examine the relationship between input and output so as to infer the properties of the system. This procedure is called the system identification. For example, suppose we are interested in the effect of temperature on the yield from a chemical process. Here we have a physical system, which we assume, initially at least, is approximately linear over the range of interest. By examining the relationship between observations on temperature (the input) and yield (the output) we can infer the properties of the chemical process. The identification process is straightforward if the input to the system can be controlled and if the system is ‘not contaminated by noise’. In this case, we can simply apply an impulse or step change input, observe the output, and hence calculate the impulse response or step response function. Alternatively,

< previous page

file:///C:/Documents and Settings/Yang/

page_190

next page >

面/The analysis of time series an introduction/files/page_190.html [5/24/2009 16:53:04]

page_191

< previous page

page_191

next page >

Page 191 we can apply sinusoidal inputs at a range of different frequencies and observe the corresponding amplitude and phase shift of the output (which should be sinusoidal at the same frequency as the input if the system really is linear). This enables us to estimate the gain and phase diagrams. However, many systems are contaminated by noise as illustrated in Figure 9.7, where N(t) denotes a noise process. This noise process need not be white noise but it is usually assumed that it is uncorrelated with the input process X(t).

Figure 9.7 A linear system with added noise. A further difficulty arises when the input can be observed but cannot be controlled. In other words one cannot make changes, such as a step change, to the input in order to see what happens to the output. For example, attempts have been made to treat the economy of a country as a linear system and to examine the relationship between observed variables like the retail price index (regarded as the input) and average wages (regarded as the output). However, price increases can only be controlled to a certain extent by governmental decisions, and this makes system identification difficult. Moreover, we will see later in Section 9.4.3 that there is an additional problem with this sort of data in that the output (e.g. wage increases) may in turn affect the input (e.g. price increases) and this is called a feedback problem. When the system is affected by noise or the input is not controllable, more refined techniques are required to identify the system. We will describe two alternative approaches, one in the frequency domain and one in the time domain. Section 9.4.1 shows how cross-spectral analysis of input and output may be used to estimate the frequency response function of a linear system. Section 9.4.2 describes the Box-Jenkins approach to estimating the impulse response function of a linear system. 9.4.1 Estimating the frequency response function Suppose that we have a system with added noise, as depicted in Figure 9.7. Although the structure of the system is unknown, it is often reasonable to assume that it is linear, time-invariant and stable, and that the noise is a stationary process that is uncorrelated with the input and has mean zero. Suppose that we have observations on the input and output over some time period. The input should be observations on a stationary process, in which case the output will also be stationary. Given these observations, we wish to estimate the frequency response function of the system. We will denote the

< previous page

file:///C:/Documents and Settings/Yang/

page_191

next page >

面/The analysis of time series an introduction/files/page_191.html [5/24/2009 16:53:05]

page_192

< previous page

page_192

next page >

Page 192 (unknown) impulse response and frequency response functions of the system by h(u), H( The reader may think that Equation (9.21), namely

), respectively.

can be used to estimate the gain of the system, by estimating the spectra of input and output. However, this equation does not hold in the presence of noise and does not in any case give information about the phase of the system. Instead we derive a relationship involving the cross-spectrum of input and output. In continuous time, the output Y(t) is given by (9.29) Note that we are only considering physically realizable systems, so that h(u) is zero for u

面/The analysis of time series an introduction/files/page_192.html [5/24/2009 16:53:06]

page_193

page_193

< previous page

next page >

Page 193 where ƒXY is the cross-spectrum of input and output and ƒX is the (auto)spectrum of the input. Thus, once again, a convolution in the time domain corresponds to a multiplication in the frequency domain, and Equation (9.32) is much simpler than (9.30). Estimates of fXY( ) and ƒX( ) can now be used to estimate H( ) using Equation (9.32). Denote the estimated spectrum of the input by

and the estimate cross-spectrum by

. Then

In practice, we normally use the equation and estimate the gain and phase separately. We have

(9.33)

where αXY( ) is the cross-amplitude spectrum (see Equation (8.13)). We also find

(9.34) where q( ), c( ) are the quadrature and co-spectra, respectively (see Equation (8.14)). Thus, having estimated the cross-spectrum, Equations (9.33) and (9.34) enable us to estimate the gain and phase of the linear system, whether or not there is added noise. We can also use cross-spectral analysis to estimate the properties of the noise process. The discrete-time version of Equation (9.29) is

(9.35) For mathematical convenience, we again assume that E(Nt)=E(Xt)=0 so that E(Yt)=0. If we multiply both sides of (9.35) by Yt−m, we find

Taking expectations we find

since {Xt} and {Nt} are assumed to be uncorrelated. Taking Fourier transforms of both sides of this equation, we find However,

< previous page

file:///C:/Documents and Settings/Yang/

page_193

next page >

面/The analysis of time series an introduction/files/page_193.html [5/24/2009 16:53:06]

page_194

page_194

< previous page

next page >

Page 194 where C( ) denotes the coherency—see Equations (8.15) and (8.16). Thus (9.36)

Thus an estimate of fN( ) is given by

(9.37) Equation (9.36) also enables us to see that if there is no noise, so that there is a pure linear relation between Xt and Yt, then ƒN( )=0 and G( )=1 for all . On the other hand if C( )=0 for all , then fY( )=ƒN( ) and the output is not linearly related to the input. This confirms the point mentioned in Chapter 8 that the coherency C( ) measures the linear correlation between input and output at frequency . The results of this section not only show us how to identify a linear system by cross-spectral analysis but also give further guidance on the interpretation of functions derived from the cross-spectrum, particularly the gain, phase and coherency. Examples are given, for example, by Bloomfield (2000, Chapter 10) and Jenkins and Watts (1968). In principle, estimates of the frequency response function of a linear system may be transformed to give estimates of the impulse response function (Jenkins and Watts, 1968, p. 444; Box et al. 1994, Appendix A11.1) but I do not recommend this. For instance, Example 8.4 appears to indicate that the sign of the phase may be used to indicate which series is ‘leading’ the other. However, for more complicated lagged models of the form given by Equation (9.35), it becomes increasingly difficult to make inferences from phase estimates. For such time-domain models, it is usually better to try to estimate the relationships directly (perhaps after pre-whitening the series), rather than via spectral estimates. One approach is the so-called Box-Jenkins approach, described in the next subsection. 9.4.2 The Box-Jenkins approach This section gives a brief introduction to the method proposed by Box and Jenkins (1970, Chapters 10 and 11) 6 for identifying a physically realizable linear system, in the time domain, in the presence of added noise. The input and output series are both differenced d times until both are stationary, and are also meancorrected. The modified series will be denoted by {Xt}, {Yt}, respectively. We want to find the impulse response function {hk} of the system, where

The ‘obvious’ way to estimate {hk} is to multiply through Equation (9.38) by 6 There is virtually no change in Chapters 10 and 11 in Box et al. (1994).

< previous page

file:///C:/Documents and Settings/Yang/

page_194

(9.38)

next page >

面/The analysis of time series an introduction/files/page_194.html [5/24/2009 16:53:07]

page_195

page_195

< previous page

next page >

Page 195 Xt−m and take expectations to give (9.39) assuming that Nt is uncorrelated with the input. If we assume that the weights {hk} are effectively zero beyond say k=K, then the first K+1 equations of type (9.39) for m=0, 1,…, K, can be solved for the K+1 unknowns h0, h1,…, hk, on substituting estimates of γXY and γX. Unfortunately these equations do not, in general, provide good estimators for the {hk}, and, in any case, assume knowledge of the truncation point K. The basic trouble, as already noted in Section 8.1.2, is that autocorrelation within the input and output series will increase the variance of cross-correlation estimates. Box and Jenkins (1970) therefore propose two modifications to the above procedure. First, they suggest ‘prewhitening’ the input before calculating the sample cross-covariance function. Second, they propose an alternative form of Equation (9.35), which will in general require fewer parameters by allowing the inclusion of lagged values of the output7. They represent the linear system by the equation (9.40) This is rather like Equation (9.4), but is given in the notation used by Box and Jenkins (1970, Chapter 10) and involves an extra parameter b, which is called the delay of the system. The delay can be any nonnegative integer. Using the backward shift operator B, Equation (9.40) may be written as (9.41)

where and

Box and Jenkins (1970) describe Equation (9.41) as a transfer function model, which is a potentially misleading description in that the term ‘transfer function’ is sometimes used to describe some sort of transform of the impulse response function. Indeed Box and Jenkins (1970, Chapter 10) describe the generating function of the impulse response function, namely as the transfer function8. 7 This is similar to the idea that the general linear process, represented by an MA model of possibly infinite order, can often be parsimoniously approximated by a mixed ARMA model of low order—see Section 3.4.5. 8 Note that Box and Jenkins (1970) use the notation {vk} for the impulse response function. Box and Jenkins also use xt and yt for the differenced input and output, respectively, but we retain the convention of using capital letters to denote random variables and lower case letters for observed values.

< previous page

file:///C:/Documents and Settings/Yang/

page_195

next page >

面/The analysis of time series an introduction/files/page_195.html [5/24/2009 16:53:08]

page_196

page_196

< previous page

next page >

Page 196 The Box-Jenkins procedure begins by fitting an ARMA model to the (differenced) input. Suppose this model is of the form (see Section 3.4.5) where {at} denotes a purely random process, in the notation of Box and Jenkins (1970, Chapter 11). Thus we can transform the input to a white noise process by Suppose we apply the same transformation to the output, to give and then calculate the cross-covariance function of the filtered input and output, namely, {at} and {βt}. It turns out that this function gives a better estimate of the impulse response function, since if we write so that then we find

If we now evaluate the cross-covariance function of the two derived series, namely, {αt} and {βt}, then we find (9.42) since {αt} is a purely random process, and Nt is uncorrelated with {at}. Equation (9.42) is of a much simpler form than Equation (9.39) and will give more reliable estimates than those obtained by solving equations of type (9.39). This is partly because the sample cross-covariances of the derived series will have lower variances than those for the original series because there is less autocorrelation in the two series. Observed values of the derived prewhitened series can be found by calculating where , denote estimates of and θ when fitting an ARMA model to the input, and xt denotes the observed value of the input at time t. The same estimated transform is applied to the observed output values, and the sample cross-covariance function of observed variance of

t, namely,

and

namely, cαβ(k), is then computed. The

, is also computed. Then an estimate of hk is given by

(9.43) Box and Jenkins (1970) give the theoretical impulse response functions for various models given by Equation (9.40) and go on to show how the shape of

< previous page

file:///C:/Documents and Settings/Yang/

page_196

next page >

面/The analysis of time series an introduction/files/page_196.html [5/24/2009 16:53:09]

page_197

< previous page

page_197

next page >

Page 197 the estimated impulse response function given by Equation (9.43) can be used to suggest appropriate values for the integers r, b and s in Equation (9.40). They then show how to obtain least squares estimates of δ1, δ2,…, , ,…, given values of r, b and s. These estimates can in turn be used to obtain refined estimates of {hk} if desired. Box and Jenkins go on to show how a transfer function model, with added noise, can be used for forecasting and control. Several successful case studies have been published (e.g. Jenkins, 1979; Jenkins and Mcleod, 1982) and the method looks potentially useful. However, it should be noted that the main example discussed by Box and Jenkins (1970, Chapter 11), using some gas furnace data, has been criticized by Young (1984) and Chatfield (1977, p. 504) on a number of grounds, including the exceptionally high correlation between input and output, which means that virtually any identification procedure will give good results. Finally, it is worth noting that a somewhat similar method to the Box—Jenkins approach has been independently developed in the control engineering literature (Astrom and Bohlin, 1966; Astrom, 1970) and is called the Astrom—Bohlin approach. This method also involves prewhitening and a model similar to Equation (9.40), but the control engineering literature does not discuss identification and estimation procedures in the same depth as the statistical literature. One difference in the Astrom-Bohlin approach is that non-stationary series may be converted to stationarity by high-pass filtering methods other than differencing. The reader should note that we have implicitly assumed throughout this subsection that the output does not affect the input—in other words there is no feedback, as discussed below in Section 9.4.3. If there is a suspicion that feedback may be present, then it may be advisable to use alternative methods or try to fit the more general multivariate (vector) ARMA model as discussed later in Section 12.3. In fact we will see that a transfer function model can be regarded as a special case of the vector AR model. At this point the reader may be wondering whether it is better to adopt a frequency-domain approach using cross-spectral analysis or to fit the sort of time-domain parametric model considered in this subsection. My view is that it is unwise to attempt to make general pronouncements on the relative virtues of time-domain and frequency-domain methods. The two approaches are complementary rather than rivals, and it may be helpful to try both. It depends in part on the context, on the nature of the data and on what sort of model is desired. 9.4.3 Systems involving feedback A system of the type illustrated in Figure 9.7 is called an open-loop system, and the procedures described in the previous two sections are appropriate for data collected under these conditions. However, data are often collected from systems where some form of feedback control is being applied, and then

< previous page

file:///C:/Documents and Settings/Yang/

page_197

next page >

面/The analysis of time series an introduction/files/page_197.html [5/24/2009 16:53:10]

page_198

page_198

< previous page

next page >

Page 198 we have what is called a closed-loop system as illustrated in Figure 9.8. For example, when trying to identify a full-scale industrial process, it could be dangerous, or an unsatisfactory product could be produced, if some form of feedback control is not applied to keep the output somewhere near target. Similar problems arise in an economic context. For example, attempts to find a linear relationship showing the effect of price changes on wage changes are bedevilled by the fact that wage changes will in turn affect prices.

Figure 9.8 A closed-loop system. The problem of identifying systems in the presence of feedback control is discussed, for example, by Gustavsson et al. (1977) and Priestley (1983). The comments by Granger and Newbold (1986, Section 7.3) on ‘Causality and Feedback’ are also relevant. The key message is that open-loop procedures may not be applicable to data collected in a closed-loop situation. The problem can be explained more clearly in the frequency domain. Assuming that all processes are stationary, let ƒXY( ) denote the cross-spectrum of X(t) and Y(t) in Figure 9.8, and let ƒX( ), fN( ), fV( ) denote the spectra of X(t), N(t) and V(t), respectively. Then if H( ) and J( ) denote the frequency response functions of the system and controller, respectively, it can be shown that (9.44) where all terms are functions of frequency, and

is the complex conjugate of J. Only if

is the ratio ƒXY /ƒX equal to H as is the case for an open-loop system (Equation (9.32)). Thus the estimate of H provided by

will be poor unless ƒN/fV is small. In particular, if

will provide an estimate of J−1and not of H. Similar remarks apply to an analysis in the time domain. The time-domain equivalent of Equation (9.44) is given by Box and MacGregor (1974). The above problem is not specifically discussed by Box et al. (1994), although it is quite clear from the remarks in their Section 11.6 that their methods are only intended for use in open-loop systems. However, some confusion could be created by the fact that Box et al. (1994, Section 13.2) do

< previous page

file:///C:/Documents and Settings/Yang/

page_198

next page >

面/The analysis of time series an introduction/files/page_198.html [5/24/2009 16:53:11]

page_199

< previous page

page_199

next page >

Page 199 discuss ways of choosing optimal feedback control, which is quite a different problem. Having identified a system in open loop, they show how to choose feedback control action so as to satisfy some chosen criterion. Unfortunately, open-loop identification procedures have sometimes been used for a closed-loop system where they are not appropriate. Tee and Wu (1972) studied a paper machine while it was already operating under manual control and proposed a control procedure that has been shown to be worse than the existing form of control (Box and MacGregor, 1974). In marketing, several studies have looked at the relationship between advertising expenditure and sales of products such as washing-up liquid and coffee. However, expenditure on advertising is often chosen as a result of changes in sales levels, so that any conclusions obtained by an open-loop analysis are open to doubt. What then can be done if feedback is present? Box and MacGregor (1974) suggest one possible approach in which the analyst deliberately adds an independent noise sequence on top of the noise V(t). Alternatively, one may have some knowledge of the noise structure or of the controller frequency response function. Akaike (1968) claims that it is possible to identify a system provided only that instantaneous transmission of information does not occur in both system and controller, and an example of his, rather complicated, procedure is given by Otomo et al. (1972). We have left to last the final, and perhaps most important concern, namely, whether feedback is present within a particular system. Sometimes it is clear from the context whether feedback is present. For example, if one wanted to study the relationship between average (ambient) temperature and sales of a seasonal product, like ice cream, then it is clear that sales cannot possibly affect temperature. Thus one has an openloop system and open-loop procedures, such as the Box-Jenkins approach in Section 9.4.2, can be used. However, it is not always clear from the context alone as to whether feedback is present, particularly in economics and marketing. However, some contextual information may still be available. For example, economists know that prices typically affect wages, and wages in turn affect prices. If contextual information is not available, then the analyst may have to rely on data analysis to give some indication. For example, if significantly large cross-correlation coefficients between (prewhitened) input and output are observed at a zero or positive lag, then feedback may be present. However, this sort of inference is generally rather difficult to make in practice. The key point to remember is that, if feedback is present, then cross-correlation and cross-spectral analysis of the raw data may give misleading results if analysed in an open-loop way. Thus it is best to be conservative and allow for the possible presence of feedback if unsure about the true position. Exercises 9.1 Which of the following equations define a time-invariant linear system?

< previous page

file:///C:/Documents and Settings/Yang/

page_199

next page >

面/The analysis of time series an introduction/files/page_199.html [5/24/2009 16:53:11]

page_200

< previous page

page_200

next page >

Page 200 (a) yt=2xt (b) yt=0.7xt−3 (c) yt=0.5xt+0.3xt−1 (d) yt=0.5yt−1+0.3xt−1 (e) y(t)=1.5txt (f) (g) y(t)=1/x(t) (h) 9.2 Find the impulse response function, the step response function, the frequency response function, the gain and the phase shift for the following linear systems (or filters): (a) (b) (c) (d) where in each case t is integer valued. Plot the gain and phase shift for filters (a) and (c). Which of the filters are low-pass and which high-pass? If filters (a) and (b) are joined in series, find the frequency response function of the combined filter. 9.3 Find the frequency response functions of the following linear systems in continuous time: (a) (b) where g, T and are positive constants. 9.4 Consider the AR(1) process, given by Xt=αXt−1+Zt, where Zt denotes a purely random process with zero mean and constant variance and |a|

Page 201 using two different methods: (a) by transforming the autocovariance function; (b) by using the approach of Section 9.3.4. Suppose now that {Zt} is any stationary process with power spectrum fz( ). when Then is the power spectrum of {Xt} as defined by the above AR(1) model? 9.5 If {Xt} is a stationary time series in discrete time with power spectral density function f( ), show that the smoothed time series

where the ap are real constants, is a stationary process with power spectral density function

In particular, if ap=1/3 for p=0, 1, 2, show that the power spectrum of Yt is (Hint: Use Equation (9.21) and the trigonometric relation 9.6 Show that the power spectral density function of the ARMA(1, 1) process for 0<

面/The analysis of time series an introduction/files/page_201.html [5/24/2009 16:53:13]

page_202

< previous page

page_202

next page >

page_202

next page >

Page 202 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_202.html [5/24/2009 16:53:14]

page_203

< previous page

page_203

next page >

Page 203 CHAPTER 10 State-Space Models and the Kalman Filter A general class of models, arousing much interest in many areas of application, is that of state-space models. They were originally developed by control engineers, particularly for applications requiring continuous updating of the current position. An example, from the field of navigation systems, is ‘controlling the position of a space rocket’. However, state-space models have also found increasing use in many types of time-series problems, including parameter estimation, smoothing and prediction. This chapter introduces state-space models for the time-series analyst, as well as describing the Kalman filter, which is an important general method of handling state-space models. Essentially, Kalman filtering is a method of signal processing, which provides optimal estimates of the current state of a dynamic system. It consists of a set of equations for recursively estimating the current state of a system and for finding variances of these estimates. More details may be found, for example, in Harvey (1989), Janacek and Swift (1993) and Durbin and Koopman (2001). 10.1 State-Space Models When a scientist or engineer tries to measure any sort of signal, it will typically be contaminated by noise, so that the actual observation is given (in words) by (10.1)

The corresponding equation that is more familiar to statisticians is written as

Both these equations can be intuitively helpful in understanding the key distinction between ‘explained’ and ‘unexplained’ variation. However, as is customary, we prefer to use Equation (10.1) here because it seems more natural for introducing state-space models. The key step to defining the class of state-space models (in their basic linear form) is as follows. We assume that Equation (10.1) holds, but further assume that the signal is a linear combination of a set of variables, called state variables, which constitute what is called the state vector at time t. This vector describes the state of the system at time t, and is sometimes called the ‘state of nature’. The state variables can take many forms and the reader

< previous page

file:///C:/Documents and Settings/Yang/

page_203

next page >

面/The analysis of time series an introduction/files/page_203.html [5/24/2009 16:53:14]

page_204

< previous page

page_204

next page >

Page 204 should wait to examine the various examples given below before expecting to fully understand this concept. Although the above jargon derives from control engineering, it should be emphasized that the ideas are equally applicable in many other scientific areas. For example, in economics, the observation could be an economic variable, such as the unemployment rate, and the state variables could then include such (unobserved) quantities as the current true underlying level and the current seasonal factor (if any). It is an unfortunate complication that there is no standard notation for state-space models and the Kalman filter. We confine attention to the case of a univariate observed time series and denote the observation at time t by Xt. We denote the (m×1) state vector at time t by θt, and write Equation (10.1) as (10.2) where the (m×1) column vector ht is assumed to be a known vector1 and nt denotes the observation error. The state vector θt which is of prime importance, cannot usually be observed directly (i.e. is unobservable). The state variables are typically model parameters of some sort, such as regression coefficients in a regression model (see Section 10.1.6) or parameters describing the state of a system in a rather different way (see Sections 10.1.1–10.1.3). Thus, the analyst will typically want to use the observations on Xt to make inferences about θt. However, in some applications (see Section 10.1.4), the state vector will at least be partially known, but the state-space formulation and the Kalman filter may still be useful for making predictions and handling missing values, quite apart from estimating model parameters. Although θt may not be directly observable, it is often reasonable to assume that we know how it changes through time, and we denote the updating equation by (10.3) where the (m×m) matrix Gt is assumed known, and wt denotes a (m×1) vector of deviations such that . The pair of equations in (10.2) and (10.3) constitute the general form of the (univariate) state-space model. Equation (10.2) is called the observation (or measurement) equation, while (10.3) is called the transition (or state or system) equation. The ‘errors’ in the observation and transition equations are generally assumed to be serially uncorrelated and also to be uncorrelated with each other at all time periods. We may further assume that nt is N(0, ) while wt is multivariate normal with zero mean vector and a known variance-covariance matrix2 denoted by Wt. Note that if wt is independent of θt, θt−1, θt−2,…, 1 We use the superscript T throughout to denote the transpose of a matrix or vector. Here ht is a column is a row vector. vector, and so its transpose 2 This is a square symmetric matrix, of size (m×m), that specifies the variances of each

< previous page

file:///C:/Documents and Settings/Yang/

page_204

next page >

面/The analysis of time series an introduction/files/page_204.html [5/24/2009 16:53:15]

page_205

< previous page

page_205

next page >

Page 205 then the sequence {θt} is said to have the Markov property in that θt depends on θt−1 but not on earlier values. The state-space model can readily be generalized to the case where Xt is a vector by making ht a matrix of appropriate size and by making nt a vector of appropriate length. It is also possible to add terms involving known linear combinations of explanatory (or exogenous) variables to the right-hand side of Equation (10.2). The application of state-space models to engineering problems, such as controlling a dynamic system, is fairly clear. There the equations of motion of a system are often assumed to be known a priori, as are the properties of the system disturbances and measurement errors, although some model parameters may have to be estimated from data. Neither the equations nor the ‘error’ statistics need be constant as long as they are known functions of time. However, at first sight, state-space models may appear to have little connection with earlier time-series models. Nevertheless it can be shown, for example, that it is possible to put many types of time-series models into a state-space form. They include regression and autoregressive moving average (ARMA) models as well as various trend-and-seasonal models for which exponential smoothing methods are thought to be appropriate. In fact, the class of state-space models actually covers a very wide collection of models, often appearing under different names in different parts of the literature. For example, the so-called unobserved components models, used widely by econometricians, are of a state-space form. Bayesian forecasting (see Section 10.1.5) relies on a class of models, called dynamic linear models, which are essentially a statespace representation, while some models with time-varying coefficients can also be represented in this way. Moreover, Harvey (1989) has described a general class of trend-and-seasonal models, called structural models, which involve the classical decomposition of a time series into trend, seasonality and irregular variation, but which can also be represented as state-space models. We pay particular attention to these important models. Note that the model decomposition must be additive in order to get a linear state-space model. If, for example, the seasonal effect is thought to be multiplicative, then logarithms must be taken in order to fit a structural model, although this implicitly assumes that the ‘error’ terms are also multiplicative. A key feature of structural models (and more generally of linear state-space models) is that the observation equation involves a linear function of the state variables and yet does not restrict the model to be constant through time. Rather it allows local features, such as trend and seasonality, to be updated through time using the transition equation. Several examples of state-space models are presented in the following subsections, starting with the random walk plus noise model that involves just one state variable. element of wt on its diagonal and the covariances between pairs of elements of wt as the off diagonal terms. Thus the (i, j)th element of Wt is given by Cov{wi,t, wj, t}.

< previous page

file:///C:/Documents and Settings/Yang/

page_205

next page >

面/The analysis of time series an introduction/files/page_205.html [5/24/2009 16:53:16]

page_206

< previous page

page_206

next page >

Page 206 10.1.1 The random walk plus noise model Suppose that the observation equation is known to be given by where the unobservable local level µt is assumed to follow a random walk given by

(10.4)

(10.5) Here, Equation (10.5) is the transition equation, and the state vector θt consists of a single state variable, namely, µt. Thus θt is a scalar, rather than a vector, while ht and Gt are also constant scalars, namely, unity. The model involves two error terms, namely, nt and wt, which are usually assumed to be independent and normally distributed with zero means and respective variances namely,

and

The ratio of these two variances,

, is called the signal-to-noise ratio and is an important quantity in determining the

features of the model. In particular, if then µt is a constant and the model reduces to a trivial, constant-mean model. The state-space model defined by Equations (10.4) and (10.5) is usually called the random walk plus noise model, but has also been called the local level model and the steady model. Note that no trend term is included. While relatively simple, the model is very important since it can be shown that simple exponential smoothing produces optimal forecasts, not only for an ARIMA(0, 1, 1) model (see Chapter 5) but also for the above model (see Section 10.2 and Exercise 10.1). The reader can readily explore the relation with the ARIMA(0, 1, 1) model by taking first differences of Xt in Equation (10.4) and using Equation (10.5) to show that the first differences are stationary and have the same autocorrelation function as an MA(1) model. It can also be shown that the random walk plus noise model and the ARIMA(0, 1, 1) model give rise to the same forecast function. Thus the random walk plus noise model is an alternative to the ARIMA(0, 1, 1) model for describing data showing no long-term trend or seasonality but some short-term correlation. 10.1.2 The linear growth model The linear growth model is specified by these three equations

(10.6) The first equation is the observation equation, while the next two are transition equations. The state vector has two components, which can naturally be interpreted as the local level µt, and the local trend (or growth rate) βt. Note that the latter state variable does not actually appear in the observation equation. Comparing with the general state-space form in

< previous page

file:///C:/Documents and Settings/Yang/

page_206

next page >

面/The analysis of time series an introduction/files/page_206.html [5/24/2009 16:53:16]

page_207

< previous page

page_207

next page >

Page 207 and are both Equations (10.2) and (10.3), the reader may readily verify that constant through time. The title ‘linear growth model’ is reasonably self-explanatory, although it might be clearer to add the adjective ‘local’. If the trend term βt is constant, then the current level µt changes linearly through time. However, the trend (or growth rate) may also evolve through time. Of course if w1,t and w2,t have zero variance, then the trend is constant (or deterministic) and we have what is called a global linear trend model. However, this situation is arguably unlikely to occur in practice and the modern preference is to use the above local linear trend model where the trend is allowed to change. In any case the global model is a special case of Equation (10.6) and so it seems more sensible to fit the latter, more general model. The reader may easily verify that the second differences of Xt in Equation (10.6) are stationary and have the same autocorrelation function as an MA(2) model. In fact it can be shown that two-parameter exponential smoothing (where level and trend are updated) is optimal for an ARIMA(0, 2, 2) model and also for the above linear growth model (e.g. see Abraham and Ledolter, 1986). It is arguably easier to get a variety of trend models from special cases of a general structural state-space model than from the Box-Jenkins ARIMA class of models. 10.1.3 The basic structural model There are various ways of incorporating seasonality into a state-space model. An important example is the following model specified by four equations: (10.7)

Here, µt denotes the local level, βt denotes the local trend, it denotes the local seasonal index and s denotes the number of periods in 1 year (or season). The model also incorporates four separate ‘error’ terms that are all assumed to be additive and to have mean zero. Note that the fourth equation in (10.7) assumes that the expectation of the sum of the seasonal effects over 1 year is zero. In this model, the state vector has s+2 components, namely, µt, βt, it, it−1,…, it−s+1. The model is similar in spirit to that implied by the additive Holt-Winters method (see Section 5.2.3). The latter depends on three smoothing parameters, which correspond in some sense to the three error variance ratios, namely,

and

where

and for i=1, 2, 3. The above model is called the basic structural model by Harvey (1989),

< previous page

file:///C:/Documents and Settings/Yang/

page_207

next page >

面/The analysis of time series an introduction/files/page_207.html [5/24/2009 16:53:17]

page_208

< previous page

page_208

next page >

Page 208 who discusses the properties of the model in some detail. Harvey (1989) also discusses various extensions of the model, such as the incorporation of explanatory variables. Andrews (1994) gives some encouraging empirical results as regards the forecasting ability of structural models. An alternative class of seasonal state-space models are described by Ord et al. (1997) and have the feature that they incorporate a single source of error. Some special cases of this class of models are found to be models for which exponential smoothing is optimal (Chatfield et al., 2001). 10.1.4 State-space representation of an AR(2) process We illustrate the connection between state-space models and ARIMA models by considering the AR(2) model. The latter can be written as (10.8) Consider the (rather artificial) state vector at time t defined by equation may be written (trivially) as with equation

and

=

. Then the observation

, while Equation (10.8) may be written as the first line of the transition

(10.9) . since This looks (and is!) a rather contrived piece of mathematical trickery, and we normally prefer to use Equation (10.8), which appears more natural than Equation (10.9). (In contrast the state-space linear growth model in Equation (10.6) may well appear more natural than an ARIMA(0, 2, 2) model.) However, Equation (10.9) does replace two-stage dependence with two equations involving one-stage dependence, and also allows us to use the general results relating to state-space models, such as the recursive estimation of parameters, should we need to do so. For example, the Kalman filter provides a general method of estimation for ARIMA models (e.g. Kohn and Ansley, 1986). However, it should also be said that the approach to identifying statespace models is generally quite different to that for ARIMA models in that more knowledge about model structure is typically assumed a priori (see Section 10.1.7). Note that the state-space representation of an ARMA model is not unique and it may be possible to find many equivalent representations. For example, the reader may like to find alternative state-space representations of Equation (10.8) using the state vector

or using the (more useful?)

, where is the optimal one-step-ahead forecast at time t—see state vector Exercise 10.4. Of course, in this case, the state vector can be observed directly, and the problem is no longer one of estimating θt.

< previous page

file:///C:/Documents and Settings/Yang/

page_208

next page >

面/The analysis of time series an introduction/files/page_208.html [5/24/2009 16:53:18]

page_209

< previous page

page_209

next page >

Page 209 However, the state-space formulation may still be useful for other purposes, such as making predictions. For the other two forms of the state vector, the first component can be observed directly, but the second component contains unobserved parameters and so does need to be estimated. 10.1.5 Bayesian forecasting Bayesian forecasting (West and Harrison, 1997) is a general approach to forecasting that includes a variety of methods, such as regression and exponential smoothing, as special cases. It relies on a model, called the dynamic linear model, which is closely related to the general class of state-space models. The Bayesian formulation means that the Kalman filter is regarded as a way of updating the (prior) probability distribution of θt when a new observation becomes available to give a revised (posterior) distribution. The Bayesian approach also enables the analyst to consider the case where several different models are entertained and it is required to choose a single model to represent the process. Alternatively, when there are several plausible candidate models, the approach allows the analyst to compute some sort of combined forecast. For example, when the latest observation appears to be an outlier, one could entertain the possibility that this represents a step change in the process, or that it arises because of a single intervention, or that it is a ‘simple’ outlier with no change in the underlying model. The respective probabilities of each model being ‘true’ are updated after each new observation. An expository introduction to the Bayesian approach, together with case studies and computer software, is given by Pole et al. (1994). Further developments are described by West and Harrison (1997). The approach has some staunch adherents, while others find the avowedly Bayesian approach rather intimidating. This author has no practical experience with the approach. Fildes (1983) suggested that the method is generally not worth the extra complexity compared with alternative, simpler methods. However, some of the examples in Pole et al. (1994) and in West and Harrison (1997) are persuasive, especially for short series with prior information. 10.1.6 A regression model with time-varying coefficients This example demonstrates how state-space models can be used to generalize familiar constant-parameter models to the situation where the model parameters are allowed to change through time. Suppose that the observed variable Xt is known to be linearly related to a known explanatory variable ut by If the parameters at and bt are constant, then we have the familiar linear regression model, but we suppose instead that the regression coefficients at and bt are allowed to evolve through time according to a random walk. The two

< previous page

file:///C:/Documents and Settings/Yang/

page_209

next page >

面/The analysis of time series an introduction/files/page_209.html [5/24/2009 16:53:19]

page_210

< previous page

page_210

next page >

Page 210 regression coefficients may be taken as the components of the (unobservable) state vector θt, while the values of the explanatory variable ut are put into the known vector ht. Thus writing

and

the model may be written in in state-space form as

(10.10) where the transition matrix Gt is constant, namely, the constant (2×2) identity matrix given by . Of course if the elements of wt have zero variance, then θt is constant, say , and we are back to the usual linear regression model with constant coefficients. In this case, the transition equation is a trivial identity and there is little point in using the state-space format. The advantage of Equation (10.10) is that it covers a much more general class of models, including ordinary linear regression as a special case, but also allowing the model parameters to change through time. As well as covering a wider class of possibilities, the state-space formulation means that we can apply the general theory relating to state-space models. 10.1.7 Model building An important difference between state-space modelling in time-series applications and in some engineering problems is that the structure and properties of a time series will usually not be assumed known a priori. In order to apply state-space theory, we need to know ht and Gt in the model equations and also to know the variances and covariances of the disturbance terms, namely, and Wt. The choice of a suitable statespace model (i.e. the choice of suitable values for ht and Gt) may be accomplished using a variety of aids including external knowledge and a preliminary examination of the data. For example, Harvey (1989) claims that the basic structural model (see Section 10.1.3) can describe many time series showing trend and seasonality, but, to use the standard basic structural model, the analyst must, for example, check that the seasonal variation really is additive. If it is not additive, the analyst should consider transforming the data or trying an alternative model. In other words, the use of a state-space model does not take away the difficult problem of finding a suitable model for a given set of data. As usual, model fitting is easy, but model building can be hard. Another problem in time-series applications is that the error variances are generally not known a priori. This can be dealt with by guesstimating them, and then updating them in an appropriate way, or, alternatively, by estimating them from a set of data over a suitable fit period.

< previous page

file:///C:/Documents and Settings/Yang/

page_210

next page >

面/The analysis of time series an introduction/files/page_210.html [5/24/2009 16:53:20]

page_211

< previous page

page_211

next page >

Page 211 10.2 The Kalman Filter In state-space modelling, the prime objective is usually to estimate the signal in the presence of noise. In other words we want to estimate the (m×1) state vector θt, which cannot usually be observed directly. The Kalman filter provides a general method for doing this. It consists of a set of equations that allow us to update the estimate of θt when a new observation becomes available. We will see that this updating procedure has two stages, called the prediction stage and the updating stage. Suppose we have observed a univariate time series up to time (t−1), and that is the ‘best’ estimator for θt−1 based on information up to this time. Here ‘best’ is defined as the minimum mean square error estimator. Further, suppose that we have evaluated the (m×m) variance-covariance matrix3 of which we denote by Pt−1. The first stage, called the prediction stage, is concerned with forecasting θt from data up to time (t−1), and we denote the resulting estimator in an obvious notation by . Considering Equation (10.3), where wt is still unknown at time t−1, the obvious estimator for θt is given by (10.11)

with variance-covariance matrix

(10.12) Equations (10.11) and (10.12) are called the prediction equations. Equation (10.12) follows from standard results on variance-covariance matrices for vector random variables (e.g. Chatfield and Collins, 1980, Equation (2.9)). When the new observation at time t, namely, Xt, has been observed, the estimator for θt can be modified to take account of this extra information. At time (t−1), the best forecast of Xt is given by the prediction error is given by

so that

This quantity can be used to update the estimate of θt and of its variance-covariance matrix. It can be shown that the best way to do this is by means of the following equations: (10.13)

and

(10.14)

where

(10.15) is called the Kalman gain matrix. In the univariate4 case, Kt is just a vector of size (m×1). Equations (10.13) and (10.14) constitute the second updating stage of the Kalman filter and are called the updating equations. 3 See footnote on p. 204. 4 The observation Xt is univariate, but remember that the state vector θt is (m×1).

< previous page

file:///C:/Documents and Settings/Yang/

page_211

next page >

面/The analysis of time series an introduction/files/page_211.html [5/24/2009 16:53:21]

page_212

page_212

< previous page

next page >

Page 212 We will not attempt to derive the updating equations or to demonstrate the optimality of the Kalman filter. However, we note that the results may be found via least squares theory or using a Bayesian approach. A clear introduction to the Kalman filter is given by Meinhold and Singpurwalla (1983), while more detailed accounts are given by Harvey (1989; 1993, Chapter 4), Aoki (1990) and Durbin and Koopman (2001). A major practical advantage of the Kalman filter is that the calculations are recursive, so that, although the current estimates are based on the whole past history of measurements, there is no need for an everexpanding memory. Rather the new estimate of the signal is based solely on the previous estimate and the latest observation. A second advantage of the Kalman filter is that it converges fairly quickly when there is a constant underlying model, but can also follow the movement of a system where the underlying model is evolving through time. The Kalman filter equations look rather complicated at first sight, but they may readily be programmed in their general form and reduce to much simpler equations in certain special cases. For example, consider the random walk plus noise model of Section 10.1.1 where the state vector θt consists of just one state variable, the current level µt. After some algebra (e.g. Abraham and Ledolter, 1986), it can be shown that the Kalman filter for this model in the steady-state case (as t→∞) reduces to the simple recurrence relation (10.16) where the smoothing constant a is a (complicated) function of the signal-to-noise ratio 10.1). Equation (10.16) is, of course, simple exponential smoothing. When

(see Exercise

tends to zero, so that µt is a

constant, we find that α tends to zero as would intuitively be expected, while as becomes large, then α approaches unity. As a second example, consider the linear regression model with time-varying coefficients in Section 10.1.6. Abraham and Ledolter (1983, Section 8.3.3) show how to find the Kalman filter for this model. In particular, it is easy to demonstrate that, when Wt is the zero matrix, so that the regression coefficients are constant, then Gt is the identity matrix, while

. Then the Kalman filter reduces to the equations

where

Abraham and Ledolter (1983, Section 8.3.3) demonstrate that these equations are the same as the ‘wellknown’ updating equations for recursive least squares provided that starting values are chosen in an appropriate way. In order to initialize the Kalman filter, we need estimates of θt and Pt at

< previous page

file:///C:/Documents and Settings/Yang/

page_212

next page >

面/The analysis of time series an introduction/files/page_212.html [5/24/2009 16:53:22]

page_213

< previous page

page_213

next page >

Page 213 the start of the series. This can be done by a priori guesswork, relying on the fact that the Kalman filter will rapidly update these quantities so that the initial choices become dominated by the data. Alternatively, one may be able to estimate the (m×1) vector θt at time t=m by least squares from the first m observations, since if we can write where independent ‘error’ terms, then

, M is a known non-singular (m×m) matrix and e is an m-vector of

(10.17) is the least squares estimate of θm (since M is a square matrix). An example is given in Exercise 10.2. Once a model has been put into state-space form, the Kalman filter can be used to provide recursive estimates of the signal, and they in turn lead to algorithms for various other calculations, such as making predictions and handling missing values. For example, forecasts may readily be obtained from the statespace model using the latest estimate of the state vector. Given data to time N, the best estimate of the state vector is written as

N and the h-step-ahead forecast is given by

where we assume hN+h and future values of Gt are known. Of course if Gt is a constant, say G, then (10.18) If future values of ht or Gt are not known, then they must themselves be forecasted or otherwise guesstimated. The Kalman filter is applied to state-space models that are linear in the parameters. In practice many timeseries models, such as multiplicative seasonal models, are non-linear. Then it may be possible to apply a filter, called the extended Kalman filter, by making a locally linear approximation to the model. Applications to data where the noise is not necessarily normally distributed are also possible but we will not pursue these more advanced topics here (see, for example, Durbin and Koopman, 2001). 10.2.1 The Kalman filter for the linear growth model As an example, we will evaluate the Kalman filter for the linear growth model of Section 10.1.2. Suppose that, from data up to time (t−1), we have estimates and of the level and trend. At time (t−1) the best forecasts of w1,t and W2,t are both zero so that the best forecasts of µt and βt in Equation (10.6) are clearly given by

< previous page

file:///C:/Documents and Settings/Yang/

page_213

next page >

面/The analysis of time series an introduction/files/page_213.html [5/24/2009 16:53:23]

page_214

< previous page

page_214

next page >

Page 214 and These agree with Equation (10.11). When Xt becomes available, we can find the prediction error, namely, equations, namely

, and this can then be inserted into Equation (10.13) to give the following pair of scalar

and where k1,t, k2,t are the elements of the Kalman gain ‘matrix’ (here a 2×1 vector) Kt, which can be evaluated after some algebra. It is interesting to note that these two equations are of similar form to those in Holt’s two-parameter (non-seasonal) version of exponential smoothing (see Section 5.2.3). There the level and trend are denoted by Lt, Tt, respectively, and we have, for example, that

where et=Xt−[Lt−1+Tt−1]. In the steady state as t→∞, it can be shown that k1,t tends to a constant, which corresponds to the smoothing parameter α. This demonstrates that the forecasting method called Holt’s (twoparameter) exponential smoothing is optimal for the linear growth model. An intuitively reasonable way to initialize the two state variables from the first two observations is to take and (see Exercise 10.2). Exercises 10.1 Consider the random walk plus noise model in Section 10.1.1, and denote the signal-to-noise ratio by c. Show that the first-order autocorrelation coefficient of (1−B)Xt is −1/(2+c) and that higherorder autocorrelations are all zero. For the ARIMA(0,1,1) model show that the first-order autocorrelation coefficient of (1−B)Xt is θ/(1+θ2) and that higher-order autocorrelations are all zero. Thus the two models have equivalent autocorrelation properties when . Hence show that the invertible solution, with Applying the Kalman filter to the random walk plus noise model, we find (after some algebra) that, in the steady state (as t→∞ and Pt→constant), we have

< previous page

file:///C:/Documents and Settings/Yang/

page_214

next page >

面/The analysis of time series an introduction/files/page_214.html [5/24/2009 16:53:24]

page_215

page_215

< previous page

next page >

Page 215 and

and this is simple exponential smoothing. Now the ARIMA model is invertible provided that −1

Page 219 Non-linear models can also be used to explain, and give forecasts for, data exhibiting regular cyclic behaviour. As such they provide an interesting alternative to the use of harmonic components, especially if the behaviour is asymmetric. For some non-linear models, if the noise process is ‘switched off’, then the process will converge asymptotically to a strictly periodic form called a limit cycle (Priestley, 1988; Tong, 1990). Questions about non-linearity also arise when we consider transforming a variable using a non-linear transformation such as the Box-Cox transformation (see Section 2.4). Data may be transformed for a variety of reasons such as to make the data more normally distributed or to achieve constant variance. However, if we are able to fit a linear model to the transformed data, this will imply that a non-linear model is appropriate for the original data. In particular, a series that shows multiplicative seasonality can be transformed to additive seasonality by taking logs and can then be handled using linear methods. However, the multiplicative model for the original data will be non-linear. Before embarking on a non-linear analysis, it is sensible to check that the data really are non-linear (Darbellay and Slama, 2000). There are various technical procedures described in the literature for assessing and testing different aspects of non-linearity. A recent survey, with further references, is given by Tsay (2001), and the so-called BDS test, in particular, is briefly discussed later in this chapter. However, tests for non-linearity often have poor power, and the simplest, and arguably the most important, tool (as in the rest of time-series analysis) is a careful inspection of the time plot. Behaviour such as that seen in Figure 11.1 can be self-evident provided the scales are chosen carefully. It should also be noted that tests for non-linearity can have difficulty in distinguishing between data from a non-linear model and data from a linear model to which outliers have been added. While some non-linear models can give rise to occasional sharp spikes, the same is true of a linear model with occasional outliers. Here again, a careful inspection of the time plot can be both crucial and fruitful, especially when allied to expert contextual knowledge as to when, where and why unusual observations might occur. This highlights the close connection between non-linearity and nonnormality. If, for example, a time series exhibits more ‘spikes’ up than down, then it is often not clear if this is due to non-linearity, non-normality, or both. 11.1.2 What is a linear model? The first point to make is that there is no clear consensus as to exactly what is meant by a linear stochastic time-series model and hence no consensus as to what is meant by a non-linear model. In much of statistical methodology, the term general linear model is used to describe a model that is linear in the parameters but that could well involve non-linear functions of the explanatory variables in the so-called design matrix. In contrast a linear model (or linear system) in time series would certainly exclude non-linear functions

< previous page

file:///C:/Documents and Settings/Yang/

page_219

next page >

面/The analysis of time series an introduction/files/page_219.html [5/24/2009 16:53:27]

page_220

< previous page

page_220

next page >

Page 220 of lagged or explanatory variables. To complicate matters further, an MA process, which is certainly a linear process (see Chapter 3) and which, at first sight, appears to be linear in the parameters, is actually regarded as non-linear in the parameters in that the one-step-ahead errors (on which least-squares estimation is based) are non-linear functions of the parameters. This means that explicit analytic formulae for estimators may not be available (see Section 4.3.1) and Box and Jenkins (1970) use the term non-linear estimation to describe procedures for minimizing a sum-of-squares function when numerical methods (such as hillclimbing) have to be used. This chapter only discusses the use of the term ‘non-linear’ as applied to models and forecasting methods. The most obvious example of a linear model is the general linear process (see Section 3.4.7), which arises when the value of a time series, say Xt, can be expressed as a linear function of the present and past values of a purely random process, say Zt. This class of models includes stationary autoregressive (AR), moving average (MA), and ARMA models. The linearity of the process is clear when the model is viewed as a linear system (see Chapter 9) for converting the sequence of Zts into a sequence of Xts. In addition the statespace model defined by Equations (10.2) and (10.3) is generally regarded as linear provided the disturbances are normally distributed, ht is a constant known vector and Gt, Wt are constant known matrices (or at least are non-stochastic, so that if they change through time, they do so in a predetermined way). A linear forecasting method is one where the h-steps-ahead forecast at time N can be expressed as a linear function of the observed values up to, and including, time N. This applies to exponential smoothing, and the additive (though not the multiplicative) version of Holt-Winters. It also applies to minimum mean square error (MMSE) forecasts derived from a stationary ARMA model with known parameters, as would be expected for a general linear process. However, note that when the model parameters have to be estimated from the data (as is normally the case), the MMSE forecasts will not be linear functions of past data. The status of (non-stationary) ARIMA models is not so obvious. Apart from the non-stationarity (which means they can’t be expressed as a general linear process), they look linear in other respects. An ARI model, for example, can be regarded as a linear system by treating {Xt} and {Zt} as if they were the input and output, respectively, although it is really the other way round. Moreover MMSE forecasts from ARIMA models (assuming known model parameters) will be linear functions of past data. This suggests that it might be possible to define a linear model as any model for which MMSE forecasts are linear functions of observed data. However, while this is a necessary condition, it is not sufficient, because some models give linear prediction rules while exhibiting clear non-linear properties in other respects. A further complication is that it is possible to have models that are locally linear, but globally non-linear (see Sections 2.5 and 10.1.2). Thus it appears that it may not be feasible to define linearity precisely, but rather that it is possible to move gradually away from linearity towards non-linearity.

< previous page

file:///C:/Documents and Settings/Yang/

page_220

next page >

面/The analysis of time series an introduction/files/page_220.html [5/24/2009 16:53:28]

page_221

< previous page

page_221

next page >

Page 221 11.1.3 What is a non-linear model? A non-linear model could be defined by exclusion as any model that is not linear. However, this is not helpful since (1) linear models have not been exactly defined, and (2) there is a blurred borderline between linearity and non-linearity. For example, some long-memory models (see Section 13.3) are more linear than nonlinear, while for non-stationary models, the non-stationarity property is often more important than whether the model is linear. In any case, as noted earlier, a variable, which can be described by a linear model, becomes non-linear after applying a non-linear transformation. This chapter therefore restricts attention to certain classes of models that are conventionally regarded as nonlinear models, even though some of them have some linear characteristics while some excluded models have non-linear characteristics. 11.1.4 What is white noise? When examining the properties of non-linear models, it can be very important to distinguish between independent and uncorrelated random variables. In Section 3.4.1, white noise (or a purely random process) was defined to be a sequence of independent and identically distributed (i.i.d.) random variables. This is sometimes called strict white noise (SWN), and the phrase uncorrelated white noise (UWN) is used when successive values are merely uncorrelated, rather than independent. Of course if successive values follow a normal (Gaussian) distribution, then zero correlation implies independence so that Gaussian UWN is SWN. However, with non-linear models, distributions are generally non-normal and zero correlation need not imply independence. In reading the literature, it is also helpful to understand the idea of a martingale difference. A series of random variables {Xt} is called a martingale if E[Xt+1| data to time t] is equal to the observed value of Xt, say xt. Then a series {Yt} is called a martingale difference (MD) if E[Yt+1| data to time t]=0. This last result follows by letting {Yt} denote the first differences of a martingale, namely, Yt=Xt−Xt−1.An MD is like UWN except that it does not need to have constant variance. Of course a Gaussian MD with constant variance is SWN. UWN and MDs have known linear, second-order properties. For example, they have constant mean and zero autocorrelations. However, the definitions say nothing about the non-linear properties of such series. In particular, although {Xt} may be UWN or an MD, the series of squared observations

need not be. Only if

{Xt} is SWN, will be UWN. There are many tests for linearity whose power depends on the particular type of non-linearity envisaged for the alternative hypothesis (e.g. see Brock and Potter, 1993; Tsay, 2002, Section 4.2). These tests generally involve looking at the properties of moments of {Xt}, which are higher than second order, particularly at the autocorrelation function of which involves fourth-order moments. Some analysts also like to look at polyspectra, which are the frequency

< previous page

file:///C:/Documents and Settings/Yang/

page_221

next page >

面/The analysis of time series an introduction/files/page_221.html [5/24/2009 16:53:28]

page_222

< previous page

page_222

next page >

Page 222 domain equivalent of this and involve taking the Fourier transform of higher order moments of the process. The simplest example is the bispectrum, which is the transform of third-order terms of the general form XtXt −jXt−k. For linear Gaussian processes, all polyspectra (including the bispectrum) are identically zero for order three or more and so bispectra have been used as part of test procedures for normality and for linearity. The sunspots data and the lynx data, for example, are both found to be non-linear and non-normal using tests based either on the bispectrum or on time-domain statistics. This author has little experience with polyspectra and will not discuss them here. It appears that they can be useful as a descriptive tool for the analyst interested in frequency domain characteristics of data, but they can be difficult to interpret and may not be best suited for use in tests of linearity. 11.2 Some Models with Non-Linear Structure This section introduces some stochastic time-series models with a non-linear structure. Attention is restricted to some important classes of models that are of particular theoretical and practical importance. 11.2.1 Non-linear autoregressive processes An obvious way to generalize the (linear) AR model of order p is to assume that (11.1) where f is some non-linear function and Zt denotes a purely random process. This is called a non-linear autoregressive (NLAR) model of order p. Note that the ‘error’ term is assumed to be additive. A more general error structure is possible, perhaps with Zt incorporated inside the ƒ function, but will not be considered here. For simplicity consider the case p=1. Then we can rewrite Equation (11.1) as (11.2) where

is some non-constant function. It can be shown that a sufficient condition for Equation (11.2) to

describe a stable model is that model such as

must satisfy the constraint that | (x)|

面/The analysis of time series an introduction/files/page_222.html [5/24/2009 16:53:29]

page_223

< previous page

page_223

next page >

Page 223 (unusual) or stochastically or determined in some way by past data. Considering the stochastic option first, and taking the first-order case as an example, we could let (11.3)

where

(11.4) and γ, β are constants with {εt} being an i.i.d. sequence independent of the {Zt} sequence. Thus the parameter of the AR(1) process for Xt itself follows an AR(1) process. Such models are called time-varying parameter models (e.g. see Nicholls and Pagan, 1985). In the case when β=0 in Equation (11.4), the model reduces to what is sometimes called a random coefficient model. While such models appear intuitively plausible at first sight—we all know the world is changing—they can be tricky to handle and it is hard to distinguish between the constant and time-varying parameter cases. It is also not immediately obvious if the model is linear or non-linear. The model may appear to be linear in Equation (11.3), but appears non-linear if Equations (11.3) and (11.4) are combined, with β=0 for simplicity, as The last term is non-linear. The ‘best’ point forecast at time t is

. This looks linear at first sight, but of

course will involve past data leading to a non-linear function (though a similar situation arises for the constant-parameter AR model when the parameter(s) have to be estimated from the data). A more convincing argument for non-linearity is that, when Equation (11.3) is expressed as a state-space model (as in Equation 10.2), it is not a linear model since ht=at changes stochastically through time. Moreover it can be shown that the width of prediction intervals depends in a non-linear way on the latest value of the series. Another general possibility arises if we assume that the function f in Equation (11.1) is piecewise linear. This means that it consists of two or more linear functions defined over different regions of the lagged values of Xt. Consider, for example, the NLAR model of order 1, namely (11.5) If we assume that f is piecewise linear, then the parameters of f could, for example, depend on whether the value of Xt−1 is larger or smaller than a critical value, customarily called a threshold. Thus we effectively allow the model parameters to be determined by past data. This leads to the idea of a threshold AR model, which will be considered in the next subsection. 11.2.2 Threshold autoregressive models Following from Equation (11.5), a threshold AR model is a piecewise linear model where the parameters of an AR model are determined by the values

< previous page

file:///C:/Documents and Settings/Yang/

page_223

next page >

面/The analysis of time series an introduction/files/page_223.html [5/24/2009 16:53:30]

page_224

< previous page

page_224

next page >

Page 224 taken by one or more of the lagged values of the time series. Consider, for example, the model (11.6) where α(1), α(2), rare constants and {Zt} denotes SWN. In one sense, this is an AR(1) model, but the AR parameter depends on whether Xt−1 exceeds the value r called the threshold. Below r the AR parameter is α(1), but above r it is α(2). This feature makes the model non-linear and it is an example of a large class of models called threshold autoregressive (TAR) models. The above model can readily be extended to higher order autoregressions and to more than two thresholds depending on the values of one or more past data values. Tong (1990) calls a TAR model self-exciting (and uses the abbreviation SETAR) when the choice from the various sets of possible parameter values is determined by just one of the past values, say Xt−d where d is the delay. In Equation (11.6), the choice is determined by the value of Xt−1 and so the model is indeed self-exciting. Some theory for threshold models is given by Tong (1990). Threshold models are piecewise linear in that they are linear in a particular subset of the sample space, and can be thought of as providing a piecewise linear approximation to some general non-linear function as in Equation (11.1). Threshold models sometimes give rise to periodic behaviour with a limit cycle and it can be fruitful to demonstrate such a property by plotting Xt against Xt−1, or more generally by plotting Xt against Xt−k. Such a graph is sometimes called a phase diagram (actually a discrete-time version of such a diagram). Such graphs are useful, not just for identifying TAR models, but more generally in assessing the general form of a lagged relationship, particularly whether it is linear or non-linear. Estimation and forecasting for TAR models are, perhaps inevitably, rather more difficult than for linear AR models and will not be covered here. Estimation usually involves some sort of iterative procedure, while the difficulties involved in forecasting can be illustrated for the first-order model in Equation (11.6). Suppose we have data up to time N and that xN happens to be larger than the threshold r. Then it is obvious from the model that , and so the one-step-ahead forecast is easy to find. However, finding the two-steps-ahead forecast for the above model is much more complicated, as it will depend on whether the next observation happens to exceed the threshold. The expectation that arises is algebraically intractable and some sort of integration or approximation will need to be made to evaluate expectations over future error terms and the corresponding thresholds. An interesting application of threshold models was made by Chappel et al. (1996) in regard to exchange rates within the European Union. These rates are supposed to stay within prescribed bounds. A threshold model led to improved forecasts as compared with a random walk model. A second example, using

< previous page

file:///C:/Documents and Settings/Yang/

page_224

next page >

面/The analysis of time series an introduction/files/page_224.html [5/24/2009 16:53:31]

page_225

< previous page

page_225

next page >

Page 225 economic data, is given by Tiao and Tsay (1994). Although the latter authors found little improvement in forecasts using a threshold model, the modelling process required to fit a non-linear model led to greater insight into economic relationships, particularly that the economy behaves in a different way when it is going into, or coming out of, recession. Tsay (1998) has extended threshold models to the multivariate case, and gives details on testing, estimation and modelling together with some examples. A key feature of TAR models is the discontinuous nature of the AR relationship as the threshold is passed. Those who believe that nature is generally continuous may prefer an alternative model such as the smooth threshold autoregressive (STAR) model where there is a smooth continuous transition from one linear AR model to another, rather than a sudden jump. For example, consider the model

where I is a smooth function with sigmoid characteristics. One example is the logistic function which depends on two parameters, namely, r, which is comparable to the threshold, and δ, which controls the speed of the switch from one model to the other. Of course, a STAR model reduces to a simple threshold model by choosing I to be an indicator function taking the value zero below a threshold and one above it. Further details and references may be found in Teräsvirta (1994), but note that he uses the T in STAR to denote the word transition rather than threshold. 11.2.3 Bilinear models A class of non-linear models, called the bilinear class, may be regarded as a plausible non-linear extension of the ARMA model, rather than of the AR model. Bilinear models incorporate cross-product terms involving lagged values of the time series and of the innovation process. The model may also incorporate ordinary AR and MA terms. Denoting the time series by {Xt} and the innovation process by {Zt}, a simple example could be (11.7) where a and β are constants. As well as the innovation term Zt, this model includes one AR term plus one cross-product term involving Zt−1 and Xt−1. It is this cross-product term that is the non-linear term and makes this a bilinear model. One natural way for such a model to arise is to consider an AR(1) model where the AR parameter is not constant but is itself subject to innovations. If the perturbations in the AR parameter are such that the model for Xt is

< previous page

file:///C:/Documents and Settings/Yang/

page_225

next page >

面/The analysis of time series an introduction/files/page_225.html [5/24/2009 16:53:31]

page_226

< previous page

page_226

next page >

Page 226 given by (11.8) then this leads to Equation (11.7). A bilinear model could appear to be uncorrelated white noise (UWN) when examined in the usual way by inspecting, and perhaps testing, the sample ac.f. For example, consider the model (11.9) It can be shown (after some horrid algebra) that ρ(k)=0 for all k≠0 Thus, the series has the second-order properties of UWN, even though it has a clear structure. If, instead, we examine the series , then its ac.f. turns out to be of similar form to that of an ARMA(2,1) model. Thus Equation (11.9) is certainly not SWN. This example re-emphasizes the earlier remarks that there is no point in looking at second-order properties of {Xt} and hoping they will indicate any non-linearity, because they won’t! Rather, the search for non-linearity must rely on specially tailored procedures. As noted earlier, one general approach is to look at the properties of the series. If both {Xt} and appear to be UWN, then {Xt} can reasonably be treated as SWN. Bilinear models are interesting theoretically but are perhaps not particularly helpful in providing insight into the underlying generating mechanism and my impression is that they have been little used in practice. They have, for example, been found to give a good fit to the famous sunspots data (see Figure 11.1), but subsequent analysis revealed that they gave poor long-term forecasts. The forecasts diverged and were not able to predict the periodic behaviour observed, because bilinear models are not designed to reflect such behaviour. 11.2.4 Some other models Several other classes of non-linear models have been introduced in the literature and we refer briefly to two of them. State-dependent models are described by Priestley (1988). They include threshold and bilinear models as special cases and can be thought of as a locally linear ARMA model. We say no more about them here. A second class of non-linear models, called regime-switching models, has been widely applied in econometrics. The key feature is that the generating mechanism is different at different points in time and may be non-linear. When the model changes, it is said to switch between regimes. The time points at which the regime changes may be known in advance, or the regimes may change according to a Markov process. Alternatively, as with SETAR models, the change-points could be partly or wholly determined by past data. The models are usually multivariate (unlike SETAR models) and may be partly deterministic as well as nonlinear. A simple illustration is provided by the

< previous page

file:///C:/Documents and Settings/Yang/

page_226

next page >

面/The analysis of time series an introduction/files/page_226.html [5/24/2009 16:53:32]

page_227

< previous page

page_227

next page >

Page 227 following model for sales at time t in terms of demand and supply, namely Here the two regimes are (1) demand exceeds supply; and (2) supply exceeds demand. While the above equation is deterministic, as well as non-linear, separate equations are needed to model demand and supply and these are likely to be stochastic. The reader is referred to Harvey (1993, Section 8.6) for an introduction to these models. In fact, although it is often convenient to treat regime-switching models as a separate class of models, they can be written as special cases of the general threshold model (see Tong, 1990) using indicator variables. 11.3 Models for Changing Variance

Figure 11.2 A time plot of a financial time series exhibiting periods of excessive volatility. (Note: The units of price are excluded for confidentiality.) The non-linear models introduced in the previous section could be described as having structural nonlinearity. They allow improved point forecasts of the observed variable to be made when the true model is known. This section considers various classes of non-linear models of a completely different type, which are primarily concerned with modelling changes in variance. They do not generally lead to better point forecasts of the measured variable, but may lead to better estimates of the (local) variance. This, in turn, allows more reliable prediction intervals to be computed and hence a better assessment of risk. This can be especially important when modelling financial time series, where there is often clear evidence of changing variance in the time plot of

< previous page

file:///C:/Documents and Settings/Yang/

page_227

next page >

面/The analysis of time series an introduction/files/page_227.html [5/24/2009 16:53:33]

page_228

< previous page

page_228

next page >

Page 228 the data—see, for example, Figure 11.2. In this context, it is customary to talk about the volatility of the series, rather than the variance, and to look, in particular, for clusters of observations with high volatility. Suppose we have a time series from which any trend and seasonal effects have been removed and from which linear (short-term correlation) effects may also have been removed. We denote this derived series by {Yt}, to distinguish it from the original observed series, {Xt}. Thus {Yt} could, for example, be the series of residuals from a regression or AR model. Alternatively, Yt might be the first differences of a financial time series such as the natural log of a share price, and in this context is often called the returns series. In this sort of application the random walk model is often used as a first approximation for the (undifferenced) series so that the first differences are (approximately) uncorrelated. However, the variance of such a series is often found to vary through time, rather than to be constant. We may represent all such derived series having mean zero in the form (11.10) where {εt} denotes a sequence of i.i.d. random variables with zero mean and unit variance, and σt may be thought of as the local conditional standard deviation of the process. The εt may have a normal distribution but this assumption is not necessary for much of what follows. In any case the unconditional distribution of Yt generated by a non-linear model will not generally be normal but rather fat-tailed (or leptokurtic). Suppose we additionally assume that the square of at depends on the most recent value of the derived series by (11.11) where the parameters γ and α are non-negative to ensure that is non-negative. A model for Yt satisfying Equations (11.10) and (11.11) is called an autoregressive conditionally heteroscedastic model of order 1 (ARCH(1)). The adjective ‘autoregressive’ arises because the value of depends on past values of the derived series, albeit in squared form. Note that Equation (11.11) does not include an ‘error’ term and so does not define a stochastic process. More generally an ARCH(p) model arises when the variance depends on the last p squared values of the derived time series. It can be shown that ARCH models are martingale differences (MDs), so that knowledge of the value of at does not lead to improved point forecasts of Yt+h. Moreover the unconditional variance of Yt is (usually). constant, so that {Yt} behaves like UWN, even though the conditional variance of Yt does change through time. The value of modelling {at} lies in getting more reliable bounds for prediction intervals for Yt+h and in assessing risk more generally. When the derived series {Yt} has mean zero, the point forecast of Yt+1 is zero and prediction intervals are typically calculated using the appropriate percentage point of the standard normal distribution, even when there are doubts about the normality assumption (though alternative heavy-tailed distributions could be used). Thus, the onestep-ahead 100(1−α′)% prediction interval for Yt+1 is

< previous page

file:///C:/Documents and Settings/Yang/

page_228

next page >

面/The analysis of time series an introduction/files/page_228.html [5/24/2009 16:53:33]

page_229

< previous page

page_229

next page >

Page 229 typically taken to be of the form where zα′/2 denotes thevalue of the standard normal distribution for which the probability of being exceeded is α′/2. While ARCH models are UWN, it is not, of course, the case that they are SWN. For example, if {Yt} is ARCH (1), then it can be shown that has the same form of ac.f. as an AR(1) model. Once again, the analyst cannot rely on the usual second-order tests to indicate non-linearity. The ARCH model has been generalized to allow the variance to depend on past values of

as well as on

past values of . A derived variable satisfying Equation (11.10) is said to follow a generalised ARCH (or GARCH) model of order (p, q) when the local conditional variance is given by

(11.12) where γ≥0 and αi, βj≥0 for all i, j. GARCH models are also MDs and have a constant finite variance provided that the sum of ∑αi and ∑βj is less than unity. Identifying an appropriate GARCH model is not easy, and many analysts assume GARCH(1, 1) as the ‘standard’ model. A GARCH model behaves like UWN from its second-order properties, but, like ARCH models, is of course not SWN. Once again the non-linearity has to be demonstrated by examining the properties of , which, for a GARCH(1, 1) model, can be shown to have an ac.f. of the same form as an ARMA(1, 1) process. The use of ARCH and GARCH models does not affect point forecasts of the original observed variable, and it is therefore rather difficult to make a fair comparison of the forecasting abilities of different models for changing variance. Thus the modelling aspect (understanding the changing structure of a series), and the assessment of risk, are both more important than their ability to make point forecasts. GARCH models have typically been used in forecasting the price of derivatives such as options (e.g. the right to buy a certain share at a pre-specified time in the future) where estimation of variance is important. Also note that failure to account for changing variance can lead to biases in tests on other properties of the series. Further information about ARCH and GARCH models is given by Bollerslev et al. (1992, 1994), Enders (1995, Chapter 3), Franses (1998, Chapter 7), Gourieroux (1997) and Shephard (1996). The reader will notice that the formulae for in both ARCH and GARCH models (Equations (11.11) and (11.12), respectively) are essentially deterministic in that there is no ‘error’ term in either equation. An alternative to ARCH or GARCH models is to assume that σt in Equation (11.10) follows a stochastic process. This is usually done by modelling the logarithm of

or of σt to ensure that

remains positive. A simple

example is to assume that , say, follows an AR process with an ‘error’ component that is independent of the {εt} series in Equation (11.10). Models of this type are called stochastic volatility or stochastic variance models. Although the likelihood function is more difficult to handle, the model ties in more naturally

< previous page

file:///C:/Documents and Settings/Yang/

page_229

next page >

面/The analysis of time series an introduction/files/page_229.html [5/24/2009 16:53:34]

page_230

< previous page

page_230

next page >

Page 230 with other finance models and is easier to generalize to the multivariate case. Moreover it seems intuitively more reasonable to assume that σt changes stochastically through time rather than deterministically, especially when one sees the sudden changes in volatility that can occur in the financial market as a result of a special event like a war involving oil-producing countries. More details may be found in Harvey (1993, Section 8.4). Taylor (1994) suggests that a judicious combination of both ARCH and stochastic volatility models may provide more satisfactory results than a single model. Several other classes of models, which allow conditional volatility, have also been proposed. They include further varieties of ARCH models, a class of models called flexible Fourier forms, as well as neural network models, which will be introduced in Section 11.4. It is difficult to choose between different models for changing variance using the data alone and so it is advisable, as usual, to use the context and any background theory, to supplement the results of the exploratory analysis of the data. 11.4 Neural Networks Neural networks (NNs) provide the basis for an entirely different non-linear approach to the analysis of time series. NNs originated in attempts at mathematical modelling of the way that the human brain works, but this connection is rather tenuous and probably not very helpful. Thus an NN is sometimes called an artificial NN (ANN) to emphasize that it is a mathematical model. NNs have been applied to a wide variety of mathematical and statistical problems, many of which have little or no relation with time-series analysis. For example, NNs have been widely used in pattern recognition, where applications include the automatic reading of handwriting and the recognition of acoustic and visual facial features corresponding to speech sounds. Some of these applications have been very successful and the topic has become a rapidly expanding research area. In recent years, NNs have also been applied in timeseries analysis and forecasting and we naturally concentrate on these applications. Recent reviews from a statistical perspective include Faraway and Chatfield (1998), Stern (1996), and Warner and Misra (1996), while the introductory chapter in Weigend and Gershenfeld (1994) presents a computer scientist’s perspective on the use of NNs in time-series analysis. A neural net can be thought of as a system connecting a set of inputs to a set of outputs in a possibly nonlinear way. The connections between inputs and outputs are typically made via one or more hidden layers of neurons, sometimes alternatively called processing units or nodes. Figure 11.3 shows a simple example of an NN with three inputs, one hidden layer containing two nodes, and one output. The arrows indicate the direction of each relationship and the NN illustrated is typical in that there are no connections between units in the same layer and no feedback. This NN may therefore be described

< previous page

file:///C:/Documents and Settings/Yang/

page_230

next page >

面/The analysis of time series an introduction/files/page_230.html [5/24/2009 16:53:35]

page_231

< previous page

page_231

next page >

Page 231 as being of a feed-forward design, and NNs are generally assumed to have this structure unless otherwise stated.

Figure 11.3 An example of a neural network applied to time-series data with three inputs, one hidden layer of two neurons, and one output. The three inputs are a constant and lagged values at times (t−1) and (t−4). The output is the forecast at time t. The structure, or architecture, of an NN has to be determined by the analyst. This includes determining the number of layers, the number of neurons in each layer and which variables to choose as inputs and outputs. In Figure 11.3, the architecture is chosen to forecast the value of a time series at time t (the output) using lagged values at time (t−1) and (t−4) together with a constant (the three inputs). The use of values at lags one and four would be natural when trying to forecast quarterly data. The number of layers is often taken to be one, while the number of hidden neurons is often found by trial and error using the data. Thus the architecture is chosen in general by using the context and the properties of the given data. How then do we compute the output of an NN from the inputs, given the structure of the network? In general we denote the m input variables by x1, x2,…, xm, one of which will usually be a constant. We further assume there are H neurons in one hidden layer. We then attach the weight wij to the connection between input xi and the jth neuron in the hidden level. These weights effectively measure the ‘strength’ of the different connections and are parameters that need to be estimated from the given data, as described below. Given values for the weights, the value to be attached to each neuron may then be found in two stages. First, a linear function of the inputs is found, say

for j=1, 2,…, H. Second, the quantity υj is converted to the final value for the jth neuron, say Zj, by applying a function, called an activation function, which has to be selected by the analyst. This function could be linear, but is more usually a non-linear sigmoid transformation such as the logistic function,

< previous page

file:///C:/Documents and Settings/Yang/

page_231

next page >

面/The analysis of time series an introduction/files/page_231.html [5/24/2009 16:53:36]

page_232

page_232

< previous page

next page >

Page 232 , or the hyperbolic tangent, . It is also possible to choose a discontinuous non-linear activation function such as the indicator function, which takes the value one when υj exceeds a threshold value and zero otherwise. Having calculated values for each neuron, a similar pair of operations can then be used to get the predicted value for the output using the values at the H neurons. This requires a further set of weights, say w′j, for j=1, 2,…, H, to be attached to the links between the neurons and the output, and also requires an appropriate activation function to be selected for this new stage. If there is a direct link between the constant input and the output, as in Figure 11.3, then we also need a weight for this connection, say output, y say, is related to the inputs by the rather complicated-looking expression

where

and

. Overall the

denote the activation functions at the output and hidden layers, respectively. In many

applications of NNs, and are often chosen to have the same form, but in time-series forecasting this could be disastrous. The logistic function, for example, always gives a number between 0 and 1, and so this would only work for data that are scaled to lie between 0 and 1. Thus, in time-series forecasting, is often chosen to be the identity function so that the operation at the output stage remains linear. Of course, all the above operations have to be carried out for every time t, but we have simplified the presentation by omitting the subscript t that should really be applied to all values of the inputs xi, to the neuron values vj and Zj, and to the output y. The above exposition can be generalized in obvious ways to handle more than one hidden layer of neurons and more than one output. Overall, an NN can be likened to a sort of non-linear regression model. Note that the introduction of a constant input ‘variable’, connected to every neuron in the hidden layer and also to the output as in Figure 11.3, avoids the necessity of separately introducing what computer scientists call a bias, and what statisticians would call an intercept term, for each relation. Essentially the ‘biases’ are replaced by the relevant weights, which become part of the overall set of weights (the model parameters) that can all be estimated in the same way. How is this model fitting to be done? In time-series analysis, the weights are usually estimated from the data by minimizing the sum of squares of the within-sample one-step-ahead forecast errors, namely, This is done over a suitable portion of the data, so as to get a good fit. Here we assume the output, y from the NN is the one-step-ahead forecast . Choosing the weights so as to minimize S is no easy task, and is a non-linear optimization problem. It is sound practice to divide the data into two sections, to fit the NN model to the first part of the data, called the training set, but

< previous page

file:///C:/Documents and Settings/Yang/

page_232

next page >

面/The analysis of time series an introduction/files/page_232.html [5/24/2009 16:53:37]

page_233

< previous page

page_233

next page >

Page 233 to hold back the last part of the data, called the test set, so that genuine out-of-sample forecasts can be made and compared with the actual observations. This gives an independent check on the model’s predictive ability. Various fitting algorithms have been proposed for NN models, and many specialized packages are now available to implement them. A technique called back-propagation is commonly used, though other algorithms exist that may be more efficient. Details will not be given here—see, for example, Bishop (1995). The NN literature typically describes the iterative estimation procedure as being a ‘training’ algorithm that ‘learns by trial and error’, and this is just one example of how the NN literature often uses different jargon from that used by statisticians. Unfortunately, some procedures may take several thousand iterations to converge, and yet may still converge to a local minimum. This is partly because of the non-linear nature of the objective function, and partly because there are typically much larger numbers of parameters to estimate than in traditional time-series models. For example, the relatively simple architecture in Figure 11.3 still involves nine connections and hence has nine parameters (weights). The large number of parameters means there is a real danger that model fitting will ‘overtrain’ the data and produce a spuriously good fit that does not lead to better forecasts. This motivates the use of model comparison criteria, such as Akaike’s information criterion (AIC) (see Section 13.1), which penalizes the addition of extra parameters when comparing different NN architectures. It also motivates the use of an alternative fitting technique called regularization (e.g. Bishop, 1995, Section 9.2) wherein the ‘error function’ is modified to include a penalty term, which prefers ‘small’ parameter values. In order to start the iterative procedure, the analyst must select starting values for the weights, and this choice can be crucial. It is advisable to try several different sets of starting values to see if consistent results are obtained. Research is continuing on ways of fitting NN models, and more specialist software is becoming available, as well as NN macros for general-purpose packages such as SAS and S-PLUS. As well as the choice of software to fit an NN, we have seen that the analyst must also consider various other questions, such as how to choose the training set, what architecture to use and what activation function to apply. The approach is non-parametric in character in that little subject-domain knowledge is used in the modelling process (except in the choice of which input variables to include), and there is no attempt to model the ‘error’ component. When applied to forecasting, the whole process can be completely automated on a computer, which may be seen as an advantage or a disadvantage. The use of NNs then has the character of a black-box approach where a particular model is selected from a large class of models in a mechanistic way using little or no subjective skill and giving little understanding of the underlying mechanisms. A problem with black boxes is that they can sometimes give silly results and NNs are no exception—see below. My experience (Faraway and Chatfield, 1998) suggests that black-box modelling is generally unwise, but rather that a good NN model for time-series data must be selected by combining traditional

< previous page

file:///C:/Documents and Settings/Yang/

page_233

next page >

面/The analysis of time series an introduction/files/page_233.html [5/24/2009 16:53:37]

page_234

< previous page

page_234

next page >

Page 234 modelling skills with knowledge of time-series analysis and of the particular problems involved in fitting NN models. Model building will, as always (see Section 4.8), take account of the context and the properties of the data. The empirical evidence in regard to the forecasting ability of NNs is mixed (see Faraway and Chatfield, 1998; Zhang et al., 1998). There are, in fact, difficulties in making a fair comparison between the use of NNs and of alternative time-series forecasting methods, since few statisticians know much about NNs, while computer scientists may know little about statistical alternatives. Moreover, measures of forecast accuracy, like MSE, are really intended for comparing linear models, while the importance of ensuring that forecasts are genuinely ‘out-of-sample’ is not always appreciated. More complicated models usually do better within sample, but this proves nothing. Clearly there are exciting opportunities for collaborative work between statisticians and other scientists. One important empirical study was the so-called Santa Fe competition where six series were analysed (Weigend and Gershenfeld, 1994). The series were very long compared with most time series that need to be forecasted (e.g. 34,000 observations) and five were clearly non-linear when their time plots were inspected. There was only one economic series. The organizers kept holdout samples for three of the series. The participants in the competition tried to produce the ‘best’ forecasts of the holdout samples using whichever method they preferred, though it is worth noting that little contextual information was provided for them. The results showed that the better NN forecasts did comparatively well for some series, but that some of the worst forecasts were also produced by NNs when applied in black-box mode without using some sort of initial data analysis before trying to fit an appropriate NN model. In particular, predictions “based solely on visually examining and extrapolating the training data did much worse than the best techniques, but also much better than the worst”. The results also showed that there are “unprecedented opportunities to go astray”. For the one economic series on exchange rates, there was a “crucial difference between training set and test set performance “and “out-of-sample predictions are on average worse than chance”. In other words, better forecasts could have been obtained with the random walk. This is disappointing to say the least! Other empirical evidence is less clear-cut. In subject areas where NNs have been applied successfully, there are often several thousand observations available for fitting, and the series may exhibit clear non-linear characteristics (as for some of the Santa Fe series). The evidence for shorter series is much less convincing, especially as researchers tend to publish results when new methods do better, but not otherwise. An exception is Racine (2001) who failed to replicate earlier results showing good NN forecasts for stock returns, but rather found linear regression more accurate. Recent applications in economic and sales forecasting have sometimes tried to use as few as 150 observations, and this now seems generally unwise. For many economic and financial series, a random walk forecast of no change is often better than an NN forecast.

< previous page

file:///C:/Documents and Settings/Yang/

page_234

next page >

面/The analysis of time series an introduction/files/page_234.html [5/24/2009 16:53:38]

page_235

< previous page

page_235

next page >

Page 235 Simulations show linear methods do better than NNs for data generated by a linear mechanism, as one would intuitively expect. While it may be too early to make a definitive assessment of the forecasting ability of NNs, it is clear that they are not the universal panacea that some advocates have suggested. Although they can be valuable for long series with clear non-linear characteristics, it appears that the analyst needs several hundred, and preferably several thousand, observations to be able to fit an NN with confidence. Even then, the resulting model is usually hard to interpret, and there is plenty of scope for going badly wrong during the modelling process. For the sort of short time series typically available in sales, economic and financial forecasting, there is rarely enough data to reliably fit an NN and I cannot recommend their use. 11.5 Chaos The topic of chaos has attracted much attention in recent years, especially from applied mathematicians. Chaotic behaviour arises from certain types of non-linear models, and a loose definition is ‘apparently random behaviour that is generated by a purely deterministic, non-linear system’. A non-technical overview is given by Gleick (1987) while Kantz and Schreiber (1997) provide a readable introduction from the perspective of mathematical physics. Chan and Tong (2001) and Isham (1993) provide a statistical perspective, while further helpful material is given by Tong (1990, Chapter 2). If a chaotic deterministic system can appear to behave as if it were ‘random’, the statistician then has the task of deciding what is meant by random and further has to try to decide whether an apparently random time series has been generated by a stochastic model, by a chaotic (non-linear) deterministic model or by some combination of the two. It would be of great interest to scientists if fluctuations, previously thought to be random, turned out to have a deterministic explanation. Unfortunately, distinguishing between the different possibilities can be difficult in practice The main idea is well illustrated by the famous example of chaos called the logistic map, sometimes alternatively called the quadratic map. This is an example of what mathematicians would call a difference equation or map, but which statisticians would probably regard as a deterministic time series. Suppose a time series is generated by the (deterministic) equation for t=1, 2, 3,… with x0 (0,1). Then, provided 0

面/The analysis of time series an introduction/files/page_237.html [5/24/2009 16:53:40]

page_238

< previous page

page_238

next page >

Page 238 BDS test (Granger and Teräsvirta, 1993, Chapter 6; Brock and Potter, 1993) is often used to test the null hypothesis that a series is (linear) i.i.d. against a (non-linear) chaotic alternative, but it is not clear when this test is superior to other tests for non-linearity. More generally the extent to which a non-linear deterministic process retains its properties when corrupted by noise is also unclear. It is, however, worth noting that classical (linear) methods for smoothing series, which are designed to separate the signal from the noise, can actually make things worse for chaotic series (Kantz and Schreiber, 1997, Example 1.1). Work continues on these difficult questions. Sadly, it does appear to be the case that very large samples are needed to identify attractors in high-dimensional chaos. A few years ago, there were high hopes that the use of chaotic models might lead to improved economic forecasts. Sadly, this has not yet occurred. For example, Granger (1992) says that it seems unlikely that the stock market could obey a simple deterministic model, while Granger and Teräsvirta (1993, p. 36) and Brock and Potter (1993) both say that there is strong evidence for nonlinearity in economic data but weak evidence that they are also chaotic. On the other hand, May (1987) has argued that chaos is likely to be pervasive in biology and genetics. My current viewpoint is that the analyst will generally find it difficult to apply models with chaotic properties to real time-series data, but research is continuing (e.g. Tong, 1995) and the position may well change in the future. In any case, the study of chaotic models is fascinating and may contribute to our understanding of random behaviour in time-series modelling. It may even cause us to re-examine the meaning of the word ‘random’. Even if a system under study is regarded in principle as deterministic, the presence of chaotic effects with unknown initial conditions (as will usually be the case in practice) means that prediction becomes difficult or impossible. Moreover a system may appear deterministic at the microscopic level, but appear stochastic at the macroscopic level. Putting this in a different way, it can be argued that whether a system is ‘random’ depends not on its intrinsic properties, but on how it appears to an observer in the given context with available knowledge. These are deep waters!! 11.6 Concluding Remarks The real world generally changes through time and often behaves in a non-linear way. Thus alternatives to linear models with constant parameters should often be considered. There are several ways that the need for a non-linear model may be indicated, namely • Looking at the time plot and noting asymmetry, changing variance, etc. • Plotting xt against xt−1, or more generally against xt−k for k=1, 2,…, and looking for strange attractors, limit cycles, etc. • Looking at the properties of { } as well as at those of {xt} • Taking account of context, background knowledge, known theory, etc.

< previous page

file:///C:/Documents and Settings/Yang/

page_238

next page >

面/The analysis of time series an introduction/files/page_238.html [5/24/2009 16:53:41]

page_239

< previous page

page_239

next page >

Page 239 Non-linear models often behave quite differently to linear models. In general, they are harder to handle and require considerable technical and numerical skill. In particular, it is generally much more difficult to construct forecasts from a non-linear, rather than linear, model, for more than one step ahead. In the latter case, analytic formulae are generally available, but non-linear forecasting becomes increasingly difficult for longer lead times where some sort of numerical approach will generally be needed to calculate conditional expectations. Fortunately, specialist computer packages, such as STAR (see Tong, 1990, p. xv), are becoming available to do this. Two general references on non-linear forecasting are Lin and Granger (1994) and Chatfield (2001, Section 4.2.4). An additional feature of non-linear models, which may be unexpected at first, is that the width of prediction intervals need not increase with the lead time. This may happen, for example, for data exhibiting multiplicative seasonality, where prediction intervals tend to be narrower near a seasonal trough rather than near a seasonal peak. Another peculiarity of non-linear models, even with normal errors, is that the distribution of the forecast error is not in general normal, and may even be bimodal or have some other unexpected shape. In the bimodal case, a sensible prediction interval may comprise, not a single interval, but two disjoint intervals. This seems most peculiar at first sight, but second thoughts remind us of situations where we might expect a high or low outcome but not an intermediate result. Sales of a new fashion commodity could be like this. In such circumstances it could be particularly misleading to give a single point forecast by calculating the conditional expectation. It is difficult to give advice on how to choose an appropriate non-linear model. As always contextual information and background knowledge are vital, but the only statistical advice I can give is that periodic behaviour suggests trying a threshold model, while, with a very long series, some analysts believe that it may be worth trying a neural net. It is also worth remembering that alternatives to linear models include, not only non-linear models but also models where the parameters change through time in a pre-determined way and models that allow a sudden change in structure. While the latter may be regarded as non-linear, it may be more helpful to think of them as non-stationary (see Section 13.2). Non-linear models are mathematically interesting and sometimes work well in practice. The fitting procedure is more complicated than for linear models, but may lead to greater insight, and it should be remembered that the prime motivation for modelling is often to improve understanding of the underlying mechanism. This being so, it would be a bonus if non-linear models gave better forecasts as well. Sadly, it appears from the literature that gains in forecasting accuracy are often (usually?) modest if they exist at all, and may not by themselves compensate for the additional effort required to compute them. These results are reviewed by Chatfield (2001, Sections 3.4 and 6.4).

< previous page

file:///C:/Documents and Settings/Yang/

page_239

next page >

面/The analysis of time series an introduction/files/page_239.html [5/24/2009 16:53:42]

page_240

< previous page

page_240

next page >

Page 240 Thus even when the data appear to exhibit clear non-linear properties, it may still be safer to use a linear model for forecasting purposes. 11.7 Bibliography The literature on non-linear models is growing rapidly and many references have been given throughout this chapter. Alternative introductions are given by Franses (1998, Chapter 8), Granger and Newbold (1986, Chapter 10) and Harvey (1993, Chapter 8). More detailed accounts are given by Priestley (1981, Chapter 11; 1988) and by Tong (1990). The advanced text by Granger and Teräsvirta (1993) is concerned with economic relationships and extends discussion to multivariate non-linear models. Exercise It is difficult to set exercises on non-linear models that are mathematically and practically tractable. The reader may like to try the following exercise on the logistic map. 11.1 Consider the logistic map with k=4, namely This series is non-linear, deterministic and chaotic. Given the starting value x0=0.1, evaluate the first three terms of the series. Change x0 to 0.11 and repeat the calculations. Show that x3 changes from 0.289 to 0.179 (to 3 decimal places). Thus a change of 0.01 in the starting value leads to a change of 0.110 in the value of x3 and this provides a simple demonstration that a chaotic series is very sensitive to initial conditions.

< previous page

file:///C:/Documents and Settings/Yang/

page_240

next page >

面/The analysis of time series an introduction/files/page_240.html [5/24/2009 16:53:42]

page_241

< previous page

page_241

next page >

Page 241 CHAPTER 12 Multivariate Time-Series Modelling 12.1 Introduction Observations are often taken simultaneously on two or more time series. For example, in meteorology we might observe temperature, air pressure and rainfall at the same site for the same sequence of time points. In economics, many different measures of economic activity are typically recorded at regular intervals. Examples include the retail price index, the gross domestic product and the level of unemployment. Given multivariate data like these, it may be helpful to develop a multivariate model to describe the interrelationships among the series. Section 5.3.1 briefly discussed some multivariate forecasting procedures, focusing particularly on the use of multiple regression models. This chapter takes a more detailed look at some multivariate time-series models, giving particular attention to vector autoregressive (VAR) models. The enormous improvement in computing capability over recent years has made it much easier to fit a given multivariate model from a computational point of view. However, my experience suggests that the overall model-building process is still much more difficult for multivariate than univariate models. In particular, there are typically (far) more parameters to estimate than in the univariate case. Moreover, the pool of candidate models is much wider, and this can be seen as a strength—more possibilities—or as a weakness—it is harder to find the ‘right’ model. As regards the latter point, it appears that multivariate models are more vulnerable to misspecification than simpler univariate models, and this emphasizes the importance of getting sufficient background information so as to understand the context and identify all relevant explanatory variables before starting the modelling process. As always, it is vital to ask appropriate questions and formulate the problem carefully. An iterative approach to model building (see Section 4.8) is generally required, and the use to which the model will be put should be considered as well as the goodness of fit. There is always tension between seeking a parsimonious model (so that fewer parameters need to be estimated) while ensuring that important effects are not mistakenly omitted from the model. With multivariate time-series data, the modelling process is complicated by the need to model the serial dependence within each series, as well as the interdependence between series. Information about the latter is provided by the cross-correlation function, which was introduced in Section 8.1. However, as noted in Section 8.1.3, the interpretation of cross-correlations is difficult

< previous page

file:///C:/Documents and Settings/Yang/

page_241

next page >

面/The analysis of time series an introduction/files/page_241.html [5/24/2009 16:53:43]

page_242

< previous page

page_242

next page >

Page 242 and choosing appropriate candidate models is not easy. There also tend to be more problems with the data in the multivariate case. Having more variables to measure means that there is more chance of mistakes in the data. Moreover, multivariate data are sometimes unsuitable for fitting multivariate models, perhaps because some explanatory variables have been held more or less constant in the past. While univariate models can be very useful for describing short-term correlation effects, for forecasting large numbers of series, and as a benchmark in comparative forecasting studies, it is clear that multivariate models should also have much to offer in gaining a better understanding of the underlying structure of a given system and (hopefully) in getting better forecasts. Sadly, as noted in Section 5.4.2, the latter does not always happen. While multivariate models can usually be found that give a better fit than univariate models, there are a number of reasons why better forecasts need not necessarily result (though of course they sometimes do). We have seen that multivariate models, being more complicated, are generally more difficult to fit than univariate ones, while multivariate data may leave much to be desired. Furthermore, multivariate forecasts may require values of explanatory variables that are not yet available and so must themselves be forecast. If this cannot be done very accurately, then poor forecasts of the response variable may also result. Overall, the analyst should be prepared for the possibility that multivariate forecasts are not always as good as might be expected. 12.1.1 One equation or many? One basic question is whether the model should involve a single equation or several equations. In multiple regression, for example, the model explains the variation in a single response variable, say y, in terms of the variation in one or more predictor, or explanatory, variables, say x1, x2,…. This is done with the aid of a single equation. For a single-equation model to be appropriate, there must be only one response variable of interest (e.g. forecasts are only required for this one variable), and there should be no suggestion that the value of the response variable could itself affect the predictor variables. In other words, the regression equation assumes there is an open-loop system (see Figure 9.7). If the relationship between a single predictor variable and a single response variable can be modelled by a regression equation, some people would say that there is a causal relationship between the variables, though in practice it may be difficult to decide whether there is a direct link or if the link comes via relationships with a third, possibly unobserved, variable. A completely different situation arises when the ‘outputs’ affect the ‘inputs’ so that there is feedback in a closed-loop system (see Figure 9.8). For example, in economics, we know that a rise in prices will generally lead to a rise in wages, which will in turn lead to a further rise in prices. Then a regression model is not appropriate. Instead, a model with more than one equation will be needed to satisfactorily model the system.

< previous page

file:///C:/Documents and Settings/Yang/

page_242

next page >

面/The analysis of time series an introduction/files/page_242.html [5/24/2009 16:53:43]

page_243

< previous page

page_243

next page >

Page 243 One type of model with more than one equation is the econometric simultaneous equation model (see Section 5.3.2). This model comprises a number of equations, which need not be linear, and which are generally constructed using economic theory. By including relevant policy variables, they may be used to evaluate alternative economic strategies, as well as improving understanding of the system (the economy) and producing forecasts (although the latter may not be the prime objective). Models of this type are typically constructed by econometricians and so we do not attempt to describe the modelling process in this time-series text, as the problems are usually more of an economic nature than statistical. We will, however, make the following general remarks. We have already contrasted the differing viewpoints of econometricians and statisticians, while emphasizing the complementary nature of their skills. It is certainly true that economic models based on theory need to be validated with real data, but a statistical data-based approach to modelling the economy will not get very far by itself without some economic guidelines. This is because an unrestricted analysis will find it difficult to select a sensible model from a virtually infinite number of choices. However, we note the following three general points in regard to building models for economic data: 1. Economic data is naturally affected by feedback, and this always makes model building difficult. For a physical system, such as a chemical reactor, where the feedback is well-controlled, there may not be enough information in the available data to identify the structure of the system. However, this is not a major problem in that known perturbations can be superimposed on the system in order to see what effect they have. However, it is generally less easy to control an economy than something like a chemical reactor, and efforts to control the economy are often made in a fairly subjective way. As a result, the amount of information in economic data may be less than one would like. Furthermore, it is difficult to carry out experiments on the economy in the same sort of way that perturbations can be added to a physical system. 2. The economy has a complex, non-linear structure, which may well be changing through time, and yet data sets are often quite small. 3. Statistical inference is usually carried out conditional on an assumed model and focuses on uncertainty due to sampling variation and having to estimate model parameters. However, specification errors, arising from choosing the wrong model, are often more serious, particularly in economics—see Section 13.5. The main time-series alternative to econometric simultaneous equation models are VAR models and they will be covered in Section 12.3. 12.1.2 The cross-correlation function A key tool in modelling multivariate time-series data is the cross-correlation function, which was defined in Section 8.1 in the context of a (bivariate) linear

< previous page

file:///C:/Documents and Settings/Yang/

page_243

next page >

面/The analysis of time series an introduction/files/page_243.html [5/24/2009 16:53:44]

page_244

< previous page

page_244

next page >

Page 244 system. Before continuing, it may be helpful to redefine this function for an m-variate multivariate process, . Analogous to the univariate case, we begin by defining say {Xt}, where cross-covariances. Let µt denote the vector of mean values of Xt at time t, so that its ith component is µit=E(Xit). Let Γ(t, t+k) denote the cross-covariance matrix of Xt and Xt+k, so that its (i, j)th element is the cross-covariance coefficient of Xit and Xj,t+k. A multivariate process is said to be second-order stationary if the mean and the cross-covariance matrices at different lags do not depend on time. Then µt will be a constant, say µ, while Γ(t, t+k) will be a function of the lag k only, say Γ(k). Then the (i, j)th element of Γ(k), say γij(k), is given by (12.1) —see Equation (8.1). In the stationary case, the set of cross-covariance matrices, Γ(k) for k=0, ±1, ±2,…, is called the covariance matrix function. It has rather different properties to the (auto)covariance function in that it is not an even function of lag, but rather we find the (i, j)th element of Γ(k) equals the (j, i)th element of Γ(−k) so that Γ(k)=ΓT(−k) (though the diagonal terms, which are auto-rather than crosscovariances, do have the property of being an even function of lag). Given the covariance matrix function, it is easy to standardize any particular element of any matrix (by dividing by the product of the standard deviations of the two relevant series) to find the corresponding crosscorrelation and hence construct the set of (m×m) cross-correlation matrices, P(k) for k=0, ±1, ±2,…, called the correlation matrix function of the process. Thus the (i, j)th element of P(k) is given by (12.2) . where σi, the standard deviation of Xit, can also be expressed as Assuming that the same number of observations, say N, have been collected on the m variables over the same time period, the sample cross-covariances and cross-correlations may be calculated by a natural extension of the formulae given in Section 8.1.2. For example, the sample cross-covariance coefficient of Xi and Xj at lag k is given by

(12.3)

and the sample cross-correlation coefficient of Xi and Xj at lag k is given by (12.4) where

denotes the sample standard deviation of observations on the ith variable.

< previous page

file:///C:/Documents and Settings/Yang/

page_244

next page >

面/The analysis of time series an introduction/files/page_244.html [5/24/2009 16:53:45]

page_245

< previous page

page_245

next page >

Page 245 12.1.3 Initial data analysis In time-series analysis, the first step should normally be to plot the data. With multivariate data, this step is no less important. A time plot for each variable will indicate the presence of trend, seasonality, outliers and discontinuities. For stationary series, it will also be helpful to calculate the autocorrelation function (ac.f.) for each series in order to suggest an appropriate univariate model for each series. It may also be helpful to calculate the cross-correlation function for all meaningful pairs of variables, but the reader should refer back to Section 8.1.3, where the difficulties in interpreting cross-correlations are explained. In brief, crosscorrelation variances are inflated by autocorrelation within the series and so some prefiltering is usually desirable to avoid spuriously large values. With a good interactive graphics package, it can also be fruitful to scale all the series to have zero mean and unit variance, and then to plot pairs of variables on the same graph. One series can be moved backwards or forwards in time so as to make the visible characteristics of the series agree as closely as possible. If the cross-correlations between two series are generally negative around zero lag, then it may be necessary to turn one series ‘upside-down’ in order to get a good match. This type of approach may be helpful in alerting the analyst to the possible presence of linear relationships, perhaps with an in-built delay mechanism. However, the approach suffers from similar dangers as those involved in interpreting cross-correlations. With a large number of variables, it can be very difficult to build a good multivariate time-series model, especially when many of the cross-correlations between predictor variables are ‘large’. Then it may be fruitful to make some sort of multivariate transformation of the data (e.g. by using principal component analysis) so as to reduce the effective dimensionality of the data. This type of approach will not be considered here (see, for example, Peña and Box, 1987). 12.2 Single Equation Models Section 12.1.1 said that it may be appropriate to model multivariate time-series data with a single equation when (1) there is a single response variable and several explanatory variables, and (2) there is no feedback from the response variable to the explanatory variables. The most obvious type of model to use is a multiple regression model, but the difficulties in fitting such models to time-series data have already been discussed in Section 5.3.1. We extend that discussion by considering the following simple model, namely (12.5) where a, b, d are constants and et denotes an error term (which may be autocorrelated).

< previous page

file:///C:/Documents and Settings/Yang/

page_245

next page >

面/The analysis of time series an introduction/files/page_245.html [5/24/2009 16:53:45]

page_246

< previous page

page_246

next page >

Page 246 If d is an integer greater than zero in Equation (12.5), then Xt is said to be a leading indicator for Yt. Given data on X and Y until time t, this model enables forecasts of Yt to be made directly for up to d steps ahead. However, to forecast more than d steps ahead, the required value of Xt will probably not be available and must itself be forecasted. In this sort of situation, a multivariate model is only able to give ‘good’ forecasts when forecasts of explanatory variables can be made (much) more accurately than those of the response variable. Although Equation (12.5) appears to be a simple linear regression model at first sight, we said that the error terms may be autocorrelated and this makes the model non-standard. Furthermore, depending on the physical context, it may not be possible to control the values of Xt or there may even be feedback from Yt to Xt, although this is not always immediately apparent. For all these reasons, it is often safer to fit an alternative class of models, which may include Equation (12.5) as a special case. For an open-loop causal relationship between a single explanatory variable and a response variable, it is often worth trying to fit a member of the class of transfer function models. The latter were introduced in Section 9.4.2 and have the general form (12.6) where h(B)=h0+h1B+h2B2+… is a polynomial in the backward shift operator, B, d denotes a non-negative integer and Nt denotes noise (which may be autocorrelated). If d>0, then Xt is said to be a leading indicator for Yt. As noted in Section 9.4.2, Equation (12.6) can sometimes be parsimoniously rewritten in the form (12.7)

, while et=δ(B)Nt is where δ(B), (B) are low-order polynomials in B such that usually assumed to follow some sort of autoregressive moving average (ARMA) process. Note that Equation (12.7) may include lagged values of Yt as well as of Xt. Models of this type can be fitted using the tools summarized in Section 9.4.2 and are fully described by Box et al. (1994, Chapter 11). The dynamic regression models of Pankratz (1991) are of a somewhat similar type to those given above. Note that econometricians may refer to a model of the type described by Equation (12.6) as a distributed lag model, and when a polynomial lag function such as h(B) is written as a ratio of polynomials as in the transfer function model of Equation (12.7), then econometricians may use the term rational distributed lag. 12.3 Vector Autoregressive Models There are many situations where a single-equation model is inappropriate for multivariate time-series data. For example, there may be more than one

< previous page

file:///C:/Documents and Settings/Yang/

page_246

next page >

面/The analysis of time series an introduction/files/page_246.html [5/24/2009 16:53:46]

page_247

page_247

< previous page

next page >

Page 247 response variable of interest, or the data may have been generated by a closed-loop system. In the latter case, it no longer makes sense to talk about an ‘input’ (an explanatory variable) and an ‘output’ (a response variable). More generally, there are many situations where there are two (or more) variables that, to a greater or lesser extent, ‘arise on an equal footing’, and which are all interrelated. Modelling such variables is often called multiple time-series modelling, and this section introduces arguably the most important class of models for this purpose. With m variables, a natural way to represent them is by means of a (m×1) vector Xt where . For simplicity, we initially restrict attention to the case m=2. For stationary series, we may, without loss of generality, assume the variables have been scaled to have zero mean. In the latter case, a simple model would allow the values of X1t and X2t to depend linearly on the values of both series at time (t−1). The resulting model for the two series would then consist of two equations, namely (12.8) where { ij} are constants. The two ‘error’ terms ε1t and ε2t are usually both assumed to be white noise but are often allowed to be correlated contemporaneously. In other words, ε1t could be correlated with ε2t but not with past values of either ε1t or ε2t. Equation (12.8) can be rewritten in vector form as (12.9) where

and

Equation (12.9) looks like an AR(1) model except that Xt (and εt) are now vectors instead of scalars. Since Xt depends on Xt−1, it is natural to call this model a vector autoregressive model of order 1 (VAR(1)). Equation (12.9) can be further rewritten as (12.10) where B denotes the backward shift operator, I is the (2×2) identity matrix and ΦB represents the operator matrix . We can readily generalize the above model from two to m variables and from first-order autoregression to pth order. In general, a VAR model of order p (VAR(p)) can be written in the form (12.11) where Xt is a (m×1) vector of observed variables, and Φ is a matrix polynomial of order p in the backward shift operator B such that

< previous page

file:///C:/Documents and Settings/Yang/

page_247

next page >

面/The analysis of time series an introduction/files/page_247.html [5/24/2009 16:53:47]

page_248

< previous page

page_248

next page >

Page 248 where I is the (m×m) identity matrix and Φ1, Φ2,…, ΦP are (m×m) matrices of parameters. We restrict attention to stationary processes, and hence, without loss of generality, we may assume the variables have been scaled to have zero mean. This explains why there is no constant on the right-hand side of Equation (12.11). The condition for stationarity is that the roots of the equation, determinant , should lie outside the unit circle. Note that this condition reduces to the familiar condition for stationarity in the univariate case when m=1. In Equation (12.11), we have used εt to denote m-dimensional white noise. Although we introduced bivariate white noise in the VAR(1) example above, we need to define white noise more generally in m dimensions. Let denote an (m×1) vector of random variables. This multivariate time series will be called multivariate white noise if it is stationary with zero mean vector 0, and if the values of εt at different times are uncorrelated. Then the (m×m) matrix of the cross-covariances of the elements of εt with the elements of εt+j is given by

where Γ0 denotes a (m×m) symmetric positive-definite matrix and 0m denotes an (m×m) matrix of zeroes. This means that each component of εt behaves like univariate white noise. Notice that the covariance matrix at lag zero, namely, Γ0, does not need to be diagonal, as an innovation at a particular time point could affect more than one measured variable at that time point. Thus we do allow the components of εt to be contemporaneously correlated. The mathematics for a VAR model involves matrix polynomials that may look rather strange at first. To get a better feel for them, the reader is advised to look again at the first-order polynomial example included in Equation (12.10) and write out the scalar equations it represents. Given the matrix Γ0, describing the contemporaneous covariances of the white noise, it is possible in principle to evaluate the covariance matrix function, and hence the correlation matrix function of a VAR process. In practice, the algebra is usually horrid and it is only possible to find simple analytic functions in some simple cases. One gets a generalized matrix form of the Yule-Walker equations, which can be difficult to solve. Consider, for simplicity, the VAR(1) model in Equation (12.9). Multiply through on the right-hand side by and take expectations. When k>0, we get Γ(k)= Γ(k−1)ΦT,but when k=0, we get Γ(0)=Γ(−1)ΦT +∑0=Γ(1)TΦT+∑0. These equations are only easy to solve when Φ is diagonal, in which case one has m independent univariate AR(1) processes—see Exercise 12.3. Looking back at Equation (12.8), we notice that if is zero, then X1t does not depend on the lagged value of X2t. This means that, while X2t depends on the lagged value of X1t, there is no feedback from X2t to X1t. Put another way, this means any causality goes in one direction only and we can think of X1t and X2t as the input and output, respectively. The first equation in

< previous page

file:///C:/Documents and Settings/Yang/

page_248

next page >

面/The analysis of time series an introduction/files/page_248.html [5/24/2009 16:53:48]

page_249

< previous page

page_249

next page >

Page 249 (12.8) is univariate while the model for X2t could, in fact, be rewritten in the form of a transfer function model. More generally a transfer-function model may be regarded as a special case of the VAR model in Equation (12.11) if the variables can be ordered in such a way that each Φi matrix is lower triangular (meaning that all coefficients above the diagonal are zero). The last component of Xt is then the output (or response variable) in an open-loop system. In contrast, in a closed-loop system, the ‘outputs’ feed back to affect the ‘inputs’ and the general VAR model may then be appropriate to describe the behaviour of the mutually dependent variables. The definition of a VAR model in Equation (12.11) does not attempt to describe features such as trend and seasonality. It is possible to add deterministic terms to the right-hand side of Equation (12.11) to account for a non-zero mean, for trend and for seasonality (e.g. by including seasonal dummy variables). However, I think my preference for seasonal data is to deseasonalize the data before attempting to model them, especially if the aim is to produce seasonally adjusted figures and forecasts. One simple possibility is to use seasonal differencing. As regards trend, non-seasonal (first) differencing may be employed to remove trend. However, the use of differencing is also not without problems, particularly if co-integration is present (see Section 12.6 below). Thus, fitting VAR models to real data is not easy, and will be discussed separately in Section 12.5 below. 12.4 Vector ARMA Models As in the univariate case, the VAR model may be generalized to include moving average (MA) terms. Building on Equation (12.11), this is done in an ‘obvious’ way by writing (12.12) where is a matrix polynomial of order q in the backward shift operator B and Θ1, Θ2,…, Θq are (m×m) matrices of parameters. Then Xt is said to follow a vector ARMA (VARMA) model of order (p, q). Equation (12.12) is a natural generalization of the univariate ARMA model, and reduces to the familiar univariate ARMA model when m=1. For stationary models, satisfying the stationarity condition given in Section 12.3, we can impose a condition for invertibility analogous to that in the univariate case. This requires that the roots of the equation, determinant should lie outside the unit circle. This condition reduces to the usual univariate invertibility condition when m=1. If Φ(B) includes a factor of the form I(1−B), then the model is not stationary but rather acts on the first differences of the components of Xt. By analogy with the univariate case, such a model is called a vector ARIMA (VARIMA) model. Note that it may not be optimal in practice to difference each component of Xt in the same way. Moreover the possible presence of

< previous page

file:///C:/Documents and Settings/Yang/

page_249

next page >

面/The analysis of time series an introduction/files/page_249.html [5/24/2009 16:53:49]

page_250

< previous page

page_250

next page >

Page 250 co-integration (see Section 12.6 below) also needs to be considered before differencing multivariate data. Forecasts can readily be computed for VAR, VARMA and VARIMA models by a natural extension of methods employed for univariate ARIMA models. Generally speaking, minimum mean square error (MMSE) forecasts can be obtained by replacing future values of white noise with zeroes while future values of Xt are replaced with MMSE forecasts. Present and past values of Xt and of εt are replaced by the observed values and the (one-step-ahead forecast) residuals, respectively. Details will not be given here, but see Exercise 12.5. One problem with VARMA (or VARIMA) models is that there may be different, but equivalent (or exchangeable) ways of writing what is really the same model. There are various ways of imposing constraints on the parameters involved in Equation (12.12) to ensure that a model is identifiable, meaning that the model is unique, but the conditions are complicated and will not be given here. There are further problems involved in identifying and fitting a model with MA components—see Section 12.5—and, analogous to the univariate case, VARMA models are generally (much) harder to handle than VAR models. Finally, we note that VARMA models can be generalized by adding terms, involving additional exogenous variables, to the right-hand side of Equation (12.12) and such a model is sometimes abbreviated as a VARMAX model. 12.5 Fitting VAR and VARMA Models There are various approaches to the identification of VARMA models. They involve assessing the orders p and q of the model, estimating the parameter matrices in Equation (12.12) and estimating the variancecovariance matrix of the ‘noise’ components. We do not give details here, but rather refer the reader, for example, to Priestley (1981), Lütkepohl (1993) and Reinsel (1997). A recent survey of VARMA models is given by Tiao (2001). Identification of a VARMA model is inevitably a difficult and complicated process because of the large number of model parameters that may need to be estimated. The number of parameters increases quadratically with m and can become uncomfortably large when the lag length is more than one or two. This suggests that some constraints need to be placed on the model. One possibility is to use external knowledge or a preliminary analysis of the data to identify coefficient matrices where most of the parameters can a priori be taken to be zero. Such matrices are called sparse matrices. However, even with some sparse matrices included in the model, VARMA models are still difficult to fit, and so many analysts restrict attention to VAR models, which they hope will give an adequate approximation to VARMA models. Even then, there is still a danger of overfitting, and fitted VAR models do not always provide an approximation to real-life multivariate data that is as parsimonious and useful as AR models are for univariate data.

< previous page

file:///C:/Documents and Settings/Yang/

page_250

next page >

面/The analysis of time series an introduction/files/page_250.html [5/24/2009 16:53:49]

page_251

< previous page

page_251

next page >

Page 251 Because of the dangers of overfitting, a technique called Bayesian vector autoregression (BVAR) may be used to fit VAR models, in preference to using ordinary least squares. This approach can be used whether or not the analyst has a Bayesian philosophy The technique essentially aims to prevent overfitting by shrinking parameters higher than first-order towards zero. The usual prior that is used for the parameters, called the Minnesota prior, has mean values, which assume a priori that every series is expected to be a random walk. Other priors have also been tried (e.g. Kadiyala and Karlsson, 1993). A tutorial paper showing how to select an appropriate BVAR model is given by Spencer (1993). One important tool in VARMA model identification is the matrix of cross-correlation coefficients. The case of two-time series has already been considered in Section 8.1, and some difficulties involved in interpreting a sample cross-correlation function were noted. The author admits that he has typically found it difficult to interpret cross-correlation (and cross-spectral) estimates, even when there are only two variables. The analysis of three or more series is in theory a natural extension, but in practice is much more difficult and should only be attempted by analysts with substantial experience in univariate ARIMA model building. As previously noted, the interpretation of cross-correlations is complicated by the possible presence of autocorrelation within the individual series and by the possible presence of feedback between the series, and it is now generally recognized that series should be filtered or prewhitened before looking at crosscorrelations. Any example presented here would need to be long and detailed to incorporate all necessary contextual material and to cover the diagnostic decisions that need to be made. Given space constraints and the fact that this is an introductory text, no example will be given. Multivariate modelling, even more than univariate modelling, needs to be learned by actually doing it, preferably aided by a good software package. However, the interested reader can find some nice empirical examples in Tiao (2001). A number of studies have been published, which suggest that, when carefully applied, the use of VARMA, and more especially of VAR models, can lead to improved forecasts as compared with univariate and other multivariate models. For example, the results in Boero (1990) suggest that a BVAR model is better than a large-scale econometric model for short-term forecasting, but not for long-term forecasts where the econometric model can benefit from judgemental interventions by the model user and may be able to pick up non-linearities not captured by (linear) VAR models. As is usually the case, different approaches and models are complementary. As a second example, Bidarkota (1998) found that a bivariate ARMA model gave better out-of-sample forecasts of real U.S. interest rates than a univariate unobserved components model. There is evidence that unrestricted VAR models do not forecast as well as when using Bayesian vector autoregression (e.g. Kadiyala and Karlsson, 1993), presumably because unrestricted models may incorporate spuriously many parameters.

< previous page

file:///C:/Documents and Settings/Yang/

page_251

next page >

面/The analysis of time series an introduction/files/page_251.html [5/24/2009 16:53:50]

page_252

< previous page

page_252

next page >

Page 252 Of course, multivariate models are often constructed to help describe and understand the measured system, rather than (just) to produce forecasts. An example using VAR models is given by Dhrymes and Thomakos (1998)1. One unresolved question is whether and when it is better to difference one or all of the data series before trying to fit VAR models. There is conflicting evidence on this point, some of which can be found in a special issue of the Journal of Forecasting (1995, No. 3), which is entirely devoted to VAR modelling and forecasting and gives many more references on this and other aspects of VAR modelling. The question of differencing is related to the important topic of co-integration, which is considered in the next section. 12.6 Co-integration Modelling multivariate time-series data is complicated by the presence of non-stationarity, particularly with economic data. One possible approach is to difference each series until it is stationary and then fit a VARMA model. However, this does not always lead to satisfactory results, particularly if different degrees of differencing are appropriate for different series or if the structure of the trend is of intrinsic interest in itself (and, in particular, assessing whether the trend is deterministic or stochastic). An alternative approach, much used in econometrics, is to look for what is called co-integration. As a simple example, we might find that X1t and X2t are both non-stationary but that a particular linear combination of the two variables, say (X1t−kX2t) is stationary. Then the two variables are said to be cointegrated. If we now build a model for these two variables, there is no need to take first differences of both observed series, but rather the constraint implied by the stationary linear combination (X1t−kX2t) needs to be incorporated in the model. A more general definition of co-integration is as follows. A series {Xt} is said to be integrated of order d, written I(d), if it needs to be differenced d times to make it stationary. If two series {X1t} and {X2t} are both I(d), then any linear combination of the two series will usually be I(d) as well. However, if a linear combination exists for which the order of integration is less than d, say (d−b), then the two series are said to be co-integrated of order (d, b), written CI(d, b). If this linear combination can be written in the form αTXt, where then the vector a is called a co-integrating vector. Consider again the example given earlier in this section, where X1t and X2t are both non-stationary but (X1t −kX2t) is stationary. If X1t and X2t are both I(1), then d=b=1, Xt is CI(1, 1) and a co-integrating vector is αT=(1,−k). In a non-stationary vector ARIMA model, there is nothing to constrain the 1 Note that their acronym MARMA seems to be equivalent to VARMAX in the time-series literature. The use of different abbreviations and acronyms in the time-series and econometric literature can be a source of confusion.

< previous page

file:///C:/Documents and Settings/Yang/

page_252

next page >

面/The analysis of time series an introduction/files/page_252.html [5/24/2009 16:53:50]

page_253

< previous page

page_253

next page >

Page 253 individual series to make them ‘move together’ in some sense, yet the laws of economics suggest that there are bound to be long-run equilibrium forces that will prevent some economic series from drifting too far apart. This is where the notion of co-integration comes in. The constraint(s) implied by co-integration enable the analyst to fit a more realistic multivariate model. In an introductory text, it is not appropriate to give further details here. Some useful references on cointegration include Banerjee et al. (1993), Dhrymes (1997), Engle and Granger (1991) and Johansen (2001), but there have been many other contributions to the subject dealing with topics such as tests for cointegration, error-correction models and ways of describing ‘common trends’. An amusing non-technical introduction to the concept of co-integration is given by Murray (1994). I recommend that co-integration should always be considered when attempting to model multivariate economic data. 12.7 Bibliography The beginner may find it helpful to read Chapter 5 of Chatfield (2001), Chapters 7 and 8 of Granger and Newbold (1986) or Chapters 13 and 14 of Wei (1990). A thorough treatment of VAR and VARMA models is provided by Lütkepohl (1993), while Reinsel (1997) covers similar material in a somewhat terser, more mathematical style. Multivariate versions of structural modelling, of Bayesian forecasting and of non-linear modelling have not been covered in this introductory chapter and the reader is referred to Harvey (1989), to West and Harrison (1997) and to Granger and Teräsvirta (1993), respectively. Exercises 12.1 Express the following bivariate model in vector form and say what sort of model it is.

Is the model stationary and invertible? 12.2 Consider the VAR(1) model for two variables as given by Equation (12.9). Determine whether the model is stationary for the following coefficient matrices at lag one;

12.3 Find the covariance matrix function Γ(k), and the correlation matrix function P(k) for model (c) in Exercise 12.2, assuming that εt denotes bivariate white noise with covariance matrix at lag zero.

< previous page

file:///C:/Documents and Settings/Yang/

page_253

next page >

面/The analysis of time series an introduction/files/page_253.html [5/24/2009 16:53:51]

page_254

< previous page

page_254

next page >

Page 254 12.4 Consider the bivariate VARMA(0, 1) model from Equation (12.12) with . Is the model stationary and invertible? 12.5 Given data up to time N, find the one- and two-step-ahead forecasts for the bivariate VAR(1) model in Exercise 12.2 (d).

< previous page

file:///C:/Documents and Settings/Yang/

page_254

next page >

面/The analysis of time series an introduction/files/page_254.html [5/24/2009 16:53:52]

page_255

< previous page

page_255

next page >

Page 255 CHAPTER 13 Some More Advanced Topics This chapter provides a brief introduction to several more advanced topics that are unsuitable for detailed study in an introductory text. This gives a flavour of recent developments, and provides references to enable the reader to get further details if desired. Further reviews of many standard and non-standard aspects of time-series analysis, including research developments, are given, for example, by Newbold (1981, 1984, 1988) and in the collections of papers edited by Brillinger et al. (1992, 1993) and by Peña et al. (2001). 13.1 Model Identification Tools Model building is a key element of most statistical work. Some general remarks on how to formulate an appropriate model for a given time series were made in Section 4.8. As well as finding out about any background knowledge, the analyst will typically look at the time plot of the data and at various diagnostic tools. The two standard tools, which are used to identify an appropriate autoregressive moving average (ARMA) model for a given stationary time series (see Chapter 4), are the sample autocorrelation function (ac. f.) and the sample partial ac.f. This choice is often made subjectively, using the analyst’s experience to match an appropriate model to the observed characteristics. However, various additional diagnostic tools are also available, and this section gives a brief introduction to some of them. Detailed reviews of methods for determining the order of an ARMA process are given by de Gooijer et al. (1985) and Choi (1992); see also Newbold (1988). An alternative to the partial ac.f. is the inverse ac.f., whose use in identifying ARMA models is described by Chatfield (1979). The inverse ac.f. of the general ARMA model, as written in Equation (3.6a), namely is exactly the same as the ordinary ac.f. of the corresponding inverse ARMA model given by where θ and are interchanged. It turns out that the inverse ac.f. has similar properties to the partial ac.f. in that it ‘cuts off’ at lag p for an AR(p) process but generally dies out slowly for MA and ARMA processes. The inverse ac.f. often contains more information than the partial ac.f., extends easily to seasonal ARMA models, and is a viable competitor to the partial ac.f. Instead of (or in addition to) subjectively examining functions like the ac.f.,

< previous page

file:///C:/Documents and Settings/Yang/

page_255

next page >

面/The analysis of time series an introduction/files/page_255.html [5/24/2009 16:53:52]

page_256

< previous page

page_256

next page >

Page 256 two alternative approaches to model selection are to carry out a series of hypothesis tests or to use a model-selection criterion. Econometricians tend to favour the former approach and test null hypotheses such as normality, constant variance and non-linearity. Statisticians tend to favour the latter approach and several such statistics have been proposed. Given a class of models, such as the ARIMA class, the idea is to choose a model from that class so as to optimize a suitably chosen function of the data. What criterion should we use to select a model in the ‘best’ way? It is not sensible to simply choose a model to give the best fit by minimizing the residual sum of squares, as the latter will generally decrease as the number of parameters is increased regardless of whether the additional complexity is really worthwhile. There is an alternative fit statistic, called adjusted-R2, which makes some attempt to take account of the number of parameters fitted, but more sophisticated model-selection statistics are generally preferred. Akaike’s Information Criterion (AIC) is the most commonly used and is given (approximately) by: where r denotes the number of independent parameters that are fitted for the model being assessed. Thus the AIC essentially chooses the model with the best fit, as measured by the likelihood function, subject to a penalty term, to prevent over-fitting, that increases with the number of parameters in the model. For an ARMA(p, q) model, note that r=p+q+1 as the residual variance is included as a parameter. Ignoring arbitrary constants, the first (likelihood) term is usually approximated by N ln(S/N), where S denotes the residual sum of squares, and N is the number of observations. It turns out that the AIC is biased for small samples, and a bias-corrected version, denoted by AICC, is increasingly used. The latter is given (approximately) by replacing the quantity 2r in the ordinary AIC with the expression 2rN/(N−r−1). The AICC is recommended, for example, by Brockwell and Davis (1991, Section 9.3) and Burnham and Anderson (2002). An alternative, widely used criterion is the Bayesian Information Criterion (BIC) that essentially replaces the term 2r in the AIC with the expression (r+r lnN). Thus it penalizes the addition of extra parameters more severely than the AIC. Schwartz’s Bayesian criterion is yet another alternative that is similar to BIC in its dependence on log N, but replaces (r+r ln N) with r ln N. Several other possible criteria have also been proposed including Parzen’s autoregressive transfer function criterion (CAT) and Akaike’s final prediction error (FPE) criterion, which are both primarily intended for choosing the order of an AR process. Note that all the above criteria may not have a unique minimum and depend on assuming that the data are (approximately) normally distributed. Priestley (1981, Chapter 5) and Burnham and Anderson (2002) give a general review of these criteria. Following the results in Faraway and Chatfield (1998), I generally prefer to use the AICC or BIC. Computer packages routinely produce numerical

< previous page

file:///C:/Documents and Settings/Yang/

page_256

next page >

面/The analysis of time series an introduction/files/page_256.html [5/24/2009 16:53:53]

page_257

< previous page

page_257

next page >

Page 257 values for several such criteria so that analysts can pick the one they like best. The guiding principle throughout is to apply the Principle of Parsimony, introduced in Section 3.4.5. In brief this says, ‘Adopt the simplest acceptable model’. Of course, there is a real danger that the analyst will try many different models, pick the one that appears to fit best according to one of these criteria, but then make predictions as if certain that the best-fit model is the true model. Further remarks on this problem are made in Section 13.5. As noted above, an alternative approach to model selection relies on carrying out a series of hypothesis tests. However, little will be said here about this approach, because the author prefers to rely on the subjective interpretation of diagnostic tools, such as the ac.f., allied to the model-selection criteria given above. My three reasons for this preference are as follows: 1. A model-selection criterion gives a numerical-valued ranking of all models, so that the analyst can see whether there is a clear winner or, alternatively, several close competing models. 2. Some model-selection criteria can be used to compare non-nested1 models, as would arise, for example, when trying to decide whether to compute forecasts using an ARIMA, neural network or econometric model. It is very difficult to use hypothesis tests to compare non-nested models. 3. A hypothesis test requires the specification of an appropriate null hypothesis, and effectively assumes the existence of a true model that is contained in the set of candidate models. 13.2 Modelling Non-Stationary Series Much of the theory in the time-series literature is applicable to stationary processes. In practice most real time series have properties that do change with time, albeit slowly in many cases, and we have already met some methods for dealing with non-stationarity. For example, the use of differencing with ARIMA models allows stationary models to be fitted, while various explicit models for trend have already been introduced. More generally, even with a long, apparently stationary series, it is still a good idea to split the series into reasonably long, non-overlapping segments and compare the properties of the segments, particularly the general form of the ac.f. and spectrum. This section discusses the problem of non-stationarity more generally, because it is important to understand the different types of non-stationarity that may arise and methods for dealing with them (see also Priestley, 1988, Chapter 6). For example, when the non-stationary features are not of primary concern (e.g. when instrument drift arises), then it is sensible to find a method that transforms the data to stationarity (e.g. differencing), as this enables us to 1 Two models are nested when one is a subset of the other. For example, a first-order AR model is a special case of a second-order AR model when the coefficient at lag two is zero. This makes it relatively easy to test the null hypothesis that a first-order model is adequate.

< previous page

file:///C:/Documents and Settings/Yang/

page_257

next page >

面/The analysis of time series an introduction/files/page_257.html [5/24/2009 16:53:54]

page_258

< previous page

page_258

next page >

Page 258 fit stationary models and use the theory of stationary processes. Alternatively, when the non-stationary features of the data are of intrinsic interest in themselves, then it may be more rewarding to model them explicitly, rather than remove them and concentrate on modelling the stationary residuals. Thus there is, for example, a fundamental difference between fitting an ARIMA model, which describes non-stationary features implicitly, and fitting a state-space model, which describes them explicitly. Slow changes in mean are one common source of non-stationarity. If a global (deterministic) function of time can be assumed, then such components can easily be fitted and removed (e.g. by fitting a polynomial). However, we have seen that it is now more common to assume that there are local changes in the mean and perhaps fit a local linear trend that is updated through time. Some sort of filtering or differencing may then be employed. It helps if the filtered series has a natural contextual interpretation. Cyclical changes in mean (e.g. seasonality) can also be dealt with by filtering, by differencing or by fitting a global model perhaps using a few sine and cosine terms of appropriate frequency. Changes in variance may also be evident in the time plot and then the models of Section 11.3 should be considered. Turning to parametric models, let us consider an AR process as an example of a linear model, and distinguish several different ways in which non-stationarity may arise for such a model. If the AR coefficients do not satisfy the stationarity conditions, because one or more roots of Equation (3.5) lie on the unit circle, then the series can be made stationary by differencing. Alternatively, if one or more roots lie outside the unit circle, then this leads to explosive behaviour, which cannot be made stationary by differencing. This sort of model2 is much more difficult to handle. Another way that non-stationarity may arise is that the AR coefficients are changing through time, perhaps suddenly (e.g. Tyssedal and Tjostheim, 1988), or perhaps slowly (e.g. Swanson and White, 1997). The latter provides just one example of the many ways in which there can be a slow change in the underlying model structure. Changes in structure are generally more difficult to handle than something like a linear trend. Changes in the model structure can be studied in the time domain, for example, by seeing how model parameters change through time, but can can also be studied in the frequency domain, and there are various ways of generalizing the spectrum to cope with non-stationary behaviour. The use of evolutionary spectra (Priestley, 1981, Chapter 11; 1988) is one possibility. This allows the spectrum to change slowly through time in a particular type of way. Complex demodulation is an alternative approach, which studies 2 It is mathematically interesting to note, for example, that a non-stationary first-order AR process, with a lag-one coefficient greater than one, does have a stationary solution if time is reversed. However, this is usually regarded as unnatural, as the process is then no longer a causal process, by which is meant that the value of an observed time series is only allowed to depend on present and past data.

< previous page

file:///C:/Documents and Settings/Yang/

page_258

next page >

面/The analysis of time series an introduction/files/page_258.html [5/24/2009 16:53:54]

page_259

< previous page

page_259

next page >

Page 259 signals in a narrow frequency band to see how they change through time (e.g. Bloomfield, 2000, Chapter 7). Perhaps the most difficult type of non-stationarity to handle is a sudden change in structure, due perhaps to the occurrence of a known external event such as an earthquake or a labour dispute. A structural change may produce a short-term transient effect or a long-term change in the model structure, such as a change in mean. With short-term effects, one or more outliers may be visible in the time plot and these can create problems with standard time-series methods, unless the outliers are modified in some way—see Section 13.7.5. The times at which sudden changes occur are called change points and the identification of such events is an important general problem in time-series analysis with a growing literature. Wherever possible, external knowledge of the given context should be used to decide where change points have occurred, though this may need to be substantiated by examination of the data. Box et al. (1994, Chapter 12) show how to model sudden changes with a technique called intervention analysis, which is somewhat similar to the use of dummy variables in regression. Suppose, for example, that there is a sudden change in the level, of size K say, at time T. Then we can introduce a standardized step change variable, say St, of the form

such that the quantity K St describes the resulting effect. The quantity K St can then be included in the model for the observed variable, and the constant K can be estimated along with all the other model parameters. It helps if the change-point time T is known from the context, though this parameter can also be estimated if necessary. The class of models could, for example, be an ARIMA or a transfer function model. An alternative approach is to use state-space models, or the dynamic linear models of Bayesian forecasting (see Chapter 10), as they can also be constructed so as to allow the user to deal with outliers and step changes in the mean and trend. An econometric approach to looking for change point dates is given by Bai and Perron (1998). Despite the above remarks, in some situations it may be wiser to accept that there is no sensible way to model a non-stationary process. For example, Figure 5.1(a) showed data on the sales of insurance policies and this time plot shows there are some sudden changes in the underlying structure. When I was asked to produce forecasts for this series, I proceeded very cautiously. Rather than try to model these data in isolation, it is more important to ask questions to get appropriate background information as to why the time plot shows such unusual behaviour. In this case I found that the two large peaks corresponded to two recent sales drives. Thus the most important action is to find out whether there is going to be a third sales drive. Even if such information is forthcoming, it may still be difficult to incorporate it formally into a mathematical model. In the absence of such contextual information, it would be unwise to try to produce model-based forecasts.

< previous page

file:///C:/Documents and Settings/Yang/

page_259

next page >

面/The analysis of time series an introduction/files/page_259.html [5/24/2009 16:53:55]

page_260

page_260

< previous page

next page >

Page 260 13.3 Fractional Differencing and Long-Memory Models An interesting variant of ARIMA modelling arises with the use of what is called fractional differencing, leading to a fractional integrated ARMA (ARFIMA) model. The ARIMA(p, d, q) model from Chapter 3 is usually written as where (B) and θ(B) are polynomials of order p, q, respectively, in the backward shift operator B. Here p, q and d are integers and, in particular, the order of differencing d is an integer that is typically zero or one. We further assume here that and θ have all their roots outside the unit circle, so that when d=0 the process is stationary and invertible. Fractional ARIMA models extend the above class of models by allowing d to be noninteger. In other words, the formula for an ARFIMA model is exactly the same as for an ARIMA(p, d, q) model, except that d is no longer restricted to being integer. When d is not an integer, then the dth difference be represented by its binomial expansion, namely

becomes a fractional difference, and may

As such, it is an infinite weighted sum of past values. This contrasts with the case where d is an integer when a finite sum is obtained. It can be shown (e.g. Brockwell and Davis, 1991, Section 13.2) that an ARFIMA process is stationary provided that , the process is not stationary in the usual sense, but further integer differencing can be used to give a stationary ARFIMA process. For example, if an observed series is ARFIMA(p, d=1.3, q), then the first differences of the series will follow a stationary ARFIMA(p, d=0.3, q) process. A drawback to fractional differencing is that it is difficult to give an intuitive interpretation to a non-integer difference. It is also more difficult to calculate them, given that the binomial expansion will need to be truncated, and so the parameter d is often estimated in the frequency domain. Details will not be given here and the reader is referred, for example, to Beran (1994), Crato and Ray (1996) and the earlier references therein, though the reader is warned that the literature is technically demanding. A stationary ARFIMA model, with 0

Page 261 process with ac.f. ρk is said to be a long-memory process if

does not converge. In particular,

the latter condition applies when the ac.f. ρk is of the form where C is a constant, not equal to zero, and 0

面/The analysis of time series an introduction/files/page_267.html [5/24/2009 16:54:00]

page_268

page_268

< previous page

next page >

Page 268 13.7 Miscellanea This section gives an very brief introduction to a number of diverse topics, concentrating on giving appropriate references for the reader to follow up if desired. 13.7.1 Autoregressive spectrum estimation Spectral analysis is concerned with estimating the spectrum of a stationary stochastic process. The approach described in Chapter 7, which is based on Fourier analysis, is essentially non-parametric in that no model is assumed a priori. A parametric approach called autoregressive spectrum estimation is a possible alternative method, and this will now be briefly described. Many stationary stochastic processes can be adequately approximated by an autoregressive (AR) process of sufficiently high order, say The spectrum of (13.1) at frequency

is inversely proportional to

(13.1)

(13.2) In order to estimate the spectrum using this class of models, the user has to assess p, the order of the process, perhaps using AIC. Then the AR parameters are estimated, as described in Section 4.2, and substituted into the reciprocal of Equation (13.2). The approach tends to give a smoother spectrum than that given by the non-parametric approach, though it can also pick out fairly narrow peaks in the spectrum. Further details and examples are given by Priestley (1981, Section 7.8–7.9), Percival and Walden (1993, Chapter 9), Hayes (1996, Section 8.5) and Broersen (2002), and the approach is certainly worth considering. Note that the selected order of the process is typically higher than when fitting models for forecasting purposes, as values around 10 are common and values as high as 28 have been reported. An obvious development of this approach (Broersen, 2002) is to fit ARMA models rather than AR models to give what might be called ARMA spectrum estimation, but the simplicity of fitting AR models is likely to inhibit such a development. It is also worth noting that Percival and Walden (1993) suggest using the AR model as a pre-whitening device. Given that the AR model is likely to be approximation, rather than a true model, the residuals from the fitted AR model can themselves be the subject of a spectral analysis, from which the spectrum of the original series can be inferred by dividing by Equation (13.2). The reason for doing things this way is that it is generally easier to estimate a flattish spectrum than one with peaks, and we expect the residuals from the AR process to be close to white noise. Of course, if the AR model really is exactly appropriate, then the residuals will have a uniform spectrum.

< previous page

file:///C:/Documents and Settings/Yang/

page_268

next page >

面/The analysis of time series an introduction/files/page_268.html [5/24/2009 16:54:01]

page_269

< previous page

page_269

next page >

Page 269 13.7.2 Wavelets Wavelet functions are an orthonormal set of functions, which can approximate a discontinuous function better than Fourier series. They can be used to analyse non-stationary time series and give a distribution of power in two dimensions, namely, time and frequency, (rather than just one, namely, frequency, as in spectral analysis). One commonly-used wavelet family, with ‘nice’ properties, is the Daubechies family, but these functions have what appears, at first sight, to be a rather peculiar shape. Introductions to wavelets are given by Strang (1993) and Abramovich et al. (2000). The latter includes a section, with references, on applications to time-series analysis. Percival and Walden (2000) provide book-length coverage of the latter. Wavelets continue to be the subject of much current research. 13.7.3 ‘Crossing’ problems If one draws a horizontal line (an axis) through a stationary time series, then the time series will cross and recross the line in alternate directions. The times at which the series cuts the axis provide valuable information about the properties of the series. Indeed in some practical problems the crossing points are the only information available. The problems of inferring the properties of a time series from its ‘level-crossing’ properties are discussed by Cramér and Leadbetter (1967), Blake and Lindsey (1973), Kedem (1994) and Bendat and Piersol (2000, Section 5.5). The latter also discuss related topics such as when and where peak values occur. 13.7.4 Observations at unequal intervals, including missing values This book has been mainly concerned with discrete time series measured at equal intervals of time. When observations are taken at unequal intervals, either by accident or design, there are additional complications in carrying out a time-series analysis. There are various ways in which observations may arise at unequal time intervals. The most common situation is where observations are meant to be available at equal intervals but, for some reason, some are missing. In the latter case it is important to assess whether points are missing ‘at random’ (whatever that means), whether a group of consecutive observations is missing or there is some other systematic pattern, such as every fifth observation being absent. The reason why observations are missing must also be taken into account in the modelling process, and here the context is crucial. For example, special techniques are needed when the data are censored or truncated, as will arise, for example, when every observation that exceeds a certain threshold value is missing. In practice, it is common to find that a block of measurements is missing, perhaps because the measuring machine stopped working for a period, and then common sense may be used to impute the missing values. For example, given hourly temperature readings at a given site for 1 year,

< previous page

file:///C:/Documents and Settings/Yang/

page_269

next page >

面/The analysis of time series an introduction/files/page_269.html [5/24/2009 16:54:02]

page_270

< previous page

page_270

next page >

Page 270 you may find that observations for two consecutive days are missing. Then it may be reasonable to insert appropriate hourly averages for that time of year, perhaps adjusted by the difference between the previous day’s average and the appropriate monthly average. A rather different situation arises when observations are taken at random time points, as would arise, for example, if temperature measurements are only taken when it is raining. Here again the context is crucial in deciding what mechanism is appropriate for describing the pattern of time points, and hence deciding how to analyse the data. When observations are missing at random, it may be desirable to estimate, or impute, the missing values so as to have a complete time series. Alternatively, some computations can be carried out directly on the ‘gappy’ data. For example, the autocovariance coefficient at lag k—see Equation (4.1)—may readily be adapted by summing over all pairs of observations where both values are present, and then dividing by the appropriate number of pairs. It is easy to go wrong when computing the latter number. If, for example, there is just one observation missing, say xm, then two pairs could be absent, namely, (xm−k, xm) and (xm, xm+k), provide xm is not within k observations of the ends of the series. Thus the denominator in Equation (4.1) would then be (N−2). When data are collected at unequal intervals, the computation of autocovariance and autocorrelation coefficients is no longer a viable possibility. An alternative is to compute a function, called the (semi)variogram, which is introduced later in Section 13.7.8. Diggle (1990, Section 2.5.2) shows how to apply this to unequally-spaced data. In the frequency domain, the general definition of the periodogram—Equation (7.17)—can be adapted to cope with both missing observations and observations collected at unequal intervals, though, in the latter case, the values of t need no longer be integers. A technique, called spline regression, can be used to compute a smoothed approximation to a time series, even when it is unequally spaced. A spline is a continuous piecewise polynomial function increasingly used in computational work. A brief introduction is given by Diggle (1990, Section 2.2.3), while further details in a regression context are given by Green and Silverman (1994). Some further references on unequally spaced and missing observations include Jones (1985), Peña (2001) and the collection of papers edited by Parzen (1984), while additional references are listed by Priestley (1981, p. 586) and Hannan (1970, p. 48). 13.7.5 Outliers and robust methods As in other areas of statistics, the presence of outliers can disrupt a time-series analysis. We have briefly referred to the problems raised by outliers in Sections 1.3, 2.7.2 and 13.2. Here we make some more general remarks and give a few more references.

< previous page

file:///C:/Documents and Settings/Yang/

page_270

next page >

面/The analysis of time series an introduction/files/page_270.html [5/24/2009 16:54:02]

page_271

< previous page

page_271

next page >

Page 271 Outliers, or aberrant observations, are often clearly visible in the time plot of the data. If they are obviously errors, then they need to be adjusted or removed. Some measuring instruments record occasional values that are clearly anomalous and can be dealt with fairly easily. The situation is more difficult when it is not known whether the outlier is an error or a genuine extreme value. In the latter case, it could be misleading to remove the observation completely, but leaving the outlier in could ‘mess up’ the analysis. The context is crucial in deciding what to do. Most outliers that occur in practice are of the additive outlier type. They only affect the single observation where the outlier occurs, and can be thought of as the genuine observation plus or minus a one-off ‘blip’. In contrast the innovation outlier occurs in the noise process and can affect all subsequent observations in the series. As a result, the innovation outlier is more difficult to deal with and may have a long-lasting effect. A rather different type of effect arises when there is a sudden step change in the level or trend of a series. This may generate apparent outliers in the short term, but the problems are of a rather different kind. They may be best dealt with as a structural change, perhaps by the use of intervention analysis—see Section 13.2. Here again, the context is crucial in determining the times of any change points and in assessing the form of their effect. Some useful references on outliers include the following: Tsay (1986) discusses model specification in the presence of outliers, while Chang et al. (1988) focus on parameter estimation. Abraham and Chuang (1989) discuss outlier detection for time series. Ledolter (1989) discusses the effect of outliers on forecasts, while Chen and Liu (1993) show that outliers are particularly dangerous when they are near the forecast origin at the end of the series. The latter paper considers the different types of outliers as well as some strategies for reducing potential difficulties. Luceño (1998) discusses how to detect multiple (possibly non-consecutive) outliers for industrial data assumed to be generated by an ARIMA process. Franses (1998, Chapter 6) provides a general introduction, while the new edition of Box et al. (1994, Section 12.2) also includes material on outlier analysis. Peña (2001) gives a recent review and, together with Chen and Liu (1993), provides a good source of further references about outliers. Work has started on outlier detection for vector time series (e.g. Tsay et al., 2000), but note that this can be much more difficult than in the univariate case. An outlier in one component of the series can cause ‘smearing’, not only in adjacent observations on the same component, but also in adjacent observations on other components. Note that it can be difficult to tell the difference between outliers caused by errors, by non-linearity (see Chapter 11) and by having a non-normal ‘error’ distribution. It is worth bearing all three possibilities in mind when looking at apparently unusual values. Instead of adjusting or removing outliers, an alternative approach is to use robust methods, which automatically downweight extreme observations. Three examples of this type of approach will be briefly mentioned. First,

< previous page

file:///C:/Documents and Settings/Yang/

page_271

next page >

面/The analysis of time series an introduction/files/page_271.html [5/24/2009 16:54:03]

page_272

< previous page

page_272

next page >

Page 272 the use of running median smoothers (e.g. Velleman and Hoaglin, 1981) can be used to produce a smoothed version of a time series. Second, LOcally WEighted regreSSion (LOWESS or LOESS) can also be used to smooth a time series, and can also be used iteratively to get smooth estimates of different components of a time series (Cleveland, 1993). Third, we note that the Kalman filter can also be robustified (Meinhold and Singpurwalla, 1989). As a final comment, it is worth reiterating the general finding that local models seem to be more robust to departures from model assumptions than global models, and this explains, for example, why simple forecasting techniques like exponential smoothing often seem to compete empirically with more complicated alternative techniques, such as those based on ARIMA or vector autoregressive (VAR) models 13.7.6 Repeated measurements In a longitudinal study, a series of measurements is taken through time on each of a sample of individuals or subjects. The data therefore comprise a collection of time series, which are often quite short. Special methods have been developed to tackle the analysis of such data. A modern introduction is given by Diggle et al. (2002, Chapter 5), while Jones (1993) gives a more advanced state-space approach. 13.7.7 Aggregation of time series Many time series are produced by some sort of aggregation. For example, rainfall figures can be calculated by summing, or aggregating, over successive days or successive months. Aggregation over successive time periods is called temporal aggregation. If you find a model for a series at one level of aggregation (e.g. days), then it should be stressed that a completely different type of model may be appropriate for observations on the same variable over a different time period (e.g. months). Wei (1990, Chapter 16) discusses the effect of aggregation on the modelling process. An alternative type of aggregation is contemporaneous aggregation, where aggregation is carried out by summing across series for the same time period. For example, sales of a particular product in a given time period could be aggregated over different regions, over different countries or over different brand sizes of the product. This raises rather different problems to temporal aggregation. For example, in inventory control, a typical question is to ask whether better forecasts can be developed by forecasting an aggregate series (e.g. total sales in a country) and then allocating them to the subseries (e.g. sales in regions) based on historical relative frequencies—called a top-down approach, or to forecast each component series and then add the forecasts in order to forecast the grand total—called the bottom-up approach. There is some empirical evidence that the latter approach is sometimes better

< previous page

file:///C:/Documents and Settings/Yang/

page_272

next page >

面/The analysis of time series an introduction/files/page_272.html [5/24/2009 16:54:04]

page_273

< previous page

page_273

next page >

Page 273 (Dangerfield and Morris, 1992), but this question is very context dependent and it is not advisable to give general guidelines. 13.7.8 Spatial and spatio-temporal series In some scientific areas, particularly in ecology and agriculture, data often arise that are ordered with respect to one or two spatial coordinates rather than with respect to time. Methods of analysing spatial data have been described, for example, by Ripley (1981) and Cressie (1993). Although there are many similarities between time-series and spatial analysis, there are also fundamental differences (Chatfield, 1977, Section 9). One tool that is commonly used in the analysis of spatial data is the (semi)variogram. For a onedimensional series in space or time, this may be defined as (13.3) Note the factor that is customarily introduced, and it is this that explains the adjective semi. For a stationary process, V(k) is often written as E[{Xt+k−Xt}2], and, in this case, it can be shown that V(k) is related to the ac.f., ρ(k), of the process by (13.4) where γ(0) denotes the variance of the process. The function is generally used to assess spatial autocorrelation and is covered in books on geostatistics and spatial statistics (e.g. Cressie, 1993), especially in relation to a technique called kriging, which is concerned with predicting a response variable from spatial data. In recent years, several attempts have been made to apply the variogram to time-series data. For example Haslett (1997) shows that the variogram can be defined for some non-stationary series and used as a modelidentification tool. Sadly, the estimation of V(k) is rarely discussed in time-series books—an exception is Diggle (1990, Sections 2.5.2 and 5.4). When data are recorded with their position in both space and time, we move into the difficult area of spacetime modelling. One key question is whether the covariance function is separable, meaning that it can be expressed as a product of a purely temporal function and a purely spatial function. 13.7.9 Time series in finance The application of time-series analysis in finance is a rapidly growing area. A modern overview is given in the article by Rydberg (2000) and (more extensively) in the book by Tsay (2002). Various types of data may arise and they may be measured at (very) different time intervals. Examples include yearly or quarterly financial results, daily share prices and high-frequency

< previous page

file:///C:/Documents and Settings/Yang/

page_273

next page >

面/The analysis of time series an introduction/files/page_273.html [5/24/2009 16:54:04]

page_274

< previous page

page_274

next page >

Page 274 stock exchange data measured at intervals as short as 10 minutes3. High-frequency data are of particular interest and can generate very long series consisting of many thousands of observations. Many financial time series, such as share prices, can be modelled (approximately) by a random walk. If we denote such a series by Rt, then it is common practice to analyse the first differences, Rt−Rt−1, or (more usually) the first differences of the logarithms, namely, . The latter series is often called the (log) returns. Such series typically show little serial correlation and might be thought to be (approximately) a series of independent and identically distributed random variables. In fact, such series are usually not exactly independent because the ac.f. of the squared returns typically shows small but persistently positive values that decay slowly to zero. This long-memory effect may be due to nonlinearity or to changing variance. It is also tempting to assume that such data are normally distributed, but they are typically found to have high kurtosis, in that they have ‘heavy tails’ (or ‘fat tails’). This means that there are more extreme observations in both tails of the distribution, than would arise if data were normally distributed. There may also be evidence of asymmetry, as, for example, when returns show more large negative values than positive ones. Of course, the more aggregated the data, the better will be any normal approximation. Generalized autoregressive conditionally heteroscedastic (GARCH) models are widely used for looking at finance series, as they allow for changing variance (or volatility clustering). As regards heavy tails, one useful family of distributions is the class of stable distributions (e.g. Brockwell and Davis, 1991, Section 13.3). Loosely speaking, a distribution is stable when the convolution of two such distributions leads to another distribution in the same family. The family includes the normal and Cauchy distributions, and a stable distribution may be symmetric or skew depending on the parameters. Using a stable distribution may lead to processes with infinite variance. As many financial processes are close to a random walk, there is particular interest in finance in a variety of stochastic models that have similar properties. The famous Black-Scholes model, which should really be called the Samuelson-Black-Scholes model, is based on (geometric) Brownian motion and is popular because it leads to closed-form expressions for preference-independent derivative (or option) prices. However, the model assumes normally distributed increments and lacks some of the empirical features of real financial data. Thus, it is not a good model. The so-called Levy process is essentially a continuous-time random walk with a heavy-tailed step distribution and may be more realistic. However, these processes are defined in continuous time, and thus lead to stochastic differential equations rather than (discrete-time) difference equations. This means that the mathematics involved is much more difficult, and so these models will not be discussed here. 3 Data are recorded at even shorter intervals if they are ‘tick-by-tick’, meaning that every trade is recorded. Data like these are not standard time series.

< previous page

file:///C:/Documents and Settings/Yang/

page_274

next page >

面/The analysis of time series an introduction/files/page_274.html [5/24/2009 16:54:05]

page_275

< previous page

page_275

next page >

Page 275 Rather than fit a univariate time-series model, multivariate time-series models are often applied to finance data. The reader is referred to Tsay (2002). Another topic of particular interest in finance is the study of extreme values, which have obvious application in insurance and risk management—see Tsay (2002, Chapter 7). Finally, we mention that computationally intensive statistical methods are increasingly used in finance. As in other application areas, the methods are used to solve problems that cannot be solved analytically. Tsay (2002, Chapter 10) gives a clear introduction to topics like Markov Chain Monte Carlo (MCMC) methods that can be used in simulation. However, note that bootstrapping, which is widely used to solve problems computationally by sampling from the observed empirical distribution, is particularly difficult to apply to timeseries data because of the lack of independence—see Bühlmann (2002). Bootstrapping should be applied sparingly, or not at all, with time series! 13.7.10 Discrete-valued time series This book has concentrated on discrete time series, meaning series measured at discrete intervals of time. The observed variable is typically continuous, but will, on occasion, be discrete. Some discrete variables, with large means, can be treated as if they are approximately normally distributed and modelled in much the same way as variables having a continuous distribution. However, data occasionally arise where a completely different approach is required, perhaps because zero counts are observed and the distribution is highly skewed. Examples can arise in measuring the abundance of animal populations or recorded cases of infectious diseases. As one example, the number of wading birds present at a particular site could be zero on many days, but then go up to several hundred (or even several thousand) on one particular day or for several days or weeks in a row. Such data raise special problems that will not be discussed here. A key issue is whether to invest in modelling the ecological/mechanistic background or simply fit a statistical model using a ‘black-box’ type of approach. Another issue is whether to model presence/absence rather than actual counts. A relevant reference is Macdonald and Zucchini (1997).

< previous page

file:///C:/Documents and Settings/Yang/

page_275

next page >

面/The analysis of time series an introduction/files/page_275.html [5/24/2009 16:54:05]

page_276

< previous page

page_276

next page >

page_276

next page >

Page 276 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_276.html [5/24/2009 16:54:06]

page_277

< previous page

page_277

next page >

Page 277 CHAPTER 14 Examples and Practical Advice This chapter presents some detailed worked examples to give more of the flavour of practical time-series analysis. In the process, I give further practical advice including more detailed guidance on drawing a time plot, which is arguably the most important step in any time-series analysis. I also discuss computational considerations. This includes comments on computer software, and information about finding further timeseries data sets. Some suggestions for practical exercises are given. I begin with some general remarks on the difficulties involved in time-series analysis with real data. 14.1 General Comments I have been fascinated by time-series analysis since taking a course on the subject as a postgraduate student. Like most of my fellow students, I found it difficult but feasible to understand the time-domain material on such topics as autocorrelation functions (ac.f.s) and autoregressive moving average (ARMA) models, but found spectral analysis initially incomprehensible. Even today, despite many years of practice, I still find time-series analysis more difficult than most other statistical techniques (except perhaps multivariate analysis). This may or may not be comforting to the reader! It may be helpful to try to assess why timeseries analysis can be particularly challenging for the novice. The special feature of time-series data is that they are correlated through time rather than being independent. Now most classical statistical inference assumes that a random sample of independent observations is taken. Clearly any consequences of this assumption will not apply to time series. Correlated observations are generally more difficult to handle (whether time series or not) and this is arguably the main reason why time-series analysis is difficult. The analysis of time-series data is often further complicated by the possible presence of trend and seasonal variation, which can be hard to estimate and/or remove. It is difficult to define trend and seasonality satisfactorily and it is important to ensure that the right sort of trend (e.g. local or global) is modelled. More generally considerable subjective skill is required to select an appropriate time-series model for a given set of data. The interpretation of the correlogram and the sample spectrum is not easy, even after much practice. Furthermore, as in other areas of Statistics, data may be contaminated by missing observations and outliers and the sequential nature of time-series data can make such anomalies harder to deal with. If the analysis of a single

< previous page

file:///C:/Documents and Settings/Yang/

page_277

next page >

面/The analysis of time series an introduction/files/page_277.html [5/24/2009 16:54:06]

page_278

< previous page

page_278

next page >

Page 278 time series using linear models is not easy, then the analysis of multivariate, non-stationary and/or non-linear time series will be even more difficult. The reader is not advised to tackle such problems before getting plenty of experience with standard univariate problems. As regards the practicalities of analysing time-series data, the reader is advised to read Section 2.9 again at this point. Understanding the context is essential. What is the objective of the analysis? Have all important variables been measured? Has the method of measurement changed during the period of observation? Has there been some sort of sudden change during the period of observation? If so, why has it occurred and what should be done about it? The next step is to clean the data. This is an essential part of the initial examination of the data. For example, what is to be done about outliers and missing values? Then decisions must be made about the possible presence of trend and seasonality, and what to do about them. These preliminary matters are very important and their treatment should not be rushed. Getting clear time plots and assessing the properties of the data are essential. In my experience, the treatment of such problem-specific features as trend, seasonality, calendar effects (e.g. noting whether Easter is in March or April in each year), outliers, possible errors and missing observations can be more crucial than any subsequent actions during the modelling process. 14.2 Computer Software This section makes some brief remarks on computer software for carrying out time-series analysis. A wide variety of packages is now available, but there is little point in providing detailed information on them, as such information will date rapidly. The fast-changing nature of the computing world means that new packages, and new versions of existing packages, continue to come out at ever-decreasing intervals. Thus we limit ourselves to some general remarks on good software, some general cautionary remarks and a listing of some important packages. Whatever software is used, the old adage that: GARBAGE IN GARBAGE OUT still applies today, perhaps even more so, as software gets more and more sophisticated. Desirable features of ‘good’ time-series software include: • Flexible facilities for entering and editing data. • Good facilities for exploring data and producing a good clear time plot. • Technically sound and computationally efficient algorithms. • Clear, self-explanatory output. • Clear documentation and easy to use options. Packages come in all shapes and sizes. Some are written for the expert statistician, some for the novice and some purport to be for both, though it is hard to write a program that will satisfy all types of users at the same

< previous page

file:///C:/Documents and Settings/Yang/

page_278

next page >

面/The analysis of time series an introduction/files/page_278.html [5/24/2009 16:54:07]

page_279

< previous page

page_279

next page >

Page 279 time. There is a real danger that software can be misused by the novice (and even by the expert!) and software needs to be written in such a way as to avoid misuse wherever possible. Some packages are still command driven but most software is now menu driven, and written for a personal computer (PC). This can allow more misuse, as the user can select a silly choice from the available list. The examples in Section 14.3 were mostly carried out using the MINITAB package. This easy-to-use interactive package performs most of the usual statistical tasks (including regression and ANOVA) as well as time-series analysis. In addition to computing time plots and seasonal and non-seasonal differences, MINITAB will calculate the ac.f. and partial ac.f. of a single series and the cross-correlation function for two series. MINITAB also fits ARIMA and seasonal ARIMA (SARIMA) models. Further details are given in Appendix D. MINITAB does not (at the time of writing) have special commands to carry out spectral analysis, though macros could be written to do so. The package is particularly suitable for teaching purposes, but it is also suitable for intermediate-level problems. In addition to MINITAB, most general-purpose statistical packages now offer time-series options, which typically include making a time plot, calculating the ac.f. and the spectrum, and fitting ARIMA models. They include GENSTAT, SPSS, SAS and STATGRAPHICS. The S-PLUS language, and its free look-alike called R, offer a variety of time-series modules, including ARIMA and SARIMA model fitting and spectral analysis. It also offers good graphics capabilities and has been used to produce most of the time plots in this book, as well as the ac.f. in Figure 14.3. However, it is not always easy to use, as the example in Section 14.4 demonstrates. There are also many specialized time-series packages, including AUTOBOX for Box-Jenkins modelling, FORECASTPRO for exponential smoothing and forecasting, BATS for Bayesian forecasting, STAMP for structural modelling, X-12-ARIMA for combining the X-12 seasonal adjustment method with ARIMA modelling, RATS for fitting regression and ARIMA models, MTS for transfer function and VARMA modelling, EViews for VAR and econometric modelling, TSP for VAR and regression modelling and SCA for more advanced time-series analysis including vector ARMA modelling. Many other packages are available and it is difficult keeping up-to-date. The reader is advised to read software reviews such as those in the International Journal of Forecasting and the American Statistician. A comprehensive listing of forecasting software up to 1999 is given by Rycroft (1999). As a final comment on time-series computing, the reader is warned that this is one area of statistics where different packages may not give exactly the same answer to what is apparently the same question. This may be due to different choices in the way that starting values are treated. For example, when fitting an AR(1) model, conditional least squares estimation treats the first observation as fixed, while full maximum likelihood does not. More generally, the choice of algorithm can make a substantial difference as

< previous page

file:///C:/Documents and Settings/Yang/

page_279

next page >

面/The analysis of time series an introduction/files/page_279.html [5/24/2009 16:54:08]

page_280

< previous page

page_280

next page >

Page 280 Table 14.1 Average monthly air temperatures at Recife for 1953–1962 Year Jan Feb Mar Apr May June July Aug Sept Oct Nov Dec 1953 26.8 27.2 27.1 26.3 25.4 23.9 23.8 23.6 25.3 25.8 26.4 26.9 1954 27.1 27.5 27.4 26.4 24.8 24.3 23.4 23.4 24.6 25.4 25.8 26.7 1955 26.9 26.3 25.7 25.7 24.8 24.0 23.4 23.5 24.8 25.6 26.2 26.5 1956 26.8 26.9 26.7 26.1 26.2 24.7 23.9 23.7 24.7 25.8 26.1 26.5 1957 26.3 27.1 26.2 25.7 25.5 24.9 24.2 24.6 25.5 25.9 26.4 26.9 1958 27.1 27.1 27.4 26.8 25.4 24.8 23.6 23.9 25.0 25.9 26.3 26.6 1959 26.8 27.1 27.4 26.4 25.5 24.7 24.3 24.4 24.8 26.2 26.3 27.0 1960 27.1 27.5 26.2 28.2 27.1 25.4 25.6 24.5 24.7 26.0 26.5 26.8 1961 26.3 26.7 26.6 25.8 25.2 25.1 23.3 23.8 25.2 25.5 26.4 26.7 1962 27.0 27.4 27.0 26.3 25.9 24.6 24.1 24.3 25.2 26.3 26.4 26.7 McCullough (1998) demonstrates when comparing Yule-Walker estimates of partial autocorrelation coefficients with maximum likelihood. Fortunately, the resulting differences are usually small, especially for long series. However, this is not always the case as demonstrated by the rather alarming examples in Newbold et al. (1994). Thus the forecaster would be wise to give much more attention to the choice of software, as recommended persuasively by McCullough (2000). 14.3 Examples I have selected three examples to give the reader further guidance on practical time-series analysis. In addition to the examples given below there are several more scattered through the text, including three forecasting problems in Section 5.5 and a detailed spectral analysis example in Section 7.8. Many other examples can be found throughout the time-series literature. Example 14.1 Monthly air temperature at Recife. Table 14.1 shows the air temperature at Recife in Brazil, in successive months over a 10-year period. The objective of our analysis is simply to describe and understand the data. In fact this set of data has been used as an illustrative example throughout the book and it is convenient to bring the results together here. First, we plot the data as in Figure 1.2. The graph shows a regular seasonal variation with little or no trend. This is exactly what one would expect a priori. The correlogram of the raw data in Figure 2.4(a) shows little apart from the obvious seasonal variation, with high positive autocorrelations at lags 12, 24…. Likewise the spectrum of the raw data in Figure 7.5(a) shows a large peak at a frequency of one cycle per year, which corresponds to the seasonal variation. As the seasonal variation constitutes such a large proportion of the total variation (about 85%), the analysis of the raw data tells us very little that is not obvious in the time plot.

< previous page

file:///C:/Documents and Settings/Yang/

page_280

next page >

面/The analysis of time series an introduction/files/page_280.html [5/24/2009 16:54:08]

page_281

< previous page

page_281

next page >

Page 281 In order to look more closely at the non-seasonal variation, it is strongly recommended that the seasonal variation is first removed from the data. This can be done by seasonal differencing (see Section 4.6), or simply by calculating the overall January average, the overall February average and so on; and then subtracting each individual value from the corresponding monthly average. The latter procedure is recommended when there is no trend (as here) and the seasonal variation is thought to be roughly constant from year to year. The correlogram of the seasonally adjusted data was presented in Figure 2.4(b). The graph shows that the first three coefficients are significantly greater than zero. This indicates some short-term correlation in that a month that is, say, warmer than average for that time of year is likely to be followed by two or three further months that are warmer than their corresponding average. The periodogram of the seasonally adjusted data is shown in Figure 7.5(c) and exhibits wild fluctuations as is typical of a periodogram. Nothing can be learned from this graph. In order to estimate the underlying spectrum of the seasonally adjusted data, we can proceed by smoothing the periodogram or by computing a weighted transform of the sample autocovariance or autocorrelation functions. We used the latter approach with a Tukey window to produce Figure 7.5(b). This shows something of a peak at zero frequency, indicating the possible presence of a trend or some other low-frequency variation. There is nothing else worth noting in the graph and the spectral analysis has really not helped much in this case. We summarize this analysis by saying that the series shows little or no trend, and that the major source of variation is the annual temperature cycle. There is some short-term correlation, but the analysis has otherwise been rather negative in that we have found little of interest apart from what is evident in the time plot. It is, in fact, a fairly common occurrence in Statistics to find that simple descriptive techniques provide the most helpful information, and this re-emphasizes the importance of starting any time-series analysis by drawing a time plot. Example 14.2 Yield on short-term government securities. Table 14.2 shows the percentage yield on British short-term government securities in successive months over a 21-year period. The problem is to find an appropriate model for the data and hence compute forecasts for up to 12 months ahead. The first step, as always, is to construct a time plot of the data as in Figure 14.1. The graph shows some interesting features. There is no discernible seasonal variation, but we note a marked trend in the data from about 2% at the start of the series to more than 7% at the end. However, the trend is by no means regular, and it would be a gross oversimplification to fit a (deterministic) straight line to the series. Indeed subsequent values have varied widely from as high as 15% to the current (2003) low value below 4%. Thus it would be inappropriate to fit a model, or use a forecasting method that assumes seasonality or a global trend. In particular, linear regression on time is not recommended.

< previous page

file:///C:/Documents and Settings/Yang/

page_281

next page >

面/The analysis of time series an introduction/files/page_281.html [5/24/2009 16:54:09]

page_282

< previous page

page_282

next page >

Page 282 Table 14.2 Yield (%) on British short-term government securities in successive months Year Jan Feb Mar Apr May June July Aug Sept Oct Nov Dec 1 2.22 2.23 2.22 2.20 2.09 1.97 2.03 1.98 1.94 1.79 1.74 1.86 2 1.78 1.72 1.79 1.82 1.89 1.99 1.89 1.83 1.71 1.70 1.97 2.21 3 2.36 2.41 2.92 3.15 3.26 3.51 3.48 3.16 3.01 2.97 2.88 2.91 4 3.45 3.29 3.17 3.09 3.02 2.99 2.97 2.94 2.84 2.85 2.86 2.89 5 2.93 2.93 2.87 2.82 2.63 2.33 2.22 2.15 2.28 2.28 2.06 2.54 6 2.29 2.66 3.03 3.17 3.83 3.99 4.11 4.51 4.66 4.37 4.45 4.58 7 4.58 4.76 4.89 4.65 4.51 4.65 4.52 4.52 4.57 4.65 4.74 5.10 8 5.00 4.74 4.79 4.83 4.80 4.83 4.77 4.80 5.38 6.18 6.02 5.91 9 5.66 5.42 5.06 4.70 4.73 4.64 4.62 4.48 4.43 4.33 4.32 4.30 10 4.26 4.02 4.06 4.08 4.09 4.14 4.15 4.20 4.30 4.26 4.15 4.27 11 4.69 4.72 4.92 5.10 5.20 5.56 6.08 6.13 6.09 5.99 5.58 5.59 12 5.42 5.30 5.44 5.32 5.21 5.47 5.96 6.50 6.48 6.00 5.83 5.91 13 5.98 5.91 5.64 5.49 5.43 5.33 5.22 5.03 4.74 4.55 4.68 4.53 14 4.67 4.81 4.98 5.00 4.94 4.84 4.76 4.67 4.51 4.42 4.53 4.70 15 4.75 4.90 5.06 4.99 4.96 5.03 5.22 5.47 5.45 5.48 5.57 6.33 16 6.67 6.52 6.60 6.78 6.79 6.83 6.91 6.93 6.65 6.53 6.50 6.69 17 6.58 6.42 6.79 6.82 6.76 6.88 7.22 7.41 7.27 7.03 7.09 7.18 18 6.69 6.50 6.46 6.35 6.31 6.41 6.60 6.57 6.59 6.80 7.16 7.51 19 7.52 7.40 7.48 7.42 7.53 7.75 7.80 7.63 7.51 7.49 7.64 7.92 20 8.10 8.18 8.52 8.56 9.00 9.34 9.04 9.08 9.14 8.99 8.96 8.86 21 8.79 8.62 8.29 8.05 8.00 7.89 7.48 7.31 7.42 7.51 7.71 7.99 With a single, important series like government securities, it is worth investing some time and effort on the analysis. The presence of short-term correlation in the time plot suggests that the Box-Jenkins method is worth a try, and that is what we will do. From the time plot, the given time series is clearly non-stationary. Thus some sort of differencing is required in order to start the Box-Jenkins modelling procedure. To confirm this, we computed the sample ac.f. of the raw data and found, as expected, that the coefficients at low lags were all ‘large’ and positive (as in Figure 2.3), and did not come down quickly towards zero. The coefficients at lags 1 to 5 are 0.97, 0.94, 0.92, 0.89 and 0.85, while the value at lag 24 was still as high as 0.34, with all intervening values being positive. The simplest type of differencing is to take first differences and this was done next. The ac.f. of the first differences turned out to have a very simple form. The coefficient at lag 1 is 0.31, which is significantly different from zero. Here we use the rule of thumb that values exceeding are significantly different from zero, where N is the number of terms in the differenced series. Here this number is (252–1). We also found that all remaining coefficients up to lag 20 were not significantly different from zero.

< previous page

file:///C:/Documents and Settings/Yang/

page_282

next page >

面/The analysis of time series an introduction/files/page_282.html [5/24/2009 16:54:10]

page_283

< previous page

page_283

next page >

Page 283

Figure 14.1 Yield (%) on British short-term government securities in successive months over 21 years. The above analysis tells us two things. First, the differenced series is now stationary, and no further differencing is required. Second, the fact that there is only one significant coefficient (at lag 1) tells us that a very simple ARIMA model will suffice. The ac.f. of a first-order MA process has the same general form as the observed ac.f. for the first differences of these data. This suggests fitting an ARMA(0, 1) model to the first differences or equivalently, an ARIMA(0, 1, 1) model to the original observed data. The model was fitted with the MINITAB package, although many other packages could have been used. We found Given the observed trend in the data, we also tried adding a constant to the right-hand side of the above model. However, the resulting residual sum of squares was not much smaller and the estimated constant was not significantly different from zero. This may be surprising to the reader at first sight, but variation due to the trend is ‘small’ compared with other sources of variation. If we did leave the constant in, we would effectively be adding a global trend to the model and we have already seen that this would not work well on later data.

< previous page

file:///C:/Documents and Settings/Yang/

page_283

next page >

面/The analysis of time series an introduction/files/page_283.html [5/24/2009 16:54:11]

page_284

page_284

< previous page

next page >

Page 284 We are now in a position to compute forecasts. Suppose we have data up to time N, say x1, x2,…, xN. Then, . At time N, we know the observed value using the model, we have of XN, namely, xN, but ZN+1 is unknown and best forecasted as zero, while ZN is best estimated by the residual at time N. The latter cannot be written out explicitly in terms of the observations but rather is readily . Thus the best forecast of XN+1 at

computed as the model is fitted iteratively, by time N is

. Forecasting two steps ahead, we find

. The last two terms are forecasted as zero, while the first term is forecasted as N(1). Thus the two-steps-ahead forecast is the same as the one-step forecast. In fact, forecasts for 3, 4,… steps ahead are all the same. The difference between N(1) and xN+1 is the residual at time (N+1), namely, , which can in turn be used to compute forecasts from time origin (N +1). It is interesting to note that the above forecasting procedure is equivalent to simple exponential smoothing (SES). In Section 5.2.2, we saw that SES is optimal for an ARIMA(0, 1, 1) model, which is the model fitted above. Thus it appears that SES could have been used for our data, provided that the smoothing parameter is chosen optimally to correspond to the MA parameter in the model fitted above. However, SES is actually implemented rather differently, particularly in regard to starting values and so the forecasts produced by SES would actually differ somewhat from those produced by the Box-Jenkins approach. In any case, the latter leads to a forecasting method that is more satisfying for the user in this case. This example exhibits one of the most straightforward Box-Jenkins analyses that the reader is likely to come across. The only simpler class of time series are those that behave like a random walk. For example, many series of share prices are such that the raw data are clearly non-stationary, but the first differences are found to behave like a completely random series. In other words, the correlogram of the first differences is such that none of the autocorrelation coefficients are significantly different from zero. This means that the original series can be modelled by an ARIMA(0, 1, 0) model, which is a random walk. The next example presents a much more difficult Box-Jenkins analysis. Example 14.3 Airline passenger data. Table 14.3 shows the monthly totals of international airline passengers in the U.S. from January 1949 to December 1960. This famous set of data has been analysed by many people including Box and Jenkins (1970, Chapter 9) and is freely available on the Web in the StatLib index (see Section 14.6 for details). The problem is to fit a suitable model to the data and produce forecasts for up to 1 year ahead. As always, we begin by plotting the data as in Figure 14.2. This shows a clear seasonal pattern as well as an upward trend. The magnitude of the seasonal variation increases at the same sort of rate as the yearly mean levels, and this indicates that a multiplicative seasonal model is appropriate.

< previous page

file:///C:/Documents and Settings/Yang/

page_284

next page >

面/The analysis of time series an introduction/files/page_284.html [5/24/2009 16:54:12]

page_285

< previous page

page_285

next page >

Page 285 Table 14.3 Monthly totals (Xt), in thousands, of international airline passengers in the U.S. from January 1949 to December 1960 Year Jan Feb Mar Apr May June July Aug Sept Oct Nov Dec 1949 112 118 132 129 121 135 148 148 136 119 104 118 1950 115 126 141 135 125 149 170 170 158 133 114 140 1951 145 150 178 163 172 178 199 199 184 162 146 166 1952 171 180 193 181 183 218 230 242 209 191 172 194 1953 196 196 236 235 229 243 264 272 237 211 180 201 1954 204 188 235 227 234 264 302 293 259 229 203 229 1955 242 233 267 269 270 315 364 347 312 274 237 278 1956 284 277 317 313 318 374 413 405 355 306 271 306 1957 315 301 356 348 355 422 465 467 404 347 305 336 1958 340 318 362 348 363 435 491 505 404 359 310 337 1959 360 342 406 396 420 472 548 559 463 407 362 405 1960 417 391 419 461 472 535 622 606 508 461 390 432

Figure 14.2 The Box-Jenkins airline data, with monthly totals in thousands of the numbers of international airline passengers in the U.S. from January 1949 to December 1960.

< previous page

file:///C:/Documents and Settings/Yang/

page_285

next page >

面/The analysis of time series an introduction/files/page_285.html [5/24/2009 16:54:12]

page_286

< previous page

page_286

next page >

Page 286 It is relatively straightforward to use the Holt-Winters forecasting procedure on these data. This procedure can cope with trend and with additive or multiplicative seasonality (see Section 5.2.3). The details of the analysis will not be given here, as no particular difficulties arise and reasonable forecasts result. The reader who prefers a nice simple procedure is advised to use this method for these data. Following Box and Jenkins (1970) and other authors, we will also try to fit an ARIMA model and consider the many problems that arise. The first question to consider is whether or not to transform the data. The question of transformations is discussed in Section 2.4, and, in order to make the seasonal effect additive, it looks as though we should take logarithms of the data. Box and Jenkins (1970, p. 303) also took logarithms of the data. To justify this, they simply say that ‘logarithms are often taken before analysing sales data, because it is percentage fluctuation that might be expected to be comparable at different sales volumes.’ This seems rather unconvincing, and the long discussion of Chatfield and Prothero (1973) indicates some of the problems involved in transformation. Nevertheless, it seems fairly clear that we should take logarithms of the data in this example, and so we shall proceed with the analysis on this assumption. It is easy to plot the logged data using MINITAB, which shows that the size of the seasonal variation is now roughly constant. We now use MINITAB to plot the ac.f. of the raw logged data and of various differenced series in order to see what form of differencing is required. The first 36 coefficients of the ac.f. of the logged data are all positive as a result of the obvious trend in the data. Some sort of differencing is clearly required. With monthly seasonal data, the obvious operators to try are , 12, 12, and . Letting Yt(=log Xt) denote the logarithms of the observed data, we start by looking at the ac.f. of Yt. With N observations in the differenced series, a useful rule of thumb for deciding if an autocorrelation coefficient is significantly different from zero is to see whether its modulus exceeds . Here the critical value is 0.17 and we find significant coefficients at lags 1, 4, 8, 11, 12, 13, 16, 20, 23, 24, 25 and so on. There is no sign that the ac.f. is damping out, and further differencing is required. For 12Yt the first nine coefficients are all positive and significantly different from zero, after which there is a long run of negative coefficients. The series is still non-stationary and so we next try 12Yt. The correlogram of this series is shown in Figure 14.3. I actually looked at the first 38 coefficients—over 3 seasonal cycles—to be on the safe side, but only need plot the first 21. The number of terms in the differenced series is now 144−12−1=131, and an approximate critical value is . We note ‘significant’ values at lags 1 and 12, as well as the marginal value at lag 3. Most of the other coefficients are ‘small’ and there is now no evidence of nonstationarity. Thus we choose to fit an ARIMA model to 12Yt.

< previous page

file:///C:/Documents and Settings/Yang/

page_286

next page >

面/The analysis of time series an introduction/files/page_286.html [5/24/2009 16:54:13]

page_287

< previous page

page_287

next page >

Page 287

Figure 14.3 The correlogram of 12logeXt. Values outside the dotted lines are significantly different from zero. The way that we have chosen the differencing operator is somewhat subjective, and some authors prefer a more ‘objective’ approach. One possibility is to choose the differencing operator so as to minimize the variance of the differenced series, but we did not try this here. In order to identify a suitable ARMA model for 12Yt. it may help to calculate the partial ac.f. as well as the ac.f. (An alternative is to use the inverse ac.f.—see Section 11.4-but this is not available in MINITAB.) The partial ac.f. produced ‘significant’ coefficients (whose moduli exceed 0,18 as for the ac.f.) at lags 1, 3, 9 and 12. When ‘significant’ values occur at unusual lagged values, such as 9, they are usually ignored unless there is external information as to why such a lag should be meaningful. We should now be in a position to identify an appropriate seasonal ARIMA model to fit to the data. This means that we want to assess values of p, q, P, Q in the model defined by Equation (4.16). The difference parameters d=1 and D=1 have already been chosen by the choice of differencing operator. The seasonal parameters P and Q are assessed by looking at the values of the ac.f. and partial ac.f. at lags 12, 24, 36,…. In this case the values are ‘large’ at lag 12 but ‘small’ at lags 24 and 36, indicating no AR terms but one seasonal MA term. Thus we take P=0 and Q=1. The values of the non-seasonal values p and q are assessed by looking at the first few values of the ac.f. and partial

< previous page

file:///C:/Documents and Settings/Yang/

page_287

next page >

面/The analysis of time series an introduction/files/page_287.html [5/24/2009 16:54:14]

page_288

< previous page

page_288

next page >

Page 288 ac.f. The only ‘significant’ values are at lags 1 and 3, and these values are not easy to interpret. An AR(3) model, as suggested by the partial ac.f., would generally have a slowly decaying ac.f., while the MA(3) model, suggested by the ac.f., would generally have a slowly decaying partial ac.f. Noting that the coefficients at lag 3 are only just ‘significant’, we could, as a first try, take just one moving average term and set p=0 and q=1. If we now work out the standard error of the autocorrelation coefficient at lag 3 using a more exact formula, we find that it is not in fact significantly different from zero. This more exact formula (Box and Jenkins, 1970, p. 315) assumes that an MA(1) model is appropriate rather than a completely is appropriate. This result gives us more reliance on the choice random series for which the formula of p=0 and q=1. Thus we now use MINITAB to fit a seasonal ARIMA model with p=0, d=1, q=1 and P=0, D=1, Q=1. Setting

the fitted model turns out to be

MINITAB readily provides estimates of the residual variation as well as a variety of diagnostic checks. For example, one should inspect the ac.f. of the residuals to see that ‘none’ of the coefficients is significantly different from zero (but remember that about 1 in 20 coefficients will be ‘significant’ at the 5% level under the null hypothesis that the residuals are random). One can also plot the observed values in the time series against the one-step-ahead forecasts of them. In this example there is no evidence that our fitted model is inadequate and so no alternative models will be tried. MINITAB will readily provide forecasts of the logged series for up to 12 months ahead. The first three forecasts are 6.110, 6.056 and 6.178. These values need to be antilogged to get forecasts for the original series. I hope that this example has given some indication of the type of problem which may arise in a Box-Jenkins ARIMA analysis and the sort of subjective judgement that has to be made. A critical assessment of the method has been made in Section 5.4. The method sometimes works well, as in Example 14.2. However, for the airline data, the variation is dominated by trend and seasonality and yet the complicated ARMA modelling is carried out on the differenced data from which most of the trend and seasonality have been removed. Readers must decide for themselves whether they think the method has been a success here. In my view, a trend-and-seasonal modelling approach (such as Holt-Winters) would not only be easier to apply but also give more understanding of this particular set of data. For the reader’s assistance, I have given the MINITAB commands used in the above analysis in Appendix D. I also worked through the analysis using S-PLUS, getting much the same results. Figures 14.2 and 14.3 are produced by S-PLUS. An example of S-PLUS code to produce a time plot is given in Appendix D.

< previous page

file:///C:/Documents and Settings/Yang/

page_288

next page >

面/The analysis of time series an introduction/files/page_288.html [5/24/2009 16:54:15]

page_289

< previous page

page_289

next page >

Page 289

Figure 14.4 The analysis of U.K. monthly airline passenger numbers from 1949–1997. (a) raw data; (b) transformed data; (c) estimated trend for transformed data; (d) estimated seasonality for transformed data. It may be helpful to follow up the above analysis by analysing a similar, but much longer, series. The series in Table 14.3 was collected by the U.S. Federal Aviation Authority (FAA), but a much longer series is available from the British equivalent, namely, the Civil Aviation Authority (CAA). Grubb and Mason (2001) have analysed this series of monthly total U.K. airline passenger numbers from 1949 to 1997. A plot of the raw data is shown in Figure 14.4(a). Grubb and Mason noted the strong growth and the approximately multiplicative seasonality and decided to use Holt-Winters forecasting. They transformed the data by applying the Box-Cox transformation (see Section 2.4) with estimated parameter λ=0.36—close to a cube root transformation. The transformed series is shown in Figure 14.4(b). The local Holt-Winters estimates of level are shown in Figures 14.4(a) and (b), while Figures 14.4(c) and (d) show estimated trend and seasonality for the transformed series.

< previous page

file:///C:/Documents and Settings/Yang/

page_289

next page >

面/The analysis of time series an introduction/files/page_289.html [5/24/2009 16:54:16]

page_290

< previous page

page_290

next page >

Page 290 Grubb and Mason then produced long-term forecasts of the series, both with standard Holt-Winters and with a modified Holt-Winters where forecasts were damped, not towards zero as in the usual Gardner-McKenzie damping (see Section 5.2.3), but towards the overall average trend. They also computed forecasts using SARIMA modelling and the Basic Structural Model. As the series is so long, it was possible to split the given series in two, fit the model to the first part and then forecast the latter part in a genuine out-of-sample way. The authors found that the standard Holt-Winters forecasts were not quite so good as SARIMA or structural forecasts but that the modified Holt-Winters forecasts were better. The interested reader is recommended to read the cited paper. 14.4 More on the Time Plot The three examples in Section 14.3 have highlighted the need for a good, clear time plot to start any timeseries analysis. Some simple, general guidelines for drawing clear time plots were given briefly in Section 2.3. They may be expanded as follows: • Give a clear, self-explanatory title • State the units of measurement • Choose the scales carefully, including the size of the intercept • Label the axes clearly • Choose the way the points are plotted very carefully. This includes the choice of plotting symbol for each point (e.g. ·, *, or +) and whether points should be joined by a straight line. The analyst needs to exercise care and judgement in applying these rules, both in graphs drawn by hand or (more usual today) in choosing appropriate options to produce graphs on a computer. This section takes a more detailed look at the construction of time plots and gives an example of a poor graph which could readily be improved. Some useful books that cover this material are Cleveland (1994) and Chatfield (1995a, Section 6.5.3). As well as choosing the scales carefully, the overall shape of a graph needs careful attention. Some general advice, which is often given, is that the slope of a graph should preferably be between about 30° and 45° to give the best display of a relationship. This applies, for example, to Figure 14.1 where the overall trend is about 30° to 35°. However, for seasonal or cyclic time series, it is not always obvious how to apply the guideline. Thus in Figure 14.2 the slope of the trend is under 30°, while the slope of the within-year seasonal variation is much larger than 45° in later years. It may be that more than one graph is necessary to display different features of a series as in Figure 11.1. Do not be afraid to adopt a trial-and-error approach. With a good computer package, it should be easy to try out different choices for a time plot in order to see which graph is ‘best’ for the purpose intended. In particular, the analyst should be able to try different choices of scale and

< previous page

file:///C:/Documents and Settings/Yang/

page_290

next page >

面/The analysis of time series an introduction/files/page_290.html [5/24/2009 16:54:16]

page_291

< previous page

page_291

next page >

Page 291 different intercepts. The software should also allow different plotting symbols to be tried, and different options for joining up successive points. Sadly, many packages still give the analyst little or no control over the output, while others provide options that are not easy to use.

Figure 14.5 (A poor graph of) The Recife temperature data. Figure 14.5 gives an alternative presentation of the Recife temperature data plotted earlier in Figure 1.2. This has been deliberately drawn to illustrate some common graphical mistakes. The title does not describe the graph in anything like enough detail. Sadly, this is the sort of title that often appears in published graphs. The reader has to check the accompanying text but may still not be able to find out exactly what the graph represents. The vertical axis is simply labelled X. The reader will therefore not know what the variable is, nor what the units of measurement are. The horizontal scale is labelled imprecisely as Time’ and the ‘ticks’ are labelled by month and year. However, the distance between successive ticks is poorly chosen. In order to assess the seasonal variation, the horizontal scale needs to be marked in years or in multiples of 12 months. If your computer software produces a graph like Figure 14.5, do not be satisfied with it but rather take steps to compute an improved version of it using different graphical options or using a different package. Figure 14.5 was actually produced by the S-PLUS package using the default labelling scheme. It has to be said that this is a disappointing default choice, especially given the rather complex nature of the specialized options for labelling axes. Figure

< previous page

file:///C:/Documents and Settings/Yang/

page_291

next page >

面/The analysis of time series an introduction/files/page_291.html [5/24/2009 16:54:17]

page_292

< previous page

page_292

next page >

Page 292 1.2 was also produced by S-PLUS, but using some of the graphical options available, and the necessary SPLUS-code is listed in Appendix D. 14.5 Concluding Remarks I hope the examples in this chapter have illustrated some of the practical difficulties involved in time-series analysis. They include the subjective assessment of correlograms and spectra (not easy), decisions on whether to transform and/or difference the data, and decisions on a choice of model. The choice of an appropriate analysis procedure depends in part on the objectives, which depend in turn on the particular practical context. A clear formulation of the problem, including a careful statement of objectives, is vital. The possible use of ancillary information should also be considered, although it has not been possible to incorporate it into the examples. Example 14.2, for example, demonstrates how to construct a univariate model for the yield on short-term government securities, but this takes no account of other economic variables. As discussed in Chapter 5, it is not always possible to use such additional information in an explicit way in a statistical model, but neither should the information be ignored. If the fitted model is used to produce forecasts, it may, for example, be advisable to adjust them subjectively if it is known that some special event, such as a change in interest rates, is imminent. Alternatively, it may be worth contemplating the construction of a multivariate model. This is just one example of how the analyst should always be prepared to apply common sense within a given practical context in order to carry out a successful timeseries analysis (or indeed any statistical investigation). 14.6 Data Sources and Exercises In order to fully appreciate the concepts of time-series analysis, it is essential for the reader to ‘have a go’ at analysing and modelling some real time-series data. This should preferably be some data known to the reader so that the context is understood. If this is not possible, then there are numerous data sets available electronically and so there is no point in tabulating further series here. A large number of data sets, including many time series, are available in the Datasets Archive in the StatLib Index, which can be accessed via the World-Wide-Web (WWW) at: http://lib.stat.cmu.edu/ This gives access to numerous sets of data that can readily be copied back to the analyst’s home computer. They include, for example, all the data listed in Andrews and Herzberg (1985), which can be obtained by ‘clicking’ on the data set called Andrews. This data set includes the monthly sunspots data and the annual Canadian lynx trapping data referred to in Section 11.1.1 to name but two series. The StatLib Index also includes many other time

< previous page

file:///C:/Documents and Settings/Yang/

page_292

next page >

面/The analysis of time series an introduction/files/page_292.html [5/24/2009 16:54:17]

page_293

< previous page

page_293

next page >

Page 293 series including some 300 series in a data set called hipel-mcleod and the time series used in Percival and Walden (1993) in a data set called sapa. The lynx and sunspots series are also available in the data sets supplied with the S– PLUS package. Many other packages, such as MINITAB, also incorporate files containing a variety of data sets in such a way as to make them readily available to the user. Additional data sets can also be accessed in other ways via the Internet. Several books provide specimen data sets, both in tabular form and on a floppy disc. For example, Hand et al. (1994) list 510 data sets and include a data disk containing all the data listed in their book. Some time series are included, among them being the lynx data and the annual (but not the monthly) sunspot numbers. Most of the series analysed in this book are available via my Website http://www.bath.ac.uk/~mascc/ at the University of Bath, and are also in the StatLib index. It is difficult to set realistic practical exercises here. I suggest the reader might like to start by (re-)analysing the airline data (see Table 14.3) and working through Example 14.3. This analysis can then be readily extended in various ways. The data set is rather unusual because the operator 12 acting on the raw (as opposed to the logged) data actually looks rather more stationary than when applied to the logged data. Check to see whether you agree. The ac.f. may suggest the (very simple) seasonal ARIMA (SARIMA) model of order (0,1,0)×(0,1,0)12. Fit this model using MINITAB (or some other suitable package) and examine the residuals. Do any other SARIMA models look worth trying? A standard procedure for removing a trend and multiplicative seasonal effect is to use the operator rather than 12 Apply this operator and see whether you get a stationary series. Compare the out-of-sample forecasts you get of, say, the last year’s data by using your preferred SARIMA model for the raw and for the logged data. (Calculate the mean absolute forecast error or the sum of squares of the forecast errors according to taste). You will of course have to back-transform the forecasts of the logged data in order to make this comparison. Remember that for forecasts to be ‘fair’, they should not use the data to be forecast when fitting the model. Thus all your models should be constructed without using the last year’s data if the latter are to be used in your forecasting competition. This exercise is by its nature rather open-ended as will normally be the case when modelling real data. It illustrates the difficulty of modelling when the structure of the model is not assumed a priori. The reader should now move on to analyse some new data, either using data with a known background or using the data sources given above. It may be wise to start with a non-seasonal series, such as that listed in Table 14.2 and analysed in Example 14.2. The identification of a model for non-seasonal series is easier than for a seasonal series. The only operators that need to be considered are first differences, , and possibly second-order differences, . Whatever analyses you do, always start by plotting the data. Look for trend, seasonality, outliers, errors and missing observations. If you are trying to fit an ARIMA model, look at the ac.f. and the partial ac.f. of the original series

< previous page

file:///C:/Documents and Settings/Yang/

page_293

next page >

面/The analysis of time series an introduction/files/page_293.html [5/24/2009 16:54:18]

page_294

< previous page

page_294

next page >

Page 294 and of various differenced series. Consider the possibility of transforming the data. Look at the within-sample fit but also look at (genuine out-of-sample) forecasts. Remember that more complicated models with more parameters will usually give a smaller (within-sample) residual sum of squares (where a residual is a onestep-ahead forecast error) but that this better fit may not be translated into better (out-of-sample) forecasts. A measure of fit such as Akaike’s information criterion (see Section 13.1) is needed to balance goodness-of-fit with the number of parameters.

< previous page

file:///C:/Documents and Settings/Yang/

page_294

next page >

面/The analysis of time series an introduction/files/page_294.html [5/24/2009 16:54:19]

page_295

page_295

< previous page

next page >

Page 295 APPENDIX A Fourier, Laplace and z-Transforms This appendix provides a short introduction to the Fourier transform, which is a valuable mathematical tool in time-series analysis. The related Laplace and z-transforms are also briefly introduced. Given a (possibly complex-valued) function h(t) of a real variable t, the Fourier transform of h(t) is usually defined as

provided the integral exists for every real H( ) to exist is that

(A.1) . Note that H( ) is in general complex. A sufficient condition for

If Equation (A.1) is regarded as an integral equation for h(t) given H( ), then a simple inversion formula exists of the form (A.2) Then h(t) is called the (inverse) Fourier transform of H( ). The two functions h(t) and H( ) are commonly called a Fourier transform pair. The reader is warned that many authors use a slightly different definition of a Fourier transform to Equation outside the integral in Equation (A.1), and then (A.1). For example, some authors put a constant the inversion formula for h(t) has a symmetric form. In time-series analysis many authors (e.g. Cox and Miller, 1968, p. 315) find it convenient to put a constant 1/2π outside the integral in Equation (A.1). In the inversion formula, the constant outside the integral is then unity. In time-series analysis, it is often convenient to work with the variable resulting Fourier transform pair is

rather than

. The

(A.3)

Note that the constant outside each integral is now unity. When working with discrete-time series, we typically use the discrete form

< previous page

file:///C:/Documents and Settings/Yang/

page_295

(A.4)

next page >

面/The analysis of time series an introduction/files/page_295.html [5/24/2009 16:54:20]

page_296

page_296

< previous page

next page >

Page 296 of the Fourier transform where h(t) is only defined for integer values of t. Then

is the discrete Fourier transform of h(t). Note that H( transform is

(A.5) ) is only defined in the interval [−π, π]. The inverse

(A.6) Fourier transforms have many useful properties, some of which are used during the later chapters of this book. However, we do not attempt to review them all here, as there are numerous mathematical books that cover the Fourier transform in varying depth. One special type of Fourier transform arises when h(t) is a real-valued even function such that h(t)=h(−t). The autocorrelation function of a stationary time series has these properties. Then, using Equation (A.1) with a constant 1/π outside the integral, we find

and it is easy to see that H(

(A.7)

) is a real-valued even function. The inversion formula is then

(A.8) Equations (A.7) and (A.8) are similar to a discrete Fourier transform pair and are useful when we only wish to define H( ) for >0. We have generally adopted the latter convention in this book. When h(t) is only defined for integer values of t, Equations (A.7) and (A.8) become

(A.9)

and H( ) is now only defined on [0, π]. The Laplace transform of a function h(t), which is defined for t>0, is given by

As compared with the Fourier transform, note that the lower limit of the

< previous page

file:///C:/Documents and Settings/Yang/

page_296

(A.10)

(A.11)

next page >

面/The analysis of time series an introduction/files/page_296.html [5/24/2009 16:54:20]

page_297

< previous page

page_297

next page >

Page 297 integral is zero, and not −∞, and that s is a complex variable, which may have real and imaginary parts. The integral converges when the real part of s exceeds some number called the ‘abscissa of convergence’. Given a function h(t), such that (A.12) then the Laplace and Fourier transforms of h(t) are the same, provided that the real part of s is zero. More generally, the Laplace transform is a generalization of the Fourier transform for functions defined on the positive real line. Control engineers often prefer to use the Laplace transform when investigating the properties of a linear system, as this will cope with physically realizable systems which are stable or unstable. The impulse response function of a physically realizable linear system satisfies Equation (A.12) and so, for such functions, the Fourier transform is a special case of the Laplace transform. More details about the Laplace transform may be found in many mathematics books. The z-transform of a function h(t) defined on the non-negative integers is given by

(A.13) where z is a complex variable. Comparing Equation (A.13) with (A.11) (or with (A.14) below), we see that the z-transform can be thought of as a discrete version of the Laplace transform, on replacing es by z. In discrete time, with a function satisfying Equation (A.12), some authors (e.g. Hayes, 1996, Section 2.2.5) prefer to use the z-transform rather than the discrete Fourier transform (i.e. Equation (A.5)) or the discrete form of the Laplace transform, namely

(A.14) All three transforms have somewhat similar properties, in that a convolution in the time domain corresponds to a multiplication in the frequency domain. The more advanced reader will observe that, when {h(t)} is a probability function such that h(t) is the probability of observing the value t, for t=0, 1,…, then Equation (A.13) is related to the probability generating function of the distribution, while Equations (A.5) and (A.14) are related to the moment generating and characteristic functions of the distribution. Exercises A.1 If h(t) is real, show that the real and imaginary parts of its Fourier transform, as defined by Equation (A.1), are even and odd functions, respectively. A.2 If h(t)=e−a|t| for all real t, where a is a positive real constant, show

< previous page

file:///C:/Documents and Settings/Yang/

page_297

next page >

面/The analysis of time series an introduction/files/page_297.html [5/24/2009 16:54:21]

page_298

page_298

< previous page

next page >

Page 298 that its Fourier transform, as defined by Equation (A.1), is given by A.3 Show that the Laplace transform of h(t)=e−at for (t>0), where a is a real constant, is given by where Re(s) denotes the real part of s.

< previous page

file:///C:/Documents and Settings/Yang/

page_298

next page >

面/The analysis of time series an introduction/files/page_298.html [5/24/2009 16:54:22]

page_299

< previous page

page_299

next page >

Page 299 APPENDIX B Dirac Delta Function Suppose that (t) is any function which is continuous at t=0. Then the Dirac delta function δ(t) is such that (B.1) Because it is defined in terms of its integral properties alone, it is sometimes called the ‘spotting’ function since it picks out one particular value of (t). It is also sometimes simply called the delta function. Although it is called a ‘function’, it is important to realize that δ(t) is not a function in the usual mathematical sense. Rather it is what mathematicians call a generalized function, or distribution. This maps a function,

(t) say, into the real line, by producing the value Some authors define the delta function by

(0).

(B.2)

such that

While this is often intuitively helpful, it is mathematically meaningless. The Dirac delta function can also be regarded as the limit, as ε→0, of a pulse of width ε and height 1/ε (i.e. having unit area) defined by

This definition is also not mathematically rigorous, but is heuristically useful. In particular, control engineers can approximate such an impulse by an impulse with unit area whose duration is short compared with the least significant time constant of the response to the linear system being studied. Even though δ(t) is a generalized function, it can often be handled as if it were an ordinary function except that we will be interested in the values of integrals involving δ(t) and never in the value of δ(t) by itself. The delta function has many useful properties and we have used δ(t) in Chapter 9 to analyse particular linear systems in continuous time.

< previous page

file:///C:/Documents and Settings/Yang/

page_299

next page >

面/The analysis of time series an introduction/files/page_299.html [5/24/2009 16:54:22]

page_300

< previous page

page_300

next page >

Page 300 Exercises B.1 The function

(t) is continuous at t=t0. If a

面/The analysis of time series an introduction/files/page_300.html [5/24/2009 16:54:23]

page_301

< previous page

page_301

next page >

Page 301 APPENDIX C Covariance and Correlation This book assumes knowledge of basic statistical topics such as the laws of probability, probability distributions, expectation and basic statistical inference (including estimation, significance testing and linear regression). Any reader who is unfamiliar with these topics should consult one of the numerous elementary texts covering this material. The topics of covariance and correlation are usually studied as part of elementary probability or linear regression. However, they are not always clearly understood at first, and are particularly important in the study of time series. Thus they will now be briefly revised. Suppose two random variables X and Y have means E(X)=µX, E(Y)= µY, respectively. Then the covariance of X and Y is defined to be (C.1)

and may be denoted γXY. If X and Y are independent, then

so that the covariance is zero. If X and Y are not independent, then the covariance may be positive or negative depending on whether ‘high’ values of X tend to go with ‘high’ or ‘low’ values of Y. Here high means greater than the appropriate mean. Covariance is a useful quantity for many mathematical purposes, but it is difficult to interpret, as it depends on the units in which X and Y are measured. Thus it is often useful to standardize the covariance between two random variables by dividing by the product of their respective standard deviations to give a quantity called the correlation coefficient. If we denote the standard deviations of X and Y by σX and σY, respectively, then the correlation of X and Y is defined by (C.2) and is typically denoted by ρXY. It can readily be shown that a correlation coefficient must lie between ±1 and is a useful measure of the linear association between two variables. Given N pairs of observations, {(xi, yi); i=1,…, N}, the usual estimate

< previous page

file:///C:/Documents and Settings/Yang/

page_301

next page >

面/The analysis of time series an introduction/files/page_301.html [5/24/2009 16:54:23]

page_302

< previous page

page_302

next page >

Page 302 of the covariance between two variables is given by

If we denote the sample variances of the two variables by

and

, where

then the usual estimate of the sample correlation coefficient is given by (C.3) This is the intuitive estimate of ρXY as defined above in Equation (C.2), and also agrees with Equation (2.2) in Chapter 2 on cancelling the denominators (N−1). The above definitions are applied to time series as described in Chapters 2 and 3. If the pair of random variables are taken from the same stochastic process so as to be separated by the same time-lag k say, then the covariance coefficient of Xt and Xt−k is called the autocovariance coefficient at lag k, and the corresponding correlation coefficient is called an autocorrelation coefficient. If the process is stationary, the standard deviations of Xt and Xt−k will be the same and their product will be the variance of Xt (or of Xt−k).

< previous page

file:///C:/Documents and Settings/Yang/

page_302

next page >

面/The analysis of time series an introduction/files/page_302.html [5/24/2009 16:54:24]

page_303

< previous page

page_303

next page >

Page 303 APPENDIX D Some MINITAB and S-PLUS Commands This appendix contains some notes on the use of MINITAB and S-PLUS as applied to some examples in the book. Introduction to MINITAB. MINITAB is a general-purpose, interactive statistical computing system, which is very easy to use. This appendix gives some guidance on the time-series options and gives sample commands for use in Example 14.3. More details can readily be found using the help command (see below). Recent versions (e.g. Version 12) are windows based with a spreadsheet style data window. In order to implement a particular procedure, you may need to use a series of actions using the drop-down menus (e.g. from the Stat heading) and several clicks may be needed to get a simple command. It is quicker to type the appropriate command if you know what it is, and so we present specimen commands, as they are much easier to describe in writing and are arguably easier to use in practice. Earlier versions of MINITAB, such as Version 9.1, were command-only versions, but the commands are still available in later versions. If no command prompt is visible in your Windows setup, then click on Editor at the top and then on ‘Enable command language’. Overview. Data are stored in up to 1000 columns, denoted by c1, c2,…, c1000, in a worksheet. It is also possible to store up to 100 matrices, denoted m1, m2,…, m100, and up to 1000 constants, denoted by k1, k2, …, k997 (k998-k1000 store special numbers like π). When you want to analyse data, you type the appropriate command. There are commands to read, edit and print data; to manipulate the columns of data in various ways; to carry out arithmetic; to plot the data; and to carry out a variety of specific statistical analyses. You can get lots of help from the package. The command help overview gives general advice. A command such as help set gives help on a particular command such as set Data may be entered with the SET, READ or INSERT command. For example, SET ‘filename’ c1 reads in a data set in a file called filename into a column denoted by c1. When you have finished, the command STOP exits you from MINITAB. Commands may be given in lower case or upper case or a mixture (e.g. LET or let or Let). The package only looks at the first four letters of a command (or fewer if command is less than four letters). Thus histogram and hist are equivalent. Many commands have subcommands to

< previous page

file:///C:/Documents and Settings/Yang/

page_303

next page >

面/The analysis of time series an introduction/files/page_303.html [5/24/2009 16:54:25]

page_304

< previous page

page_304

next page >

Page 304 increase their versatility. Then the main command must end with a semicolon and the subcommand with a full stop (or period). Time-series options include the following: acf c2—calculates the ac.f. of a single time series stored in c2 pacf c2—calculates the partial ac.f. diff 1 c2 c4—puts first differences of data in c2 into c4 diff 12 c2 c5—puts seasonal differences of c2 into c5, assuming data are monthly arim 1 1 0 c5—fits an ARIMA(1, 1, 0) model to data in c5 arim p d q, P D Q, s, c2—fits a seasonal ARIMA model to data in c2. s is season length e.g. 12 for monthly data. tsplot c3—time plot of c2. MINITAB commands for Example 14.3. I give the MINITAB commands used in the analysis of Example 14.3. We read the data into c1, and put the logged data into c3. The three series of differences of the transformed data are put into c6, c7 and c8. I hope that most of the commands are self-explanatory. If not, use help. The commands, written in italics, are: set c1—reads in data in rows of, say, twelve observations separated by a space or comma. The last row of the input must be END. Data go into a column denoted by c1. OR put the data in a datafile (outside the MINITAB package) called, say, airline and read by: set ‘airline’ c1 tsplot c1—gives a time plot. High resolution graphics should be used. let c3=log(c1) tsplot c3 acf 38 c3—calculates first 38 coefficients of the ac.f. of the logged data. difference lag 1 c3 c6—or can shorten to diff 1 c3 c6—calculates first differences of logged series. acf 38 c6 diff 12 c3 c7 acf 38 c7 diff 1 c7 c8 acf 38 c8—gives ac.f. of 12 log Xt. pacf c8—gives partial ac.f. arima (p=0 d=1 q=1)(P=0 D=1 Q=1) s=12 c3 res in c10 pred c11—or can shorten to arim 01 1 01 1 12 c3 c10 c11—this gives parameter estimates plus their standard deviations, and the residual SS and MS. It also looks at the ac.f. of the residuals by calculating various values of the modified Box-Pierce (Ljung-Box) chisquare statistic (see Section 4.7) to assess goodness of fit. Up to lag 24, for example, the statistic is 25.5 on 22 DF, which is not significant and so the model is not rejected. The command puts the residuals (=1-stepahead prediction errors) into c10 and the 1-step-ahead predictions into c11.

< previous page

file:///C:/Documents and Settings/Yang/

page_304

next page >

面/The analysis of time series an introduction/files/page_304.html [5/24/2009 16:54:25]

page_305

< previous page

page_305

next page >

Page 305 tsplot c10—plots the residuals. acf c10—gives ac.f. of residuals. The largest autocorrelation coefficient of the residuals up to lag 22 is −0.156 at lag 16, which is not significantly different from zero. dele 1:13 c11—deletes the first 13 values of the 1-step-ahead prediction errors in c11, which are all zero (because of differencing). copy c3 c23 dele 1:13 c23—deletes the first 13 values of the logged data. plot c23 c11—plots observed logged values against 1-step-ahead predictions excluding first 13 observations. arim 0 1 1 0 1 1 12 c3; forecast 12 c12.—Note the semicolon in the main command and the full stop (or period) in the subcommand. This punctuation must be included. This command is the same as the previous arima command except that it now calculates forecasts up to 12 steps ahead together with associated prediction intervals. The point forecasts are placed in c12. let c14=expo(c12) —exponentiates the forecasts so as to get forecasts of the raw, rather than logged data. prin c14—prints the forecasts up to 12 steps ahead. For example, the 1-step-ahead forecast is 450.3 and the 12-step-ahead forecast is 478.2. stop Some S-PLUS code for a time plot. Most of the time plots given in this book were produced using S-PLUS. While ‘nice’ graphs can be obtained with some effort, I have found the optional commands for labelling axes to be somewhat hard to use. As an example, I therefore give the code for producing Figure 1.2. The data file, called Recife, consists of 10 rows of 12 observations as in the body of Table 14.1. The S-code is given in italics, together with explanatory comments. Note that Latex will not print a quotation mark, as found on a standard typewriter, and so the left and right quotation marks found below should all be typed in S-PLUS using the same quotation mark. postscript(file=“1.2.ps”)—tells computer to put the resulting graph in a file called 1.2.ps. recife

Page 306 mid6_as. numeric(dates (“01/01/1959”, format=“dd/mm/yyyy”)) mid7_as.numeric(dates(“01/01/1960”, format=“dd/mm/yyyy”)) mid8_as.numeric(dates(“01/01/1961”, format=“dd/mm/yyyy”)) lst_as.numeric(dates(“01/01/1962”, format=“dd/mm/yyyy”))—sets position of last tick mark. ts.plot(RecfS, xaxt=“n”, ylab=“Temperature(deg C)”, xlab=“ZYear”, type=“l”)—generates the time plot. xaxt=“n” tells computer not to include the automatic labels for x-axis. type=“l” tells computer to join up points in time order. axis(side=1, at=c(fst, midl, mid2, mid3, mid4, mid5, mid6, mid7, mid8, lst), labels=c(“Jan 53”,“Jan 5”,“Jan 5”,“Jan 5”,“Jan 5”,“Jan 5”,“Jan 59”, “Jan 60”, “Jan 61”, “Jan 62”), ticks=T)—the axis command, spread over 3 lines, tells computer how to label x-axis. dev.off()—tells the computer to save the time plot in the specified file, 1.2.ps, rather than print it on screen.

< previous page

file:///C:/Documents and Settings/Yang/

page_306

next page >

面/The analysis of time series an introduction/files/page_306.html [5/24/2009 16:54:26]

page_307

< previous page

page_307

next page >

Page 307 Answers to Exercises Chapter 2 2.1 There are various ways of assessing the trend and seasonal effects. A simple method is to calculate the four yearly averages in 1995, 1996, 1997 and 1998; and also the average sales in each of periods I, II,…, XIII (i.e. calculate the row and column averages). The yearly averages provide a crude estimate of trend, while the differences between the period averages and the overall average estimate the seasonal effects. With such a small downward trend, this rather crude procedure may well be adequate for most purposes. It has the advantage of being easy to understand and compute. A more sophisticated approach would be to calculate a 13-month simple moving average, moving along one period at a time. This will give trend values for each period from period 7 to period 46. The end values for periods 1 to 6 and 47 to 52 need to be found by some sort of extrapolation or by using a non-centred moving average. The differences between each observation and the corresponding trend value provide individual estimates of the seasonal effects. The average value of these differences in each of the 13 periods can then be found to estimate the overall seasonal effect, assuming that it is constant over the 4-year period. 2.2 It is hard to guess autocorrelations even when plotting xt against xt−1. Any software package will readily give r1=−0.55. 2.4 The usual limits of ‘significance’ are at . Thus r7 is just ‘significant’. However, unless there is some contextual reason for an effect at lag 7, there is no real evidence of non-randomness, as one expects 1 in 20 values to be ‘significant’ when data really are random. 2.5 (b) Two seasonal differences, , are needed to transform data to stationarity. Chapter 3 3.1 For example, ρ(1)=(0.7−0.7×0.2)/(1+0.72+0.22)=0.56/1.53. 3.3 Var(Xt) is not finite. Then consider Yt=Xt−Xt−1=Zt+(C−1)Zt−1. The latter expression denotes a stationary MA(1) process, with ac.f.

3.4 ρ(k)=0.7|k| for k=0, ±1, ±2,…. Note this does not depend on µ.

< previous page

file:///C:/Documents and Settings/Yang/

page_307

next page >

面/The analysis of time series an introduction/files/page_307.html [5/24/2009 16:54:27]

page_308

page_308

< previous page

next page >

Page 308 3.6 Looking at the roots of the auxiliary equation (see Section 3.4.4, general-order process), the roots must , it can easily be shown that λ1+λ2−1. If

the roots are complex and we find λ2>−1.

When λ1=1/3 and λ2=2/9, the Yule-Walker equations have auxiliary equation has roots

and (

). Thus the general solution is

and 3.8

which . Now use ρ(0)=1

giving ρ(1)=3/7, to evaluate A1=16/21 and A2=5/21.

3.9 All three models are stationary and invertible. For model (a), we find 3.11 First evaluate γ(k). Find

Note that , and that . 3.12 (a) p=d=q=1. (b) Model is non-stationary, but is invertible. (c) 0.7, 0.64, 0.628—decreasing very slowly as process is non-stationary. (d) 0.7, 0.15, 0.075, 0.037—decreasing quickly towards zero as invertible. 3.13 The AR(3) process is non-stationary, as the equation (1−B−cB2+cB3)= 0, has a root on the unit circle, namely, B=1. does not depend on t. . 3.14 When θ is uniformly distributed on (0, 2π), then E(cos θ) =E(sin θ)=0. Hence result. Chapter 4 4.2 The least squares normal equations are the same as the sample Yule-Walker equations in Equation (4.12) except that the constant divisor is omitted and the estimated autocovariance in Equation (4.1) is effectively estimated by summing over (N−p), rather than (N−k) cross-product terms. 4.3 When fitting an AR(2) process, π2 is the coefficient α2. For such a process, the first two Yule-Walker equations are: ρ(2)=α1ρ(1)+α2 and ρ(1)=α1+α2ρ(−1)=α1+α2ρ(+1). Solve for π2=α2 by eliminating α1.

< previous page

file:///C:/Documents and Settings/Yang/

page_308

next page >

面/The analysis of time series an introduction/files/page_308.html [5/24/2009 16:54:28]

page_309

page_309

< previous page

next page >

Page 309 4.4 πj is zero for j larger than 2, the order of the process. π2 equals the final coefficient in the model, namely, α2=2/9; and so we only have to find π1, which equals ρ(1). This is 9/21—see Exercise 3.6. 4.5 The values come down quickly to zero, suggesting a stationary process. Values outside the range are significantly different from zero, in this case just r1 and r2. A MA(2) process has an ac.f. of this shape, meaning non-zero coefficients at lags 1 and 2. Given the small sample size, it is, however, possible that an AR(2) process could be appropriate, as the theoretical autocorrelations beyond lag 2 would be quite small, even if not exactly zero as for the MA(2) model. In practice, it is often difficult to distinguish between competitor models having similar ac.f.s and there may be no ‘right answer’. All models are approximations anyway. 4.6 The ac.f. of the data indicates non-stationarity in the mean, as the values of rk are only coming down to zero very slowly. So we need to take first differences (at least). The values of rk for the first differences are . This suggests the series all ‘small’ and are not significantly different from zero (all are less than is now stationary and is just a purely random process. Thus a possible model for the original series is an ARIMA(0, 1, 0) model, otherwise known as a random walk. Of course, in practice, we would like to see a time plot of the data, check for outliers, etc. as well as getting more relevant background information about the economic variable concerned. Chapter 5 5.4 Suppose we denote the model by

Then

and

5.5 Var[eN(h)] equals Chapter 6

when h=1,

when h=2, and

when h=3.

6.1 (b) 6.2 (a) (b) 6.3 The non-zero mean makes no difference to the acv.f., ac.f. or spectrum.

< previous page

file:///C:/Documents and Settings/Yang/

page_309

next page >

面/The analysis of time series an introduction/files/page_309.html [5/24/2009 16:54:29]

page_310

page_310

< previous page

next page >

Page 310

6.4 This triangular spectrum gives rise to a rather unusual ac.f. Dividing through Equation (6.9) by

(or γ

When k=0, we find, as expected, that

(0)), we get

. When k is even, and π so that

completes an integer number of cycles between 0

and can further show that

algebra and integrating by parts (integrate

after some messy

and differentiate

). When k is odd,

is

still zero, but we find, on integrating by parts, that . 6.5 Clearly E[X(t)]=0 and Var[X(t)]=1. Thus ρ(u)=γ(u)= E[X(t)X(t+u)]=Prob[X(t) and X(t+u) have same sign]—Prob[X(t) and X(t+u) have opposite sign]. (Hint: Prob(observing an even number of changes in time u)= .) . As process is in continuous time, use Equation (6.17). Algebra is

With a unit variance, rather messy. It may help to write

6.6 Here ƒY=σ2/π and for 0<

面/The analysis of time series an introduction/files/page_311.html [5/24/2009 16:54:31]

page_312

page_312

< previous page

next page >

Page 312 Using the Kalman filter, we find Hence 10.3

, and

.

using Equation (10.13).

(a) we take

, we find

;

, for example, then we find

= [1, β]Zt. Alternatively, if

. with

(b) Try

.

Chapter 12

. The roots 12.1 This is a VAR(1) model with a coefficient matrix at lag one equal to of the equation: determinant {I−Φx}=0 are 1/1.2 and 1/0.5. Thus one of the roots lies ‘inside the unit circle’. (When, as here, the roots are real, then one only needs to look and see if the magnitude of the root exceeds unity.) This means the process is non-stationary. This could be guessed from the first equation of the model where the coefficient of X1, t−1 is ‘large’ (unity) and so, in the absence of X2t would give a (non-stationary) random walk for X1t (this is not meant to be a proper mathematical argument). 12.2 The roots of the equation: determinant {I−Φx}=0 are as follows: (a) a single root of 1/0.6; (b) a single root of 1/1.4; (c) two roots of 1/0.7 and 1/0.3; (d) two roots of 1/0.8 and 1/0.4. This means that models (a), (c) and (d) are stationary (giving roots exceeding unity in magnitude so they are ‘outside the unit circle’) but (b) is not. 12.3 Model (c) of Exercise 12.2 is stationary and has a diagonal coefficient matrix. This means that the model consists of two independent AR(1) processes. Any cross-covariances are zero and so the covariance and correlation matrix functions are diagonal. We find and 12.4 All pure MA processes, whether univariate or multivariate, are stationary. The model is invertible if the are outside the unit circle. In this case, we find that roots of the equation: determinant the roots are −1/0.8 and −1/0.2, which are both smaller than −1 and so lie outside the unit circle. Thus the model is invertible. 12.5 Denote the forecast of XN+h made at time N by

N(h). Then . Picking out the first

< previous page

file:///C:/Documents and Settings/Yang/

page_312

next page >

面/The analysis of time series an introduction/files/page_312.html [5/24/2009 16:54:32]

page_313

page_313

< previous page

next page >

Page 313 component of X, for example, we find

while .

< previous page

file:///C:/Documents and Settings/Yang/

page_313

next page >

面/The analysis of time series an introduction/files/page_313.html [5/24/2009 16:54:33]

page_314

< previous page

page_314

next page >

page_314

next page >

Page 314 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_314.html [5/24/2009 16:54:33]

page_315

< previous page

page_315

next page >

Page 315 References The numbers in the square brackets at the end of each reference are the section numbers in which the reference is cited. There may be more than one citation in a section. Abraham, B. and Chuang, A. (1989) Outlier detection and time-series modelling. Technometrics, 31, 241– 248. [13.7.5] Abraham, B. and Ledolter, J. (1983) Statistical Methods for Forecasting. New York: Wiley. [5.2, 10.2] Abraham, B. and Ledolter, J. (1986) Forecast functions implied by autoregressive integrated moving average models and other related forecast procedures. Int. Stat. Rev., 54, 51–66. [10.1.2, 10.2] Abramovich, F., Bailey, T.C. and Sapatinas, T. (2000) Wavelet analysis and its statistical applications. The Statistician, 49, 1–29. [13.7.2] Adya, M., Armstrong, J.S., Collopy, F. and Kennedy, M. (2000) An application of rule-based forecasting to a situation lacking domain knowledge. Int. J. Forecasting, 16, 477–484. [5.4.3] Akaike, H. (1968) On the use of a linear model for the identification of feedback systems. Ann. Inst. Statist. Math., 20, 425–439. [9.4.3] Anderson, T.W. (1971) The Statistical Analysis of Time Series. New York: Wiley. [1.5, 7.2, 7.3, 7.8] Andrews, D.F. and Herzberg, A.M. (1985) Data. New York: Springer-Verlag. [11.1.1, 14.6] Andrews, R.L. (1994) Forecasting performance of structural time series models. J. Bus. Econ. Stat, 12, 129– 133. [10.1.3] Ansley, C.F. and Newbold, P. (1980) Finite sample properties of estimators for autoregressive-moving average models. J. Econometrics, 13, 159–183. [4.4] Aoki, M. (1990) State Space Modeling of Time Series, 2nd edn. Berlin: Springer-Verlag. [10.2] Armstrong, J.S. (ed.) (2001) Principles of Forecasting: A Handbook for Researchers and Practitioners. Norwell, MA: Kluwer. [5.1, 5.4.4] Armstrong, J.S. and Collopy, F. (1992) Error measures for generalizing about forecasting methods: Empirical comparisons. Int. J. Forecasting, 8, 69–80. [5.4.1] Ashley, R. (1988) On the relative worth of recent macroeconomic forecasts. Int. J. Forecasting, 4, 363–376. [5.4.2] Astrom, K.J. (1970) Introduction to Stochastic Control Theory. New York: Academic Press. [5.6, 9.4.2]

< previous page

file:///C:/Documents and Settings/Yang/

page_315

next page >

面/The analysis of time series an introduction/files/page_315.html [5/24/2009 16:54:34]

page_316

< previous page

page_316

next page >

Page 316 Astrom, K.J. and Bohlin, T. (1966) Numerical identification of linear dynamic systems from normal operating records. In Theory of Self-Adaptive Control Systems (ed. P.M.Hammond), pp. 94–111. New York: Plenum Press. [9.4.2] Bai, J. and Perron, P. (1998) Estimating and testing linear models with multiple structural changes. Econometrica, 66, 47–78. [13.2] Ball, M. and Wood, A. (1996) Trend growth in post-1850 British economic history: The Kalman filter and historical judgment. The Statistician, 45, 143–152. [2.5] Banerjee, A., Dolado, J., Galbraith, J.W. and Hendry, D.F. (1993) Co-Integration, Error-Correction, and the Econometric Analysis of NonStationary Data. Oxford: Oxford Univ. Press. [12.6] Bartlett, M.S. (1990) Chance or chaos (with discussion). J. R. Stat. Soc. A, 153, 321–347. [11.5] Bell, W.R. and Hillmer, S.C. (1983) Modelling time series with calendar variation. J. Am. Stat. Assoc., 78, 526– 534. [2.6] Bendat, J.S. and Piersol, A.G. (2000) Random Data: Analysis and Measurement Procedures, 3rd edn. New York: Wiley. [7.4.5, 9.1, 9.3.1, 13.7.3] Beran, J. (1994) Statistics for Long-Memory Processes. New York: Chapman & Hall. [13.3] Berliner, L.M. (1991) Likelihood and Bayesian prediction of chaotic systems. J. Am. Stat. Assoc., 86, 938– 952. [11.5] Bidarkota, P.V. (1998) The comparative forecast performance of univariate and multivariate models: An application to real interest rate forecasting. Int. J. Forecasting, 14, 457–468. [12.5] Bishop, C.M. (1995) Neural Networks for Pattern Recognition. Oxford: Clarendon Press. [11.4] Blake, I.F. and Lindsey, W.C. (1973) Level-crossing problems for random processes. IEEE Trans., IT-19, No. 3, 295–315. [13.7.3] Bloomfield, P. (2000) Fourier Analysis of Time Series, 2nd edn. New York: Wiley. [7.2, 7.4.4, 7.4.5, 7.6, 8.2.2, 9.4.1, 13.2] Boero, G. (1990) Comparing ex-ante forecasts from a SEM and VAR model: An application to the Italian economy. J. Forecasting, 9, 13–24. [12.5] Bollerslev, T., Chou, Y. and Kroner, K.F. (1992) ARCH models in finance. J. Econometrics, 52, 5–59. [11.3] Bollerslev, T., Engle, R.F. and Nelson, D.B. (1994) ARCH models. In Handbook of Econometrics, Vol. IV (eds. R.F.Engle, and D.L.McFadden), pp. 2959–3038. Amsterdam: Elsevier. [11.3] Box, G.E.P. and Jenkins, G.M. (1970) Time-Series Analysis, Forecasting and Control San Francisco: HoldenDay (revised edn., 1976). [1.5, 4.6, 9.4.2, 11.1.2, 14.3] Box, G.E.P., Jenkins, G.M. and Reinsel, G.C. (1994) Time Series Analysis, Forecasting and Control, 3rd edn. Englewood Cliffs, NJ: Prentice-Hall. [1.3, 1.5, 3.4.5, 4.3.1, 4.6, 4.7, 5.2.4, 9.1, 9.3.1, 9.4, 12.2, 13.2, 13.6, 13.7.5] Box, G.E.P. and MacGregor, J.F. (1974) The analysis of closed-loop dynamic-stochastic systems. Technometrics, 16, 391–398. [9.4.3]

< previous page

file:///C:/Documents and Settings/Yang/

page_316

next page >

面/The analysis of time series an introduction/files/page_316.html [5/24/2009 16:54:34]

page_317

< previous page

page_317

next page >

Page 317 Box, G.E.P. and Newbold, P. (1971) Some comments on a paper of Coen, Gomme and Kendall. J. R. Stat. Soc. A, 134, 229–240. [5.3.1, 8.1.3] Brillinger, D.R. (2001) Time Series: Data Analysis and Theory, classics edn. Philadelphia: SIAM. [1.5] Brillinger, D., Caines, P., Geweke, J., Parzen, E., Rosenblatt, M. and Taqqu, M.S. (eds.) (1992, 1993) New Directions in Time Series Analysis: Parts I and II. New York: Springer-Verlag. [13.0] Brock, W.A. and Potter, S.M. (1993) Non-linear time series and macroeconometrics. In Handbook of Statistics, Vol. 11, Econometrics (eds. G.S.Maddala, C.R.Rao and H.D.Vinod), pp. 195–229. Amsterdam: North-Holland. [11.1.4, 11.5] Brockwell, P.J. and Davis, R.A. (1991) Time Series: Theory and Methods, 2nd edn. New York: SpringerVerlag. [1.5, 8.1.2, 13.1, 13.3, 13.7.9] Brockwell, P.J. and Davis, R.A. (2002) Introduction to Time Series and Forecasting, 2nd edn. New York: Springer. [1.5] Broersen, P.M.T. (2002) Automatic spectral analysis with time series models. IEEE Trans. Instrumentation Meas., 51, 211–216. [13.7.1] Brown, R.G. (1963) Smoothing, Forecasting and Prediction. Englewood Cliffs, NJ: Prentice-Hall. [1.3, 5.2.3] Bühlmann, P. (2002) Bootstraps for time series. Stat. Sci., 17, 52–72. [13.7.9] Burnham, K.P. and Anderson, D.R. (2002) Model Selection and Multi-Model Inference, 2nd edn. New York: Springer-Verlag. [4.8, 13.1] Butter, F.A.G.den and Fase, M.M.G. (1991) Seasonal Adjustment as a Practical Problem. Amsterdam: Elsevier. [2.6] Chan, K.-S. and Tong, H. (2001) Chaos: A Statistical Perspective. New York: Springer. [11.5] Chang, I., Tiao, G.C. and Chen, C. (1988) Estimation of time-series parameters in the presence of outliers. Technometrics, 30, 193–204. [13.7.5] Chappell, D., Padmore, J., Mistry, P. and Ellis, C. (1996) A threshold model for the French Franc/Deutschmark exchange rate. J. Forecasting, 15, 155–164. [11.2.2] Chatfield, C. (1974) Some comments on spectral analysis in marketing. J. Marketing Res., 11, 97–101. [7.8] Chatfield, C. (1977) Some recent developments in time-series analysis. J. R. Stat. Soc. A, 140, 492–510. [9.4.2, 13.7.8] Chatfield, C. (1978) The Holt-Winters forecasting procedure. Appl. Stat., 27, 264–279. [5.4.2] Chatfield, C. (1979) Inverse autocorrelations. J. R. Stat. Soc. A, 142, 363–377. [13.1] Chatfield, C. (1988) What is the best method of forecasting? J. Appl. Stat., 15, 19–38. [5.4] Chatfield, C. (1993) Calculating interval forecasts (with discussion). J. Bus. Econ. Stat., 11, 121–144. [5.2.6, 13.5] Chatfield, C. (1995a) Problem-Solving: A Statistician’s Guide, 2nd edn. London: Chapman & Hall. [4.8, 14.4]

< previous page

file:///C:/Documents and Settings/Yang/

page_317

next page >

面/The analysis of time series an introduction/files/page_317.html [5/24/2009 16:54:35]

page_318

< previous page

page_318

next page >

Page 318 Chatfield, C. (1995b) Model uncertainty, data mining and statistical inference (with discussion). J. R. Stat Soc. A, 158, 419–466. [13.5] Chatfield, C. (1995c) Positive or negative? Int. J. Forecasting, 11, 501–502. [5.4.1] Chatfield, C. (1996) Model uncertainty and forecast accuracy. J. Forecasting, 15, 495–508. [13.5] Chatfield, C. (2001) Time-Series Forecasting. Boca Raton: Chapman & Hall/CRC Press. [5.2, 5.3, 5.4, 11.6, 12.7, 13.5] Chatfield, C. (2002) Confessions of a pragmatic statistician. The Statistician, 51, 1–20. [5.5] Chatfield, C. and Collins, A.J. (1980) Introduction to Multivariate Analysis. London: Chapman & Hall. [10.2] Chatfield, C., Koehler, A.B., Ord, J.K. and Snyder, R.D. (2001) Models for exponential smoothing: A review of recent developments. The Statistician, 50, 147–159. [5.2.2, 10.1.3] Chatfield, C. and Prothero, D.L. (1973) Box-Jenkins seasonal forecasting: Problems in a case study (with discussion). J. R. Stat. Soc. A, 136, 295–352. [1.1, 5.5, 14.3] Chatfield, C. and Yar, M. (1988) Holt-Winters forecasting: Some practical issues. The Statistician, 37, 129– 140. [5.2.3, 5.4.2, 5.4.3] Chen, C. (1997) Robustness properties of some forecasting methods for seasonal time series: A Monte Carlo study. Int. J. Forecasting, 13, 269–280. [5.4.1] Chen, C. and Liu, L.-M. (1993) Forecasting time series with outliers. J. Forecasting, 12, 13–35. [13.7.5] Choi, B. (1992) ARMA Model Identification. New York: Springer-Verlag. [13.1] Choudhury, A.H., Hubata, R. and St. Louis, R.D. (1999) Understanding time-series regression estimators. Am. Statistician, 53, 342–348. [5.3.1] Clemen, R.T. (1989) Combining forecasts: A review and annotated bibliography. Int. J. Forecasting, 5, 559– 583. [13.5] Cleveland, W.S. (1993) Visualizing Data. Summit, NJ: Hobart Press. [13.7.5] Cleveland, W.S. (1994) The Elements of Graphing Data, 2nd edn. Summit, NJ: Hobart Press. [11.1.1, 14.4] Coen, P.J., Gomme, E.J. and Kendall, M.G. (1969) Lagged relationships in economic forecasting. J. R. Stat. Soc. A, 132, 133–163. [8.1.3] Collopy, F. and Armstrong, J.S. (1992) Rule-based forecasting: Development and validation of an expert systems approach to combining time series extrapolations. Man. Sci., 38, 1394–1414. [5.4.3] Cox, D.R. and Isham, V. (1980) Point Processes. London: Chapman & Hall. [1.1] Cox, D.R. and Miller, H.D. (1968) The Theory of Stochastic Processes. New York: Wiley. [3.1, 5.6, 6.2, A] Craddock, J.M. (1965) The analysis of meteorological time series for use in forecasting. The Statistician, 15, 169–190^ [7.8] Cramér, H. and Leadbetter, M.R. (1967) Stationary and Related Stochastic Processes. New York: Wiley. [13.7.3]

< previous page

file:///C:/Documents and Settings/Yang/

page_318

next page >

面/The analysis of time series an introduction/files/page_318.html [5/24/2009 16:54:36]

page_319

< previous page

page_319

next page >

Page 319 Crato, N. and Ray, B.K. (1996) Model selection and forecasting for long-range dependent processes. J. Forecasting, 15, 107–125. [13.3] Cressie, N. (1993) Statistics for Spatial Data, Rev. edn. New York: Wiley. [13.7.8] Dangerfield, B.J. and Morris, J.S. (1992) Top-down or bottom-up: Aggregate versus disaggregate extrapolations. Int. J. Forecasting, 8, 233–241. [13.7.7] Darbellay, G.A. and Slama, M. (2000) Forecasting the short-term demand for electricity: Do neural networks stand a better chance? Int. J. Forecasting, 16, 71–83. [11.1.1] Davies, N. and Newbold, P. (1979) Some power studies of a portmanteau test of time series model specification. Biometrika, 66, 153–155. [4.7] Davis, M.H.A. and Vinter, R.B. (1985) Stochastic Modelling and Control. London: Chapman & Hall. [13.6] DeJong, D.N. and Whiteman, C.H. (1993) Unit roots in U.S. macroeconomic time series: A survey of classical and Bayesian perspectives. In New Directions in Time Series Analysis: Part II (eds. D.Brillinger et al.), pp. 43– 59. New York: Springer-Verlag. [13.4] Dhrymes, P. (1997) Time Series, Unit Roots and Cointegration. San Diego: Academic Press. [12.6] Dhrymes, P.J. and Tomakos, D.D. (1998) Structural VAR, MARMA and open economy models. Int. J. Forecasting, 14, 187–198. [12.5] Diebold, F.X. (2001) Elements of Forecasting, 2nd edn. Cincinnati: South-Western. [5.2, 13.4, 13.5] Diebold, F.X. and Kilian, L. (2000) Unit root tests are useful for selecting forecasting models. J. Bus. Econ. Stat., 18, 265–273. [13.4] Diggle, P.J. (1990) Time Series: A Biostatistical Introduction. Oxford: Oxford Univ. Press. [1.5, 13.7.4, 13.7.8] Diggle, P.J., Heagerty, P., Liang, K.-Y. and Zeger, S.L. (2002) Analysis of Longitudinal Data, 2nd edn. Oxford: Oxford Univ. Press. [13.7.6] Draper, D. (1995) Assessment and propagation of model uncertainty (with discussion). J. R. Stat. Soc. B, 57, 45–97. [13.5] Durbin, J. and Koopman, S.J. (2001) Time Series Analysis by State Space Methods. Oxford: Oxford Univ. Press. [10.0, 10.2] Durbin, J. and Murphy, M.J. (1975) Seasonal adjustment based on a mixed additive-multiplicative model. J. R. Stat. Soc. A, 138, 385–410. [2.6] Enders, W. (1995) Applied Econometric Time Series. New York: Wiley. [1.5, 11.3, 13.4] Engle, R.F. and Granger, C.W.J. (1991) Long-Run Economic Relationships: Readings in Cointegration. Oxford: Oxford Univ. Press. [12.6] Faraway, J. and Chatfield, C. (1998) Time series forecasting with neural networks: A comparative study using the airline data. Appl. Stat., 47, 231–250. [11.4, 13.1] Fildes, R. (1983) An evaluation of Bayesian forecasting. J. Forecasting, 2, 137–150. [10.1.5] Fildes, R. (1985) Quantitative forecasting: The state of the art. Econometric models. J. Op. Res. Soc., 36, 549–580. [5.4.2]

< previous page

file:///C:/Documents and Settings/Yang/

page_319

next page >

面/The analysis of time series an introduction/files/page_319.html [5/24/2009 16:54:36]

page_320

< previous page

page_320

next page >

Page 320 Fildes, R. (1992) The evaluation of extrapolative forecasting methods. Int. J. Forecasting, 8, 81–98. [5.4.1] Fildes, R. and Makridakis, S. (1995) The impact of empirical accuracy studies on time series analysis and forecasting. Int. Stat. Rev., 63, 289–308. [5.4.1] Findley, D.F., Monsell, B.C., Bell, W.R., Otto, M.C. and Chen, B.-C. (1998) New capabilities and methods of the X-12-ARIMA seasonal adjustment program (with discussion and reply). J. Bus. Econ. Stat., 16, 127–177. [2.6, 5.2.4] Franses, P.H. (1998) Time Series Models for Business and Economic Forecasting. Cambridge: Cambridge Univ. Press. [2.5.1, 11.3, 11.7, 13.7.5] Franses, P.H. and Kleibergen, F. (1996) Unit roots in the Nelson-Plosser data: Do they matter for forecasting? Int J. Forecasting, 12, 283–288. [2.5.3] Fuller, W.A. (1996) Introduction to Statistical Time Series, 2nd edn. New York: Wiley. [1.5] Gardner, E.S. Jr. (1983) Automatic monitoring of forecast errors. J. Forecasting, 2, 1–21. [5.1] Gardner, E.S. Jr. (1985) Exponential smoothing: The state of the art. J. Forecasting, 4, 1–28. [5.2.2, 5.2.3] Gardner, E.S. Jr. and McKenzie, E. (1985) Forecasting trends in time series. Man. Sci., 31, 1237–46. [5.2.3, 5.4.3] Gleick, J. (1987) Chaos. New York: Viking. [11.5] Glover, I. and Grant, P. (1998) Digital Communications. London: Prentice-Hall. [9.1] Gómez, V. and Maravall, A. (2001) Seasonal adjustment and signal extraction in economic time series. In A Course in Time Series Analysis (eds. D.Peña, G.C.Tiao and R.S.Tsay), Chapter 8. New York: Wiley. [2.6] Gooijer, J.G.de, Abraham, B., Gould, A. and Robinson, L. (1985). Methods for determining the order of an autoregressive-moving average process: A survey. Int. Stat. Rev., 53, 301–329. [13.1] Gourieroux, C. (1997) ARCH Models and Financial Applications. New York: Springer-Verlag. [11.3] Granger, C.W.J. (1992) Forecasting stock market prices: Lessons for forecasters. Int. J. Forecasting, 8, 3–13. [11.5] Granger, C.W.J. and Newbold, P. (1974) Spurious regressions in econometrics. J. Econometrics, 2, 111–120. [5.3.1] Granger, C.W.J. and Newbold, P. (1986) Forecasting Economic Time Series, 2nd edn. New York: Academic Press. [2.4, 4.4, 4.7, 5.2, 5.3, 9.4.3, 11.7, 12.7] Granger, C.W.J. and Teräsvirta, T. (1993) Modelling Nonlinear Economic Relationships. New York: Oxford Univ. Press. [11.1.1, 11.5, 11.7, 12.7, 13.3] Green, P.J. and Silverman, B.W. (1994) Nonparametric Regression and Generalized Linear Models. London: Chapman & Hall. [13.7.4] Grimmett, G.R. and Stirzaker, D.R. (2001) Probability and Random Processes, 3rd edn. Oxford: Oxford Univ. Press. [3.1] Grubb, H. and Mason, A. (2001) Long lead-time forecasting of UK air passengers

< previous page

file:///C:/Documents and Settings/Yang/

page_320

next page >

面/The analysis of time series an introduction/files/page_320.html [5/24/2009 16:54:37]

page_321

< previous page

page_321

next page >

Page 321 by Holt-Winters methods with damped trend. Int. J. Forecasting, 17, 71–82. [5.2.3, 14.3] Gustavsson, I., Ljung, L. and Söderstrom, T. (1977) Identification of process in closed loop—identifiability and accuracy aspects. Automatica, 13, 59–75. [9.4.3] Hamilton, J.D. (1994) Time Series Analysis. Princeton, NJ: Princeton Univ. Press. [1.5, 4.1.3, 5.3.1, 13.4] Hand, D.J., Daly, F., Lunn, A.D., McConway, K.J. and Ostrowski, F. (eds.) (1994) A Handbook of Small Data Sets. London: Chapman & Hall. [11.1.1, 14.6] Hannan, E.J. (1970) Multiple Time Series. New York: Wiley. [7.3, 13.7.4] Harrison, P.J. (1965) Short-term sales forecasting. Appl. Stat., 14, 102–139. [5.2.5] Harvey, A.C. (1989) Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge: Cambridge Univ. Press. [5.2.5, 10.0, 10.1, 10.2, 12.7] Harvey, A.C. (1990) The Econometric Analysis of Time Series, 2nd edn. Hemel Hempstead, U.K.: Philip Allan. [5.3.2] Harvey, A.C. (1993) Time Series Models, 2nd edn. New York: Harvester Wheatsheaf. [1.5, 10.2, 11.2.4, 11.3, 11.7, 13.4] Haslett, J. (1997) On the sample variogram and the sample autocovariance for non-stationary time series. The Statistician, 46, 475–485. [13.7.8] Hayes, M.H. (1996) Statistical Digital Signal Processing and Modeling. New York: Wiley. [7.4.4, 9.3.1, 13.7.1, A] Hylleberg, S. (ed.) (1992) Modelling Seasonality. Oxford: Oxford Univ. Press. [2.6] Isham, V. (1993) Statistical aspects of chaos. In Networks and Chaos—Statistical and Probabilistic Aspects (eds. O.E.Barndorff-Nielsen et al.), pp. 124–200. London: Chapman & Hall. [11.5] Jacobs O.L.R. (1993) Introduction to Control Theory. Oxford: Oxford Univ. Press. [13.6] Janacek, G. and Swift, L. (1993) Time Series: Forecasting, Simulation, Applications. Chichester, U.K.: Ellis Horwood. [10.0] Jenkins, G.M. (1979) Practical Experiences with Modelling and Forecasting Time Series. Jersey: Gwilym Jenkins and Partners (Overseas) Ltd. [5.2.4, 5.4.2, 9.4.2] Jenkins, G.M. and McLeod, G. (1982) Case Studies in Time Series Analysis, Vol. 1. Lancaster: Gwilym Jenkins and Partners Ltd. [5.4.2, 9.4.2] Jenkins, G.M. and Watts, D.G. (1968) Spectral Analysis and its Applications. San Francisco: Holden-Day. [3.3, 4.2, 6.2, 6.3, 7.3, 7.5, 7.6, 7.8, 8.2.2, 9.3.1, 9.4.1] Johansen, S. (2001) Cointegration in the VAR model. In A Course in Time Series Analysis (eds. D.Peña, G.C. Tiao and R.S.Tsay), Chapter 15. New York: Wiley. [12.6] Jones, R.H. (1985) Time series analysis with unequally spaced data. In Handbook

< previous page

file:///C:/Documents and Settings/Yang/

page_321

next page >

面/The analysis of time series an introduction/files/page_321.html [5/24/2009 16:54:38]

page_322

< previous page

page_322

next page >

Page 322

of Statistics, Vol. 5 (eds. E.J.Hannan et al.), pp. 157–177. Amsterdam: North-Holland. [13.7.4] Jones, R.H. (1993) Longitudinal Data with Serial Correlation: A State-Space Approach. London: Chapman & Hall. [13.7.6] Kadiyala, K.R. and Karlsson, S. (1993) Forecasting with generalized Bayesian vector autoregressions. J. Forecasting, 12, 365–378. [12.5] Kantz, H. and Schreiber, T. (1997) Nonlinear Time Series Analysis. Cambridge: Cambridge Univ. Press. [11.5] Kedem, B. (1994) Time Series Analysis by Higher Order Crossings. New York: Inst. of Electrical and Electronic Engineers Press. [13.7.3] Kendall, M.G. and Ord, J.K. (1990) Time Series, 3rd edn. Sevenoaks, U.K.: Arnold. [1.5, 2.8] Kendall, M.G., Stuart, A. and Ord, J.K. (1983) The Advanced Theory of Statistics, Vol. 3, 4th edn. London: Griffin. [1.5, 2.5.2, 2,8, 4.1, 4.4, 4.7, 6.3] Kenny, P.B. and Durbin, J. (1982) Local trend estimation and seasonal adjustment of economic and social time series (with discussion). J. R. Stat. Soc. A, 145, 1–41. [2.5.2] Kohn, R. and Ansley, C.F. (1986) Estimation, prediction and interpolation for ARIMA models with missing data. J. Am. Stat. Assoc., 81, 751–761. [10.1.4] Koopmans, L.H. (1995) The Spectral Analysis of Time Series, 2nd edn. San Diego: Academic Press. [7.3] Lawrance, A.J. (1991) Directionality and reversibility in time series. Int. Stat. Rev., 59, 67–79. [11.5] Ledolter, J. (1989) The effect of additive outliers on the forecasts from ARIMA models. Int. J. Forecasting, 5, 231–240. [13.7.5] Lin, J.-L. and Granger, C.W.J. (1994) Forecasting from non-linear models in practice. J. Forecasting, 13, 1–9. [11.6] Luceño, B. (1998) Detecting possibly non-consecutive outliers in industrial time series. J. R.Stat. Soc. B, 60, 295–310. [13.7.5] Lütkepohl, H. (1993) Introduction to Multiple Time Series Analysis, 2nd edn. New York: Springer-Verlag. [12.5, 12.7] Macdonald, I.L. and Zucchini, W. (1997) Hidden Markov and Other Models for Discrete-Valued Time Series. London: Chapman & Hall. [13.7.10] Makridakis, S., Anderson, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E. and Winkler, R. (1984) The Forecasting Accuracy of Major Time Series Methods. New York: Wiley. [5.4.1] Makridakis, S., Chatfield, C., Hibon, M., Lawrence, M., Mills, T., Ord, K. and Simmons, L.F. (1993) The M2competition: A real-time judgmentally based forecasting study (with commentary). Int. J. Forecasting, 9, 5– 29. [5.4.1] Makridakis, S. and Hibon, M. (1979) Accuracy of forecasting: An empirical investigation (with discussion). J. R. Stat. Soc. A, 142, 97–145. [5.4.1, 5.4.2] Makridakis, S. and Hibon, M. (2000) The M3 competition: results, conclusions and implications. Int. J. Forecasting, 16, 451–476. [5.2.5, 5.4.1] May, R.M. (1987) Chaos and the dynamics of biological populations. Proc. R. Soc. London A, 413, 27–44. [11.5]

< previous page

file:///C:/Documents and Settings/Yang/

page_322

next page >

面/The analysis of time series an introduction/files/page_322.html [5/24/2009 16:54:38]

page_323

< previous page

page_323

next page >

Page 323 McCullough, B.D. (1998) Algorithms for (partial) autocorrelation coefficients. J. Economic Soc. Meas., 24, 265–278. [4.2.2, 14.2] McCullough, B.D. (2000) Is it safe to assume that software is accurate? Int. J. Forecasting, 16, 349–357. [14.2] McLain, J.O. (1988) Dominant tracking signals. Int. J. Forecasting, 4, 563–572. [5.1] Meade, N. (1984) The use of growth curves in forecasting market development—a review and appraisal. J. Forecasting, 3, 429–451. [2.5.1, 5.2.1] Meade, N. and Smith, I.D. (1985) ARARMA vs. ARIMA—A study of the benefits of a new approach to forecasting. Int. J. Manag. Sci., 13, 519–534. [5.2.5] Meinhold, R.J. and Singpurwalla, N.D. (1983) Understanding the Kalman filter. Am. Statistician, 32, 123–127. [10.2] Meinhold, R.J. and Singpurwalla, N.D. (1989) Robustification of Kalman filter models. J. Am. Stat. Assoc., 84, 479–486. [13.7.5] Mills, T.C. (1990) Time Series Techniques for Economists. Cambridge: Cambridge Univ. Press. [1.5] Mills, T.C. (1999) The Econometric Modelling of Financial Time Series, 2nd edn. Cambridge: Cambridge Univ. Press. [1.5] Mizon, G.E. (1995) A simple message for autocorrelation correctors: Don’t. J. Econometrics, 69, 267–288. [5.3.1] Montgomery, D.C. (1996) Introduction to Statistical Quality Control, 3rd edn. New York: Wiley. [1.1] Montgomery, D.C., Johnson, L.A. and Gardiner, J.S. (1990) Forecasting and Time Series Analysis, 2nd edn. New York: McGraw-Hill. [5.2] Murray, M.P. (1994) A drunk and her dog: An illustration of cointegration and error correction. Am. Statistician, 48, 37–39. [12.6] Neave, H.R. (1972a) Observations on ‘Spectral analysis of short series—a simulation study’ by Granger and Hughes. J. R. Stat. Soc. A, 135, 393–405. [7.5] Neave, H.R. (1972b) A comparison of lag window generators. J. Am. Stat. Assoc., 67, 152–8. [7.6] Nelson, H.L. and Granger, C.W.J. (1979) Experience with using the Box-Cox transformation when forecasting economic time series. J. Econometrics, 10, 57–69. [2.4] Newbold, P. (1981, 1984, 1988) Some recent developments in time-series analysis, I, II and III. Int. Stat. Rev., 49, 53–66; 52, 183–192; 56, 17–29. [4.7, 13.0, 13.1] Newbold, P. and Granger, C.W.J. (1974) Experience with forecasting univariate time-series and the combination of forecasts (with discussion). J. R. Stat. Soc. A, 137, 131–165. [5.4.1] Newbold, P., Agiakloglou, C. and Miller, J. (1993) Long-term inference based on short-term forecasting models. In Time Series Analysis (ed. T.Subba Rao), pp. 9–25. London: Chapman & Hall. [13.4]

< previous page

file:///C:/Documents and Settings/Yang/

page_323

next page >

面/The analysis of time series an introduction/files/page_323.html [5/24/2009 16:54:39]

page_324

< previous page

page_324

next page >

Page 324 Newbold, P., Agiakloglou, C. and Miller, J. (1994) Adventures with ARIMA software. Int. J. Forecasting, 10, 573–581. [14.2] Nicholls, D.F. and Pagan, A.R. (1985) Varying coefficient regression. In Handbook of Statistics, Vol. 5 (eds. E. J.Hannan, P.R.Krishnaiah and M.M. Rao), pp. 413–449. Amsterdam: North-Holland. [11.2.1] Ord, J.K., Koehler, A.B. and Snyder, R.D. (1997) Estimation and prediction for a class of dynamic nonlinear statistical models. J. Am. Stat. Assoc., 92, 1621–1629. [10.1.3] Otomo, T., Nakagawa, T. and Akaike, H. (1972) Statistical approach to computer control of cement rotary kilns. Automatica, 8, 35–48. [9.4.3] Pankratz, A. (1991) Forecasting with Dynamic Regression Models. New York: Wiley. [12.2] Papoulis, A. (1984) Probability, Random Variables and Stochastic Processes, 2nd edn. New York: McGraw-Hill. [3.1] Parzen, E. (1982) ARARMA models for time series analysis and forecasting. J. Forecasting, 1, 67–82. [5.2.5] Parzen, E. (ed.) (1984) Time Series Analysis of Irregularly Observed Data, Proc. of a Symposium at Texas A & M University. New York: Springer. [13.7.4] Peña, D. (2001) Outliers, influential observations and missing data. In A Course in Time Series Analysis (eds. D.Peña, G.C.Tiao and R.S.Tsay), Chapter 6. New York: Wiley. [13.7.4, 13.7.5] Peña, D. and Box, G.E.P. (1987) Identifying a simplifying structure in time series. J. Am. Stat. Assoc., 82, 836–843. [12.1.3] Peña, D., Tiao, G.C. and Tsay, R.S. (eds.) (2001) A Course in Time Series Analysis. New York: Wiley. [13.0] Percival, D.B. (1993) Three curious properties of the sample variance and autocovariance for stationary processes with unknown mean. Am. Statistician, 47, 274–276. [4.1] Percival, D.B. and Walden, A.T. (1993) Spectral Analysis for Physical Applications: Multitaper and Conventional Univariate Techniques. Cambridge: Cambridge Univ. Press. [7.4, 7.8, 13.7.1, 14.6] Percival, D.B. and Walden, A.T. (2000) Wavelet Methods for Time Series Analysis. Cambridge: Cambridge Univ. Press. [13.7.2] Phillips, P.C.B. (1986) Understanding spurious regressions in econometrics. J. Econometrics, 33, 311–340. [5.3.1] Pole, A., West, M. and Harrison, J. (1994) Applied Bayesian Forecasting and Time Series Analysis. New York: Chapman & Hall. [10.1.5] Priestley, M.B. (1981) Spectral Analysis and Time Series, Vols. 1 and 2. London: Academic Press. [1.5, 4.1.2, 4.4, 5.3.3, 7.1, 7.4.5, 7.6, 8.2, 11.7, 12.5, 13.1, 13.2, 13.6, 13.7.1, 13.7.4] Priestley, M.B. (1983) The frequency domain approach to the analysis of closed-loop systems. In Handbook of Statistics, Vol. 3 (eds. D.R.Brillinger and P.R. Krishnaiah), pp. 275–291. Amsterdam: North-Holland. [9.4.3] Priestley, M.B. (1988) Non-linear and Non-stationary Time Series Analysis. London: Academic Press. [11.1.1, 11.2.4, 11.7, 13.2]

< previous page

file:///C:/Documents and Settings/Yang/

page_324

next page >

面/The analysis of time series an introduction/files/page_324.html [5/24/2009 16:54:39]

page_325

< previous page

page_325

next page >

Page 325 Racine, J. (2001) On the non-linear predictability of stock returns using financial and economic variables. J. Bus. Econ. Stat, 19, 380–382. [11.4] Reinsel, G.C. (1997) Elements of Multivariate Time Series Analysis, 2nd edn. New York: Springer-Verlag. [12.5, 12.7] Ripley, B.D. (1981) Spatial Statistics. Chichester: Wiley. [13.7.8] Robinson, P.M. and Zaffaroni, P. (1998) Nonlinear time series with long memory: A model for stochastic volatility. J. Stat. Planning Inference, 68, 359–371. [13.3] Rogers, L.C.G. and Williams, D. (1994) Diffusions, Markov Processes and Martingales, 2nd edn. Cambridge: Cambridge Univ. Press. [3.4.8] Ross, S.M. (1997) Introduction to Probability Models, 6th edn. San Diego: Academic Press. [3.1] Rowe, G. and Wright, G. (1999) The Delphi technique as a forecasting tool: Issues and analysis. Int. J. Forecasting, 15, 353–375. [5.1] Rycroft, R.S. (1999) Microcomputer software of interest to forecasters in comparative review: Updated again. Int. J. Forecasting, 15, 93–120. [14.2] Rydberg, T.H. (2000) Realistic statistical modelling of financial data. Int. Stat. Rev., 68, 233–258. [13.7.9] Schoemaker, P.J.H. (1991) When and how to use scenario planning: A heuristic approach with illustrations. J. Forecasting, 10, 549–564. [13.5] Shephard, N. (1996) Statistical aspects of ARCH and stochastic volatility. In Time Series Models (eds. D.R. Cox, D.V.Hinkley and O.E.Barndorff-Nielsen), pp. 1–67. London: Chapman & Hall. [11.3] Smith, J. and Yadav, S. (1994) Forecasting costs incurred from unit differencing fractionally integrated processes. Int. J. Forecasting, 10, 507–514. [13.3] Snodgrass, F.E., Groves, G.W., Hasselmann, K.F., Miller, G.R., Munk, W.H. and Powers, W.H. (1966) Propagation of ocean swells across the Pacific. Phil. Trans. R. Soc. London A, 259, 431–497. [7.8] Spencer, D.E. (1993) Developing a Bayesian vector autoregression forecasting model. Int. J. Forecasting, 9, 407–421. [12.5] Stern, H. (1996) Neural networks in applied statistics (with discussion). Technometrics, 38, 205–220. [11.4] Strang, G. (1993) Wavelet transforms versus Fourier transforms. Bull. Am. Math. Soc., 28, 14–38. [13.7.2] Sutcliffe, A. (1994) Time-series forecasting using fractional differencing. J. Forecasting, 13, 383–393. [13.3] Swanson, N.R. and White, H. (1997) Forecasting economic time series using flexible versus fixed specification and linear versus nonlinear econometric models. Int. J. Forecasting, 13, 439–461. [13.2] Tashman, L.J. and Kruk, J.M. (1996) The use of protocols to select exponential smoothing procedures: A reconsideration of forecasting competitions. Int. J. Forecasting, 12, 235–253. [5.4.1] Tay, A.S. and Wallis, K.F. (2000) Density forecasting: A survey. J. Forecasting, 19, 235–254. [5.1]

< previous page

file:///C:/Documents and Settings/Yang/

page_325

next page >

面/The analysis of time series an introduction/files/page_325.html [5/24/2009 16:54:40]

page_326

< previous page

page_326

next page >

Page 326 Taylor, S.J. (1994) Modeling stochastic volatility: A review and comparative study. Math. Finance, 4, 183– 204. [11.3] Tee, L.H. and Wu, S.M. (1972) An application of stochastic and dynamic models in the control of a papermaking process. Technometrics, 14, 481–496. [9.4.3] Teräsvirta, T. (1994) Specification, estimation and evaluation of smooth transition autoregressive models. J. Am. Stat. Assoc., 89, 208–218. [11.2.2] Tiao, G.C. (2001). Vector ARMA models. In A Course in Time Series Analysis (eds. D.Peña, G.C.Tiao, and R.S. Tsay), Chapter 14. New York: Wiley. [12.5] Tiao, G.C. and Tsay, R.S. (1994). Some advances in non-linear and adaptive modelling in time series. J. Forecasting, 13, 109–131. [11.2.2] Tong, H. (1990) Non-linear Time Series. Oxford: Oxford Univ. Press. [11.1.1, 11.2.2, 11.2.4, 11.5, 11.6, 11.7, 13.3] Tong, H. (1995) A personal overview of non-linear time series analysis from a chaos perspective. Scand. J. Stat., 22, 399–446. [11.5] Tsay, R.S. (1986) Time series model specification in the presence of outliers. J. Am. Stat. Assoc., 81, 132– 141. [13.7.5] Tsay, R.S. (1998) Testing and modelling multivariate threshold models. J. Am. Stat. Assoc., 93, 1188–1202. [11.2.2] Tsay, R.S. (2001) Nonlinear time series models: Testing and applications. In A Course in Time Series Analysis (eds. D.Peña, G.C.Tiao and R.S.Tsay), Chapter 10. New York: Wiley. [11.1.1] Tsay, R.S. (2002) Analysis of Financial Time Series. New York: Wiley. [11.1.4, 13.7.9] Tsay, R.S., Peña, D., and Pankratz, A.E. (2000) Outliers in multivariate time series. Biometrika, 87, 789–804. [13.7.5] Tyssedal, J.S. and Tjostheim, D. (1988) An autoregressive model with suddenly changing parameters and an application to stock prices. Appl. Stat., 37, 353–369. [13.2] Vandaele, W. (1983) Applied Time Series and Box-Jenkins Models. New York: Academic Press. [1.5, 5.2.4] Velleman, P.F. and Hoaglin, D.C. (1981) ABC of EDA. Boston, MA: Duxbury. [13.7.5] Wallis, K.F. (1999) Asymmetric density forecasts of inflation and the Bank of England’s fan chart. Nat. Inst. Econ. Rev., no. 167, 106–112. [5.1] Warner, B. and Misra, M. (1996) Understanding neural networks as statistical tools. Am. Statistician, 50, 284– 293. [11.4] Webby, R. and O’Connor, M. (1996) Judgemental and statistical time series forecasting: A review of the literature. Int. J. Forecasting, 12, 91–118. [5.1] Wei, W.W.S. (1990) Time Series Analysis: Univariate and Multivariate Methods. Redwood City, CA: AddisonWesley. [1.5, 12.7, 13.7.7] Weigend, A.S. and Gershenfeld, N.A. (eds.) (1994) Time Series Prediction. Proc. Vol. XV, Santa Fe Institute Studies in the Sciences of Complexity. Reading, MA: Addison-Wesley. [11.4] West, M. and Harrison, J. (1997) Bayesian Forecasting and Dynamic Models, 2nd edn. New York: SpringerVerlag. [5.2.5, 10.1.5, 12.7]

< previous page

file:///C:/Documents and Settings/Yang/

page_326

next page >

面/The analysis of time series an introduction/files/page_326.html [5/24/2009 16:54:41]

page_327

< previous page

page_327

next page >

Page 327 Whittle, P. (1983) Prediction and Regulation, 2nd edn., revised. Minneapolis: Univ. of Minnesota Press. [5.6] Wiener, N. (1949) Extrapolation, Interpolation, and Smoothing of Stationary Time-Series. Cambridge, MA: MIT Press. [5.6] Williams, D. (2001) Weighing the Odds. Cambridge: Cambridge Univ. Press. [Preface] Yaglom, A.M. (1962) An Introduction to the Theory of Stationary Random Functions. Englewood Cliffs, NJ: Prentice-Hall. [Exercise 3.14, 5.6] Yao, Q. and Tong, H. (1994) Quantifying the influence of initial values on non-linear prediction. J. R. Stat. Soc. B, 56, 701–725. [11.5] Young, P.C. (1984) Recursive Estimation and Time-Series Analysis. Berlin: Springer-Verlag. [9.4.2] Zhang, G., Patuwo, B.E. and Hu, M.Y. (1998) Forecasting with artificial neural networks: The state of the art. Int. J. Forecasting, 14, 35–62. [11.4]

< previous page

file:///C:/Documents and Settings/Yang/

page_327

next page >

面/The analysis of time series an introduction/files/page_327.html [5/24/2009 16:54:41]

page_328

< previous page

page_328

next page >

page_328

next page >

Page 328 This page intentionally left blank.

< previous page

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_328.html [5/24/2009 16:54:42]

page_329

page_329

< previous page

next page >

Page 329 Index additive seasonal effect, 14, 20 additive outlier, 271 aggregation, 5, 272 AIC criterion, 256 AICC criterion, 256 airline data, 284 Akaike’s information criterion, see AIC criterion bias-corrected, see AICC criterion aliasing, 145 alignment, 165 alternating series, 25 amplitude, 107, 127 angular frequency, 107 AR model, see autoregressive process ARARMA method, 85 ARCH model, 228 ARFIMA model, 260 ARIMA model, 48, 65, 81–83, 86 ARMA model, 46–48, 53, 64–65 Astrom-Bohlin approach, 197 attractor, 235 autocorrelation coefficient, 22 autocorrelation function, 8, 35–37 estimation of, 55–59 autocovariance coefficient, 35 estimation of, 55–56 circular, 139 autocovariance function, 34, 55–56 automatic forecasting method, 74, 92 autoregressive conditionally heteroscedastic, see ARCH model autoregressive (AR) process, 41–46 estimation for, 59–62 spectrum of, 115, 186 state-space representation, 208 autoregressive moving-average process, see ARMA model autoregressive spectrum estimation, 268 back-forecasting, 63 back-propagation, 233 backward shift operator, 40 band-limited, 145 bandwidth, 142 Bartlett window, 132 basic structural model, 207 Bayesian forecasting, 209, Bayesian model averaging, 266 Bayesian vector autoregression, 251 BDS test, 219 Beveridge wheat price series, 1–2 BIC criterion, 256 bilinear model, 225 binary process, 4 bispectrum, 222 bivariate process, 155–167 black box, 233

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_329.html (1 of 2) [5/24/2009 16:54:42]

page_329

Black-Scholes model, 274 bootstrapping, 275 Box-Cox transformation, 14–15, 100 Box-Jenkins approach, 8–9, 94, 194–197 Box-Jenkins forecasting, 81–84 Box-Jenkins models, 81 Box-Jenkins seasonal model, 66 business cycles, 12 butterfly effect, 236 calendar effects, 21 CAT criterion, 256 causal relationship, 242 causal system, 171 change points, 259 chaos, 235–238 Chatfield-Prothero data, 2 closed-loop system, 197, 242 coherency, 161 co-integration, 252 combination of forecasts, 266 complex demodulation, 258

< previous page

file:///C:/Documents and Settings/Yang/

page_329

next page >

面/The analysis of time series an introduction/files/page_329.html (2 of 2) [5/24/2009 16:54:43]

page_330

page_330

< previous page

next page >

Page 330 computer software, 13, 278 context, 11, 29, 101, 278 continuous process, 49 continuous time series, 5 continuous trace, 2 control theory, 7, 266–267 convolution, 19 correlation coefficient, 22, 301 correlogram, 24–28, 57 co-spectrum, 160 covariance, 301 cross-amplitude spectrum, 161 cross-correlation function, 155–159, 244 cross-covariance function, 155–159, 244 crossing points, 269 cross-periodogram, 165 cross-spectrum, 159–166 curve-fitting, 16 cyclic variation, 12 damped trend, 80 data dredging, 265 data editing, 29 data mining, 265 Delphi technique, 73 delay, 172, 174, 182 delayed exponential response, 174 delta function, 299 demography, 3 density forecasting, 75 descriptive methods, 6, 8, 11–30 deterministic process, 51 deterministic series, 5 Dickey-Fuller test, 263 difference-stationary series, 263 differencing, 19 digitizing, 144 Dirac delta function, 299 discrete time series, 5, 275 distributed lag model, 246 double exponential smoothing, 80 Durbin-Watson statistic, 69 dynamic linear model, 205, 209 econometric model, 89 end effects, 17 endogenous variable, 89 ensemble, 34 equilibrium distribution, 35 ergodic theorems, 59 error-correction form, 77 ES, see exponential smoothing evolutionary spectrum, 258 ex-ante forecast, 88 exogenous variable, 89 exponential growth, 97 exponential response, 181

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_330.html (1 of 2) [5/24/2009 16:54:43]

page_330

exponential smoothing, 76–80 ex-post forecast, 88 extreme values, 275 fast Fourier transform (FFT), 136–139 feedback, 94, 191, 242 FFT, see fast Fourier transform filter, 17, 188–190 filtering, 17–20, 104 final prediction error (FPE) criterion, 256 finance, 273 finite impulse response (FIR) system, 172 forecasting, 7, 73–104 competition, 91–92 horizon, 73 forecast monitoring, 75 Fourier analysis, 121, 126 Fourier transform, 112, 129, 295 FPE criterion, 256 fractal, 237 fractional ARIMA (ARFIMA) model, 260 frequency, 107 frequency domain, 8, 107 frequency response function, 175 estimation of, 191–194 fundamental Fourier frequency, 125 gain, 172, 174 diagram, 179 spectrum, 161 GARCH model, 229 general exponential smoothing, 80 general linear process, 49, 220 generalized least squares (GLS), 88 Gompertz curve, 16, 76 global trend, 15, 207

< previous page

file:///C:/Documents and Settings/Yang/

page_330

next page >

面/The analysis of time series an introduction/files/page_330.html (2 of 2) [5/24/2009 16:54:43]

page_331

page_331

< previous page

next page >

Page 331 GLS, 88 growth curve, 76 Hamming, 134 Hanning, 133 Harmonic, 125, 152 analysis, 126 Henderson moving average, 17 high frequency spectrum, 115 high-pass filter, 18, 179, 189 Holt’s exponential smoothing, 78 Holt-Winters procedure, 78–80, 100 identifiable, 250 impulse response function, 171 estimation of, 196 infinite impulse response (IIR) system, 172 initial data analysis, 29, 245, 278 innovation, 40 innovation outlier, 271 integrated ARMA model, see ARIMA model interval forecast, 85 intervention analysis, 259 inverse ac.f., 255 invertibility, 39 Kalman filter, 8, 211–214 kernel, 140 kriging, 273 lag, 35 lag window, 131 Laplace transform, 296 leading indicator, 87, 93, 246 lead time, 73 limit cycle, 219 linear filter, 17 linear model, 18, 49, 219 linear growth model, 206, 213 linear system, 7–8, 169–201 identification of, 190–199 local level model, 206 local trend, 15, 207 logistic curve, 16, 76 logistic map, 235 longitudinal data, 272 long memory model, 260 low frequency spectrum, 115 low-pass filter, 18, 179, 189 Lyapunov exponent, 236 MA process, see moving average process Markov process, 41 martingale, 221 M-competition, 91 M2-competition, 91 M3-competition, 91

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_331.html (1 of 2) [5/24/2009 16:54:44]

page_331

MCMC methods, 275 mean, 34 estimation of, 58 measurement equation, 204 Minnesota prior, 251 MINITAB, 303 missing observations, 29, 269 mixed ARMA model, 46–48 model building, 70–71, 210 checking, 67–70 selection, 255–257 model uncertainty, 264–266 moving average, 17, 181 moving average (MA) process, 38–41, 62 estimation for, 62–64 spectrum of, 114, 185 multiple regression, 87 multiplicative seasonal effect, 14, 20 multivariate forecast, 74, 93 multivariate normal distribution, 36 multivariate time series, 241–253 neural networks, 230–235 neuron, 230 noise, 104 non-automatic forecasting, 74, 92, 95–97 non-linear autoregressive (NLAR) model, 222 non-linear models, 217–240 non-stationary series, 26, 257 normal process, 36 normalized spectral density function, 113 Nyquist frequency, 109, 124

< previous page

file:///C:/Documents and Settings/Yang/

page_331

next page >

面/The analysis of time series an introduction/files/page_331.html (2 of 2) [5/24/2009 16:54:44]

page_332

page_332

< previous page

next page >

Page 332 observation equation, 264 open-loop system, 197, 242 ordinary least squares (OLS), 88 out-of-sample forecasts, 92 outlier, 6, 27, 29, 98, 270–272 Parseval’s theorem, 127 parsimony, principle of, 47, 71 partial autocorrelation function, 61–62 Parzen window, 132 period, 108, 125 periodogram, 126–130 phase, 107, 127 diagram, 179, 224 shift, 176 spectrum, 161 physically realisable, 171 piecewise linear model, 15 point forecast, 75 point process, 4–5 polynomial curve, 16, 76 polyspectra, 221 portmanteau test, 28, 68 power spectral density function, 111 prediction, 7 interval, 75, 85–87 stage, 211 theory, 103 prewhitening, 150, 158 process control, 4 purely indeterministic process, 50 purely random process, 37, 114, 221 quadratic map, 235 quadrature spectrum, 160 random coefficient model, 223 random walk, 38 plus noise, 206 realization, 34 recurrence form, 77 regime-switching model, 226 regression model, 93, 209 regularization, 233 residual analysis, 67–70 returns, 228, 274 robust methods, 6, 271 rule-based forecasting, 97 runs, 28 sampled series, 5 Santa Fe competition, 234 SARIMA model, 66 scenario analysis, 266 Schwartz’s Bayesian criterion, 256 seasonal adjustment, 21

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_332.html (1 of 2) [5/24/2009 16:54:44]

page_332

seasonal ARIMA (SARIMA) model, 66–67, 83–84, 100 seasonal differencing, 21 seasonal variation, 6, 12–14, 20–22, 26 SEATS, 21 second-order stationary, 36 self-exciting, 224 serial correlation coefficient, 23 SES, see simple exponential smoothing SETAR model, 224 semivariogram, 270 signal, 104 signal-to-noise ratio, 206 simple exponential smoothing, 76, 212 Slutsky effect, 18 smoothing constant, 77 spatial data, 273 spectral analysis, 121–154 spectral density function, 8, 107 estimation of, 121–154 spectral distribution function, 107, 109 spectral representation, 109 spectral window, 140 spectrum, 111 Spencer’s 15 point moving average, 17, 19 spline, 270 S-PLUS, 305 stable system, 171 STAR model, 225 state dependent model, 226 state-space model, 8, 203–214 state variable, 203 state vector, 203 stationary time series, 13, 26 stationary process, 34–36 StatLib index, 292 steady model, 206 steady state, 177 step response function, 174 stepwise autoregression, 85

< previous page

file:///C:/Documents and Settings/Yang/

page_332

next page >

面/The analysis of time series an introduction/files/page_332.html (2 of 2) [5/24/2009 16:54:44]

page_333

page_333

< previous page Page 333 stochastic process, 5, 33 stochastic trend, 15, 263 stochastic volatility model, 229 strange attractor, 237 strict stationarity, 35 strict white noise (SWN), 221 structural change, 259 structural model, 205 summary statistics, 11 sunspots data, 218, 292 system equation, 204 systems approach, 75 tapering, 138 TAR model, 224 tests of randomness, 28 threshold autoregressive (TAR) model, 223–225 time domain, 8, 55 time invariance, 170 time plot, 6, 13–14, 290 time series, 1 time-varying parameter model, 223 tracking signal, 75 training set, 232 TRAMO, 22 transfer function, 175, 195 transfer function model, 195, 246 transformations, 14–15 transient, 177 transition equation, 204 trend, 6, 12, 15–20, 207 trend curve, 76 trend-stationary series, 263 truncation point, 131 Tukey window, 131 turning point, 28 uncorrelated white noise (UWN), 221 unequal intervals, 269 unit root testing, 262 univariate forecast, 74 unobserved components model, 205 updating equations, 211 variance, 34 variogram, 270, 273 VAR model, 246–252 VARMAX model, 250 vector ARMA (VARMA) model, 249 vector autoregressive (VAR) model, 246–252 volatility, 228 wavelength, 108 wavelets, 269 weakly stationary, 36 white noise, 38, 221 Wiener-Khintchine theorem, 110

file:///C:/Documents and Settings/Yang/

面/The analysis of time series an introduction/files/page_333.html (1 of 2) [5/24/2009 16:54:45]

page_333

Wiener-Kolmogorov approach, 103 Wold decomposition, 50 X-11 method, 17, 21 X-12 method, 17, 21 X-12-ARIMA method, 21, 84 Yule-Walker equations, 44–46, 61 z-transform, 297

< previous page

file:///C:/Documents and Settings/Yang/

page_333

面/The analysis of time series an introduction/files/page_333.html (2 of 2) [5/24/2009 16:54:45]