Wiki
Advanced13 min read

Time series forecasting

Autocorrelation, stationarity, and the rule that past values carry information — which is why the order of observations is part of the model.

Every other chapter in this domain treated rows as exchangeable — shuffle them and nothing changes. A time series is the opposite. Each observation is stamped with a position, and adjacent positions are related: today's temperature is a good guess for tomorrow's, today's stock price is a decent guess for tomorrow's, and a shuffled train-test split destroys the very structure you are trying to predict.

The explorer below builds a series from a trend, a seasonal cycle, and an AR(1) term, computes its sample autocorrelation, and fits an AR(1) model by the method of moments. Turn the persistence dial and watch the ACF stretch out and the forecast hug the last value more tightly.

Build a series, read its autocorrelation, and extrapolate an AR(1) forecast

solid = observed · dashed = AR(1) forecast · shaded = 95% band

Sample autocorrelation

lags 0–12 · φ̂ from lag-1 = 0.913

φ → 0: the ACF collapses to zero after lag 1 and the forecast decays instantly to the mean. φ → 1: correlation persists across lags and the forecast tracks the last value, with a wider band.

The ACF and the AR(1) fit are real calculations on a synthetic series; the generating process is a toy. A commercial forecaster would add seasonality explicitly (SARIMA), use exogenous regressors, and validate on a rolling origin — but every one of those builds on exactly this idea: past values carry information about the future, measured by autocorrelation.

Autocorrelation is the whole idea

A series is autocorrelated if it resembles a shifted copy of itself. The autocorrelation function measures that resemblance at each lag, and it is the single most informative diagnostic: it tells you the memory length of the process, which in turn tells you which model class can represent it.

Stationarity, and why we difference

Most classical methods assume weak stationarity: the mean and the autocovariance do not depend on absolute time, only on the lag. A trending series is not stationary, which is why the first operation is often differencing,

yt′=yt−yt−1,y'_t = y_t - y_{t-1},

which removes a linear trend. Seasonal differencing, yt−yt−my_t - y_{t-m}, removes a repeating cycle of period mm. The ARIMA family is defined by (p,d,q)(p, d, q): pp autoregressive lags, dd differences, qq moving-average lags. Choosing them is reading the ACF and partial ACF, or letting a criterion such as AIC do it under constraints.

AR, MA, and the AR(1) model

An autoregressive model predicts from past values. The simplest, AR(1), is

yt=c+φ yt−1+εt,εt∼N(0,σ2).y_t = c + \varphi\,y_{t-1} + \varepsilon_t, \qquad \varepsilon_t \sim \mathcal{N}(0, \sigma^2).

With ∣φ∣<1|\varphi| < 1 the process is stationary, its long-run mean is μ=c/(1−φ)\mu = c/(1-\varphi), and its variance is

Var⁡(y)=σ21−φ2.\operatorname{Var}(y) = \frac{\sigma^2}{1-\varphi^2}.

The theoretical ACF decays geometrically, ρk=φk\rho_k = \varphi^k, which is exactly the shape to look for in the artifact. A moving-average model, by contrast, predicts from past errors and has an ACF that cuts off abruptly after lag qq. Forecasts from AR(1) revert to the mean as the horizon grows: at horizon hh, c+φh(yT−μ)+μc + \varphi^h (y_T - \mu) + \mu, so the further out you look, the less the last value matters and the more the interval widens — which is why the shaded band fans out.

Evaluation that respects time

A random train-test split is invalid here for exactly the reason it works elsewhere. The correct scheme is a rolling origin (or expanding-window) backtest: train on everything up to time tt, predict t+1…t+ht+1 \ldots t+h, advance, repeat. Report the error over many origins, use a scale-free metric — MAPE or MASE, rather than RMSE on a series whose units change — and always compare against naive baselines: the last value, and the value from the same season one period ago. Many published gains over those baselines are small once the benchmarks are honest.

Careful

Forecast accuracy degrades with the horizon, and a point forecast without an interval hides that. Decisions that bet on the forecast — capacity, staffing, inventory — need the uncertainty, not just the mean, or they will be tuned to the median case and fail on the tail. Also resist fitting a model to the whole series and reporting in-sample fit; it is not a forecast, and it is always flattering.

Illustrative vs real

The artifact generates a synthetic series from a known recipe, so the true generating process is visible — which never happens in practice. The sample ACF and the method-of-moments AR(1) fit are computed correctly, but real series need order selection, stationarity testing, and rolling-origin validation. The explorer teaches the mechanism, not a full forecasting workflow.

Check yourself

Eduspheria wiki · Data, MLOps & Deployment, Time series

0 / 5 answered

  1. 1An AR(1) process has φ = 0.5 and innovation variance σ² = 1. What is the stationary variance σ² / (1 − φ²)?
    Numeric answer
  2. 2Why is a random train-test split invalid for a time series?
    Multiple choice
  3. 3Which operation removes a linear trend to make a series closer to stationary?
    Short answer
  4. 4In a stationary AR(1) model, the theoretical autocorrelation at lag k is φ raised to the power k.
    True / false
  5. 5Which baseline should any new forecasting model be compared against?
    Multiple choice

From the assignment paper

Modeled on NITJ AI-511, Assignment/Quiz

0 / 5 answered

  1. 1A forecasting model is automatically retrained as new production data arrives. In MLOps terms this is an example of…
    Multiple choice
  2. 2In industrial sensor data, which oddity may look like an error but can actually be normal equipment behaviour?
    Multiple choice
  3. 3A high-variance model tends to generalise poorly to unseen data.
    True / false
  4. 4An AR(1) process has φ = 0.8 and innovation variance σ² = 1. What is the stationary variance σ² / (1 − φ²)?
    Numeric answer
  5. 5Why may experimentation need to be repeated periodically over a model's life?
    Multiple choice

Where next: this closes the Data, MLOps & Deployment domain — from a decision-shaped question, through features and pipelines, to a model that is shipped, monitored, and eventually retrained.