How to detect significant change / trend in a time series data? [closed]

As has been pointed out already, you’re not looking for the derivative. You’re really looking for a “significant change” detection algorithm for a time series.

You’ll certainly want a smoothing filter (and the moving average filter is fine — see Bjorn’s answer for this part).

But in addition to the smoothing filter, you will also need a decision criteria or threshold selector beyond which you will decide whether or not the filtered changes are significant.

If the underlying statistics of your time series is stable (stationary time series), then you can use a fixed statistical threshold, in the sense of standard deviations from the mean. For example, you might choose 2 standard deviations if you want a fairly strong “alarm” threshold (think alarming only on the strongest 5% of returns).

If there is nothing in the underlying problem that suggests that your time series is stable, i.e. if series could have a trend in it, or the underlying process generating the time series can go through fundmantal changes while you’re monitoring it, then you’ll need to use a dynamic, or adaptive threshold, in the sense of signal-to-noise (mu/sigma). You might then choose to detect all “meaningful” elements that pass the signal to noise test.

Leave a Comment