time-series
Pandas finding local max and min
The solution offered by fuglede is great but if your data is very noisy (like the one in the picture) you will end up with lots of misleading local extremes. I suggest that you use scipy.signal.argrelextrema() method. The .argrelextrema() method has its own limitations but it has a useful feature where you can specify the … Read more
Compressing floating point data
Here are some ideas if you want to create your own simple algorithm: Use xor of the current value with the previous value to obtain a set of bits describing the difference. Divide this difference into two parts: one part is the “mantissa bits” and one part is the “exponent bits”. Use variable length encoding … Read more
Time Series Decomposition function in Python
I’ve been having a similar issue and am trying to find the best path forward. Try moving your data into a Pandas DataFrame and then call StatsModels tsa.seasonal_decompose. See the following example: import statsmodels.api as sm dta = sm.datasets.co2.load_pandas().data # deal with missing values. see issue dta.co2.interpolate(inplace=True) res = sm.tsa.seasonal_decompose(dta.co2) resplot = res.plot() You can … Read more
Pandas Resampling error: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex
Convert the integer timestamps in the index to a DatetimeIndex: data.index = pd.to_datetime(data.index, unit=”s”) This interprets the integers as seconds since the Epoch. For example, given data = pd.DataFrame( {‘Timestamp’:[1313331280, 1313334917, 1313334917, 1313340309, 1313340309], ‘Price’: [10.4]*3 + [10.5]*2, ‘Volume’: [0.779, 0.101, 0.316, 0.150, 1.8]}) data = data.set_index([‘Timestamp’]) # Price Volume # Timestamp # 1313331280 10.4 … Read more
Sliding window of M-by-N shape numpy.ndarray
You can do a vectorized sliding window in numpy using fancy indexing. >>> import numpy as np >>> a = np.array([[00,01], [10,11], [20,21], [30,31], [40,41], [50,51]]) >>> a array([[ 0, 1], [10, 11], [20, 21], #define our 2d numpy array [30, 31], [40, 41], [50, 51]]) >>> a = a.flatten() >>> a array([ 0, 1, … Read more