Most other answers seem to mention SQL based databases. NoSQL based databases are far superior at this kind of thing. Some Open source time-series databases: https://prometheus.io – Monitoring system and time series database http://influxdb.com/ – time series database with no external dependencies (only basic server is open-source) http://square.github.io/cube/ – Written ontop of MongoDB http://opentsdb.net/ – … Read more
In my original post, I suggested using pd.TimeGrouper. Nowadays, use pd.Grouper instead of pd.TimeGrouper. The syntax is largely the same, but TimeGrouper is now deprecated in favor of pd.Grouper. Moreover, while pd.TimeGrouper could only group by DatetimeIndex, pd.Grouper can group by datetime columns which you can specify through the key parameter. You could use a … Read more
Here is a sample result from a small project I did to partition ecg data. My approach was a “switching autoregressive HMM” (google this if you haven’t heard of it) where each datapoint is predicted from the previous datapoint using a Bayesian regression model. I created 81 hidden states: a junk state to capture data … Read more
pd.Grouper allows you to specify a “groupby instruction for a target object”. In particular, you can use it to group by dates even if df.index is not a DatetimeIndex: df.groupby(pd.Grouper(freq=’2D’, level=-1)) The level=-1 tells pd.Grouper to look for the dates in the last level of the MultiIndex. Moreover, you can use this in conjunction with … Read more
Simple plot, you can use: df.plot(x=’Date’,y=’adj_close’) Or you can set the index to be Date beforehand, then it’s easy to plot the column you want: df.set_index(‘Date’, inplace=True) df[‘adj_close’].plot() If you want a chart with one series by ticker on it You need to groupby before: df.set_index(‘Date’, inplace=True) df.groupby(‘ticker’)[‘adj_close’].plot(legend=True) If you want a chart with individual … Read more
You can implement a number of approaches: ARIMAResults include aic and bic. By their definition, (see here and here), these criteria penalize for the number of parameters in the model. So you may use these numbers to compare the models. Also scipy has optimize.brute which does grid search on the specified parameters space. So a … Read more
Courtesy of the ?strptime help file (with the example changed to your value): > z <- strptime(“2010-01-15 13:55:23.975”, “%Y-%m-%d %H:%M:%OS”) > z # prints without fractional seconds [1] “2010-01-15 13:55:23 UTC” > op <- options(digits.secs=3) > z [1] “2010-01-15 13:55:23.975 UTC” > options(op) #reset options
plt.axvline(x_position) It takes the standard plot formatting options (linestlye, color, ect) (doc) If you have a reference to your axes object: ax.axvline(x, color=”k”, linestyle=”–“)
In the meantime, a time-window capability was added. See this link. In [1]: df = DataFrame({‘B’: range(5)}) In [2]: df.index = [Timestamp(‘20130101 09:00:00’), …: Timestamp(‘20130101 09:00:02’), …: Timestamp(‘20130101 09:00:03’), …: Timestamp(‘20130101 09:00:05’), …: Timestamp(‘20130101 09:00:06’)] In [3]: df Out[3]: B 2013-01-01 09:00:00 0 2013-01-01 09:00:02 1 2013-01-01 09:00:03 2 2013-01-01 09:00:05 3 2013-01-01 09:00:06 4 … Read more
Try as.numeric(mydata$datefield[120]): gg + geom_vline(xintercept=as.numeric(mydata$datefield[120]), linetype=4) A simple test example: library(“ggplot2″) tmp <- data.frame(x=rep(seq(as.Date(0, origin=”1970-01-01″), length=36, by=”1 month”), 2), y=rnorm(72), category=gl(2,36)) p <- ggplot(tmp, aes(x, y, colour=category)) + geom_line() + geom_vline(xintercept=as.numeric(tmp$x[c(13, 24)]), linetype=4, colour=”black”) print(p)