Use the built in first_valid_index and last_valid_index they are designed specifically for this and slice your df: In [5]: first_idx = df.first_valid_index() last_idx = df.last_valid_index() print(first_idx, last_idx) df.loc[first_idx:last_idx] 1950 1954 Out[5]: sum 1950 5 1951 3 1952 NaN 1953 4 1954 8
If you have a Series like: In [116]: df[“Date”] Out[116]: 0 2012-10-08 07:12:22 1 2012-10-08 09:14:00 2 2012-10-08 09:15:00 3 2012-10-08 09:15:01 4 2012-10-08 09:15:01.500000 5 2012-10-08 09:15:02 6 2012-10-08 09:15:02.500000 7 2012-10-10 07:19:30 8 2012-10-10 09:14:00 9 2012-10-10 09:15:00 10 2012-10-10 09:15:01 11 2012-10-10 09:15:01.500000 12 2012-10-10 09:15:02 Name: Date where each object is … Read more
I know this is late to the game here but a few points. Whether or not a DataFrame is considered a TimeSeries is the type of index. In your case, your index is already a TimeSeries, so you are good to go. For more information on all the cool slicing you can do with a … Read more
Having the same ValueError, this is just the result of some testing and little research on my own, without the claim to be complete or professional about it. Please comment or answer whoever finds something wrong. Of course, your data should be in the right order of the index values, which you would assure with … Read more
I think you could use a TimeSeriesSplit() either instead of your own implementation or as a basis for implementing a CV method which is exactly as you describe it. After digging around a bit, it seems like someone added a max_train_size to the TimeSeriesSplit() in this PR, which seems like it does what you want.
You have a couple options here: pd.infer_freq pd.tseries.frequencies.to_offset I suspect that errors down the road are caused by the missing freq. You are absolutely right. Here’s what I use often: def add_freq(idx, freq=None): “””Add a frequency attribute to idx, through inference or directly. Returns a copy. If `freq` is None, it is inferred. “”” idx … Read more
I hope this helps. I’ve always had a hard time with matplotlib’s dates. Matplotlib requires a float format which is days since epoch. The helper functions num2date and date2num along with python builtin datetime can be used to convert to/from. The formatting business was lifted from this example. You can change an axis on any … Read more