If you change this line in your function: dfItem = jsonToDataFrame(data) to: dfItem = pd.DataFrame.from_records(data) it should work. I tested your function with this line replaced, using [‘INAG’] as a parameter passed to your getFinanceHistoricalStockFromByma function, and it returned a DataFrame.
If I understand well the issue, you are looking for the concat function. pandas.concat([df1, df2, df3, df4]) should work correctly if the column names are the same for both dataframes.
I’ve been searching the web for a solution but couldn’t find any so here is what we came up with (there might be better ways but at least this is a start if someone else runs into this). Specify the dtype parameter in to_sql. We went from:df.to_sql(table_name, analytics_db) to df.to_sql(table_name, analytics_db, dtype={‘name_of_json_column_in_source_table’: sqlalchemy.types.JSON}) and it … Read more
Personally, I’d mistyped the type annotation class Foo(BaseModel): bar = Optional[NonNegativeInt] Rather than; class Foo(BaseModel): bar: Optional[NonNegativeInt] Silly one ik, but double check that 🙂
As stated in merge, join, and concat documentation, ignore index will remove all name references and use a range (0…n-1) instead. So it should give you the result you want once you remove ignore_index argument or set it to false (default). df = pd.concat([df, df2], axis=1) This will join your df and df2 based on … Read more
You need inplace=True df[1].fillna(0, inplace=True)
You could use any method to check if that condition is True at least for the one value: In [36]: (s > 1).any() Out[36]: True
Here’s a way to clear exactly 10% of cells (or rather, as close to 10% as can be achieved with the existing data frame’s size). import random ix = [(row, col) for row in range(df.shape[0]) for col in range(df.shape[1])] for row, col in random.sample(ix, int(round(.1*len(ix)))): df.iat[row, col] = np.nan Here’s a way to clear cells … Read more
In my case, 3M rows having 5 columns were inserted in 8 mins when I used pandas to_sql function parameters as chunksize=5000 and method=’multi’. This was a huge improvement as inserting 3M rows using python into the database was becoming very hard for me.
The save() method has been deprecated and removed in Pandas. You should use close() instead. With older versions of Pandas you would have gotten this warning: FutureWarning: save is not part of the public API, usage can give unexpected results and will be removed in a future version With more recent versions you just need … Read more