I think need copy and omit loc for select columns:
df = df[df['my_col'].notnull()].copy()
df['my_col'] = df['my_col'].astype(int).astype(str)
Explanation:
If you modify values in df later you will find that the modifications do not propagate back to the original data (df), and that Pandas does warning.