I tried the option to add AddKeysToAgent yes to ~/.ssh/config but it doesn’t keep it between new tabs on the terminal.
The best solution I found so far is to do the following:
sudo apt install keychain
Find your hostname using the terminal:
hostname
Then add the following to your ~/.bashrc or ~/.zshrc file:
/usr/bin/keychain --nogui ~/.ssh/id_rsa
source $HOME/.keychain/YOUR-HOSTNAME-HERE-sh
Now, each time you reboot, you’ll have to enter your passphrase. But you only have to do it one time until you reboot or terminate WSL.
If you want to use the same key you already have on Windows you can follow this post Sharing SSH keys between Windows and WSL 2