You need to exclude those directories in your Windows Security’s Exclusion list.
- Type Windows Security in your windows Search bar
- Click Virus & threat protection
- Under Virus & threat protection settings header, click Manage Settings
- Under Exclusions header, click Add or remove exclusions
- Click Add an exclusion
- Select type Folder
- Enter the directory
- You are done
Yes it is secure, necessary and really improves the performance as the real-time scanning is enabled in Windows defender by default.
Read this: Minimize the impact of antivirus software on build speed.