It sounds like you are using NuGet and the packages folder is the NuGet packages folder. You can commit this to source control if you do not mind adding several large but infrequently changing binaries.
Alternatively, you can omit it from source control and configure NuGet to download packages on each build if developers and the build machine are to download it as required. This also runs the danger of someone downloading a newer version and causing conflicts.
As for other files to exclude form source control, see What .net files should be excluded from source control? and For a .Net project, what file extensions should I exclude from source control?.