If these warnings are for dependencies you use directly in your code, you should definitely add the upgraded version to your libraryDependencies.
For evicted transitive dependencies (those dependencies only used directly by your own dependencies), it’s likely best to simply leave the warnings in place. This provides documentation to you about possible incompatibilities in your dependencies, and could help you debug runtime problems that arise due to such incompatibilities.
Remember, setting dependencyOverrides merely hides the warning, it doesn’t guarantee compatibility between your libraries and the version you set.