The ControllerAdvice annotation has an element/attribute called basePackage which can be set to determine which packages it should scan for Controllers and apply the advices. So, what you can do is to separate those Controllers handling normal requests and those handling AJAX requests into different packages then write 2 Exception Handling Controllers with appropriate ControllerAdvice annotations. For example:
@ControllerAdvice("com.acme.webapp.ajaxcontrollers")
public class AjaxExceptionHandlingController {
...
@ControllerAdvice("com.acme.webapp.controllers")
public class ExceptionHandlingController {