Workaround for the issue was to override Spring Boot’s maven-surefire-plugin definition and set useSystemClassLoader to false. Read Surefire docs for more details
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<useSystemClassLoader>false</useSystemClassLoader>
</configuration>
</plugin>
</plugins>
</build>