This is caused by “instant run” feature in studio 2.3, you can disable it by:
Open the Settings or Preferences dialog.
Navigate to Build, Execution, Deployment > Instant Run.
Uncheck the box next to Enable Instant Run.
More info, please check:
https://developer.android.com/studio/run/index.html