Yes, you do:
.Net framework will read the app.config once, and never touch the app.config again. That is why you have to restart the application to pick up the change.
Why does not .Net framework detect that app.config has changed, and refresh all the config data?
The reason is simple, this is not possible in general.
Let’s use Raymond’s “imagine this could be done” logic here…