First, let’s recall the reason for the recommendation to use char[] instead of String: Strings are immutable, so once the string is created, there is limited control over the contents of the string until (potentially well after) the memory is garbage collected. An attacker that can dump the process memory can thus potentially read the password data. Meanwhile the contents of the char[] object can be overwritten after it has been created. Assuming that this is done, and that the GC hasn’t moved the object to another physical memory location in the interim, this means that the password contents can be destroyed (somewhat) deterministically after it has been used. An attacker reading the process memory after that point won’t be able to get the password.
So using char[] instead of String lets you prevent a very specific attack scenario, where the attacker has full access to the process memory,1 but only at specific points in time rather than continuously. And even under this scenario, using char[] and overwriting its contents does not prevent the attack, it just reduces its chance of success (if the attacker happens to read the process memory between the creation and the erasure of the password, they can read it).
I am not aware of any evidence that shows (a) how frequent this scenario is, nor (b) how much this mitigation reduces the probability of success under that scenario. As far as I know, this is pure guesswork.
In fact, on most systems, this scenario likely does not exist at all: an attacker who can get access to another process’ memory can also gain full tracing access. For instance, on both Linux and Windows any process that can read another process’ memory can also inject arbitrary logic into that process (e.g. via LD_PRELOAD and similar mechanisms2). So I would say that this mitigation at best has a limited benefit, and potentially none at all.
… Actually I can think of one specific counter-example: an application that loads an untrusted plugin library. As soon as that library is loaded via conventional means (i.e. in the same memory space), it has access to the parent application. In this scenario, it might make sense to use char[] instead of String, and overwrite its contents when done with it, if the password is handled before the plugin is loaded. But a better solution would be not to load untrusted plugins into the same memory space. A common alternative is to launch it in a separate process and communicate via IPC.
(See the answer by Gilles for more vulnerable scenarios. I still think that the benefit is relatively limited, but it’s clearly not nil.)
1 As shown in Gilles’ answer, this is not correct: no full memory access is required to mount a successful attack.
2 Although LD_PRELOAD specifically requires the attacker to not only have access to another process but either to launch that process, or to have access to its parent process.