If you have a checkout of the repository at hand, you can use the ^ (caret, search for it in the manual) notation to reference the root of the repo like this:
svn diff --old ^/branches/0.4.x --new ^/trunk
This works since Subversion 1.6.
If you have an older subversion or no handy checkout of the repo, you can use absolute paths, as described in the original redbook:
svn diff --old http://.../repo/branches/0.4.x --new http://.../repo/trunk/
should give you the answer you’re looking for.
Replace http://.../repo/ with the actual URL of your repository.