Yes, it’s possible, just copy the installation directory to the flash drive, then edit IDEA_HOME\bin\idea.properties file, change the values of idea.config.path and idea.plugins.path to the relative location, like: ../config and ../user-plugins (locations are relative to IDEA bin directory).
Now copy your original settings and third-party plug-ins (if any):
${user.home}/.IntelliJIdea10/config=>usb:/IDEA_HOME/config${user.home}/.IntelliJIdea10/config/plugins=>usb:/IDEA_HOME/user-plugins
Note that it’s not recommended to change idea.system.path variable, so that it remains on the fast local drive, otherwise it could affect IDEA performance and occupy a lot of USB drive space.