What is the correct way to silently close InputStream in finally block without losing the original exception?

Best way is to use Java 7 and use try with resources, or do same thing manualy and add exception from closing as suppressed exception.

Pre Java 7:
If you are throwing your custom exception, you can add in it supressed exception like it is done in Java 7 (in your exception create fields List suppressed and put there exceptions from close operation and when dealing with your exception, look there too.
If you cannot do that, I don’t know anything better than just log it.

examples:
from Java tutorials

static String readFirstLineFromFile(String path) throws IOException {
    try (BufferedReader br = new BufferedReader(new FileReader(path))) {
        return br.readLine();
    }
}

but better form is:

static String readFirstLineFromFile(String path) throws IOException {
    try (FileReader fr = new FileReader(path);
         BufferedReader br = new BufferedReader(fr)) {
        return br.readLine();
    }
}

This way even if creation of FileReader is succesfull but creation of BufferedReader fails (eg not enough memory), FileReader will be closed.