You can do a check on the object before appending it:
sb.append("Value: ");
if (s != null) sb.append(s);
System.out.println(sb);
A key point to make is that null is not the same an an empty String. An empty String is still a String object with associated methods and fields associated with it, where a null pointer is not an object at all.
From the documentation for StringBuilder’s append method:
The characters of the String argument are appended, in order, increasing the length of this sequence by the length of the argument. If str is null, then the four characters “null” are appended.