I suspect that somewhere your application has a reference to the ByteBuffer instance(s) and that is preventing it from being garbage collected.
The buffer memory for a direct ByteBuffer is allocated outside of the normal heap (so that the GC doesn’t move it!!). However, the ByteBuffer API provides no method for explicitly disposing of / deallocating a buffer. So I assume that the garbage collector will do it … once it determines that the ByteBuffer object is no longer referenced.