Jump to byte address in vim?

Yes, there is. It’s the normal mode command go.

From :h go:

[count]go         Go to {count} byte in the buffer.

For example, 42go jumps to byte 42.

In order to jump to a hexadecimal or octal address you would need to construct a command with :normal! go or the equivalent :goto, and the str2nr() or printf() function.

:exe 'normal! ' . str2nr('0x2a', 16) . 'go'
:exe 'goto' str2nr('0x2a', 16)