Queue browsing is not supported directly, but if you declare a queue with NO auto acknowledgements and do not ACK the messages that you receive, then you can see everything in it. After you have had a look, send a CANCEL on the channel, or disconnect and reconnect to cause all the messages to be requeued. This does increment a number in the message headers, but otherwise leaves the messages untouched.
I built an app where message ordering was not terribly important, and I frequently scanned through the queue in this way. If I found a problem, I would dump the messages into a file, fix them and resubmit.
If you only need to peek at a message or two once in a while you can do that with the RabbitMQ management plugin.
In addition, if you only need a message count, you can get that every time you declare the queue, or on a basic.get command.