From the HBase shell you can use LIMIT:
hbase> scan 'test-table', {'LIMIT' => 5}
From the Java API you can use Scan.setMaxResultSize(N) or scan.setMaxResultsPerColumnFamily(N).
- HBase API docs – Scan.setMaxResultSize
- HBase API docs – Scan.setMaxResultsPerColumnFamily