In Spring Transactions, you use TransactionStatus.setRollbackOnly().
The problem you have here is that you’re using @Transactional to demarcate your transactions. This has the benefit of being non-invasive, but it also means that if you want to manually interact with the transaction context, you can’t.
If you want tight control of your transaction status, you have to use programmatic transactions rather than declarative annotations. This means using Spring’s TransactionTemplate, or use its PlatformTransactionManager directly. See section 9.6 of the Spring reference manual.
With TransactionTemplate, you provide a callback object which implements TransactionCallback, and the code in this callback has access to the TransactionStatus objects.
It’s not as nice as @Transactional, but you get closer control of your tx status.