Well, it depends. Usually, constants are defined at module level. But if you have many constants for category_a and category_b, it might even make sense to add a subpackage constants with modules constants.category_a and constants.category_b.
I would refrain from using a class – it could be instantiated, which wouldn’t make sense, and it has no advantage over a module, apart from allowing you to cram more than one into one physical file (which you probably shouldn’t do if there are so many constants). The Java version would probably use a static class, but the Python equivalent is a module.
Name clashes aren’t an issue in Python except when you import * (but you shouldn’t do that anyways: according to the documentation). As long as there are no name clashes inside the module, rest assured that the user will neither pull out all the names from your module into his own, nor import it under a name that clashes with another module.