Do you have to write a custom less for use in std::map with pointers, or can you use std::less?

Two pointers can be compared with using the comparison function objects less, greater etc. Otherwise, using blanket operator< etc, this is only possible if the pointers point to elements of the same array object or one past the end. Otherwise, results are unspecified.

20.3.3/8 in C++03

For templates greater, less, greater_equal, and less_equal, the specializations for any
pointer type yield a total order, even if the built-in operators <, >, <=, >= do not.

No need to explicitly specialize and manually casting to size_t: That would lower the portability even, since the mapping of reinterpret_cast from pointers to integers is implementation defined and is not required to yield any order.

Edit: For a more detailed answer, see this one.