I might be missing what you really want, but without using geom_text hack you can still set the limits
ggplot(mtcars, aes(x = as.factor(carb))) +
geom_bar() +
scale_y_continuous(expand = c(0, 0), limits = c(0, 10.3))
# marginally cleaner