Rails by default doesn’t serve assets under public. See your production.rb:
config.serve_static_assets = true
Change that to true and you’re good to go. (Note: you don’t want that to be true in production, remember to change it back before deploying!)
See Configuring Rails Applications for details.
In rails 6, in the default production.rb there should be a line
config.public_file_server.enabled = ENV['RAILS_SERVE_STATIC_FILES'].present?
So run your server with
RAILS_SERVE_STATIC_FILES=true rails server -e production
or set config.public_file_server.enabled=true in production.rb. See answers below for rails 4 and 5.