How could/should I state colon (punctuation) in a YAML file?

You should be able to double quote the value:

test_key_html: "Test value:"

This avoids colon-confusion in the YAML and gets your colon into your HTML.

Consider this in irb:

>> { 'en' => { 'test_key_html' => 'Test value:' } }.to_yaml
=> "--- \nen: \n  test_key_html: "Test value:"\n"