I ended up with http://jsfiddle.net/7zkCU/30/ (adapted from http://maxwells.github.io/bootstrap-tags.html):
<span class="tag label label-info">
<span>Example Tag</span>
<a><i class="remove glyphicon glyphicon-remove-sign glyphicon-white"></i></a>
</span>
CSS:
.tag {
font-size: 14px;
padding: .3em .4em .4em;
margin: 0 .1em;
}
.tag a {
color: #bbb;
cursor: pointer;
opacity: 0.6;
}
.tag a:hover {
opacity: 1.0
}
.tag .remove {
vertical-align: bottom;
top: 0;
}
.tag a {
margin: 0 0 0 .3em;
}
.tag a .glyphicon-white {
color: #fff;
margin-bottom: 2px;
}