The option handle of the plugin allows you to define that is the element that can initiate the sort. You can provide a selector or an element.
If you have this html, with the .handler to be the handle to start the sort:
<ul class="sortable">
<li>
<span class="handle"></span>
My element
</li>
</ul>
Apply the option like this:
$( ".sortable" ).sortable({ handle: '.handle' });
You can style your handle element however you like.