How can I remove jquery from the frontside of my WordPress?

The correct method to completely remove a style or script is to dequeue it and deregister it. You should also note that front end scripts are handled through the wp_enqueue_scripts hook while back end scripts are handled through the admin_enqueue_scripts hook.

So with that in mind, you can do the following

add_filter( 'wp_enqueue_scripts', 'change_default_jquery', PHP_INT_MAX );

function change_default_jquery( ){
    wp_dequeue_script( 'jquery');
    wp_deregister_script( 'jquery');   
}

EDIT 1

This has been fully tested on WordPress version 4.0 and working as expected.

EDIT 2

As proof of concept, paste the following code in your functions.php. This will print a success or failure message in the head of your site, back end and front end

add_action( 'wp_head', 'check_jquery' );
add_action( 'admin_head', 'check_jquery' );
function check_jquery() {

    global $wp_scripts;

    foreach ( $wp_scripts->registered as $wp_script ) {
        $handles[] = $wp_script->handle; 
    }

    if( in_array( 'jquery', $handles ) ) {
        echo 'jquery has been loaded';
    }else{
        echo 'jquery has been removed';
    }
}