How can I get around the lack of a finally block in PHP?

Solution, no. Irritating cumbersome workaround, yes:

$stored_exc = null;
try {
    // Do stuff
} catch (Exception $exc) {
    $stored_exc = $exc;
    // Handle an error
}
// "Finally" here, clean up after yourself
if ($stored_exc) {
    throw($stored_exc);
}

Yucky, but should work.

Please note: PHP 5.5 finally (ahem, sorry) added a finally block: https://wiki.php.net/rfc/finally (and it only took a few years… available in the 5.5 RC almost four years to the date since I posted this answer…)