See the Introducing SQL Server Express Local DB Runtime presentation – gives a great overview.
The huge benefit of LocalDB is that it’s real SQL Server – it’s a special version of SQL Server Express, but it basically supports everything that “real” SQL Server has – spatial data types, stored procedures – you name it.
SQL Server Compact Edition on the other hand is a very much scaled down version – lot of features and datatypes aren’t supported. It’s smaller and more “agile” – but it lacks a lot of punch.
I personally would pick SQL Server Express with the LocalDB runtime over SQL Server CE 4 – unless you specifically need the tiny size of SQL Server CE (e.g. on a mobile device).
Also see a comparison between SQL Server CE 3.5, CE 4.0, SQL Server Express 2008 R2 and SQL Server 2012 Express LocalDB on ErikEJ’s excellent Everything SQL Server Compact web site.