You can manually create a subscription to your store and persist the changes to localStorage and also use the potential value in localStorage as default value.
Example
<script>
import { writable } from "svelte/store";
const store = writable(localStorage.getItem("store") || "");
store.subscribe(val => localStorage.setItem("store", val));
</script>
<input bind:value={$store} />