You are correct, these are temporary files created by SQLite. If you are manually deleting the main db you should probably delete these too. From what I can gather the WAL is a replacement for the rollback journal that enables SQLite to rollback changes when a transaction fails. How SQLite uses them and why they are kept around for so long is up to the authors of SQLite but in general SQLite seems pretty rock solid so I wouldn’t worry too much about them. For more info take a look here:
http://www.sqlite.org/fileformat2.html#walindexformat
These files are a new feature of SQLite 3.7. I’m not sure if their existence relates to the bug you point out but the bug report suggests a work-around anyway.
UPDATE:
Better documentation about the WAL is here:
https://www.sqlite.org/wal.html
The contents of the WAL are periodically moved to the DB file but this is not guaranteed to occur each time the process exits. Thus when WAL is enabled each SQLite DB consists of two files on disk that must be preserved, both the .db file and the .db-wal file.
The .db-shm file is a shared memory file that contains only temporary data.