I noticed a bunch of files that start with a ~ at the beginning, along with other files in the folder. Usually the rest of the name of this file is the same as the tax file name. I could try to delete them but am worried that I may lose my latest tax return information.
They are called tilde files and its a Windows/Computer thing, not directly related to the TurboTax program. They are basically backup, temporary files that are created while you are using the program. They are suppose to be removed once you close out of the program completely, but sometimes Windows keeps them around - especially if you don't close out properly.
They are just backup files, so you should be able to delete them without issue, but you won't be able to delete them with the main file open. Though, I would wait until you are completely done and have filed your return before you delete anything if you really want to remove them. Also, try making sure your program is completely closed out and do a computer restart to see if that clears them.