Windows Media Player is the problem. It is probably set to automatically sync–to make the music library on your computer and on your Fuze the same. So if you have copied something to the Fuze and taken it off your computer, WMP makes sure your Fuze matches your computer and the old song isn’t there.
The easy way to stop this is to switch modes. You’re probably in MTP mode. On the Fuze under Settings/USB mode, switch to MSC. That makes the Fuze act like a hard drive. You drag something into Musc (or anywhere) and it stays there. Purely manual.
For some reason you love Windows Media Player? Then stop it from doing things behind your back. Look in the settings on Windows Media player, different for different versions. I have WMP 10. Open WMP, connect the Sansa, go to Tools/Options (you may have to right-click the top bar, the one that says Windows Media Player, if Tools is not displayed.) Look at Devices, find the Sansa, uncheck Start Sync When Device Connects. Look at Sync. You should see a Device Setup, where Automatic is checked. Switch it to Manual. WMP will keep trying to switch back to Automatic, so be careful.
Also, under Rip Music, if for some reason you are ripping to .wma, either switch to mp3 at pretty good quality (192 kbps) OR make sure that if you are using .wma, Copy Protection is un-checked. Copy Protection will screw things up sooner or later.