First of all, are you doing all this moving around in MSC mode? Settings/System Settings/USB Mode/MSC. That’s basic drag and drop. MTP does other, something unpredictable things, and if your unit is on AUTO it may be using MTP. If you’ve been using MTP, the best thing to do is format the card and start over in MSC.
If you format the card from WIndows, it should be FAT32.
The 16GB limit was named in some instructions because those were the biggest cards at the time. 32GB is fine.
Definitely update or reinstall the firmware. Go to Settings/System Settings/Info to see whether your version is 1.x or 2.x. (Those are different versions for different hardware, not an upgrade. Use 1.x if yours is a 1.x.)
From this link
Download the All Regions firmware for your version. Unzip it and drag FuzeA.bin onto the root directory of the internal memory. When you disconnect, the firmware will reinstall. This will not disturb any files you have put on the unit.
You should be able to put albums anywhere on the card, in or out of Music.
I’m leaning toward the explanation that it’s a defective card as well. But also check the format of your music files. They have to be .mp3, .wma, .flac or .ogg. Not m4a, .aac or any other Apple format, which the Fuze will ignore.