You can do it in the player, via settings, system settings.
Clip+ and Zip need FAT32 formatting, the Sport works without extra formatting, as I have found out. But it can’t hurt to format the cards nonetheless to be on the safe side.
An exFat formatted 64 GB card has worked fine on my Sport. The same after reformatting to Fat32. BTW, formatting also helps to erase the files that come with the players and you don’t need (some of them aren’t visible in Windows when uploaded in the opposite USB mode).
You don’t need to format a 32GB card, these always come FAT32 formatted.
Forgot to mention: Yes, you’re right about 32 GB cards. (My thoughts were with 64 GB cards which I was occupied with these days.) And there is no opposite USB mode on the Sport!