Having to hold in my micro SD card.

[tapeworm]

Are you sure it is pushed in far enough for the locking mechanism to grab and lock it in?

http://forums.sandisk.com/t5/Sansa-Fuze/Fuze-and-micro-sd/m-p/137465/highlight/true#M35663