This is common behavior for the Clip Sport. Maybe make sure you are using the most recent Firmware, version 1.29.
http://forums.sandisk.com/t5/SanDisk-Clip-Sport/SanDisk-Clip-Sport-Firmware-1-29-released/td-p/352083