Try an Olympus digital voice recorder such as the Olympus WS-803 Voice Recorder. It (and the similar models) has a built in music player. The Olympus web site actually has a list of their voice recorder with built-in music players. It has a built-in FM radio. It runs 27 hours [claimed] on 1 AAA battery, it has a lcd screen, it uses a micro-sd card up to 32gb, it has a pop-out USB connection, and if you install a nickel-metal-hydride battery it will recharge the battery through the USB port. It’s not too bulky either, only 1.6" x 3.9" x .6". One big advantage for me is the large controls. I often have a hard time with the tiny controls on many MP3 players. Does not remember bookmarks though. 
I hope this is helpful.
