First, make sure that you aren’t accidentally triggering a reset by holding volume + and the center button at the same time. It can be easy to hit too many buttons while trying to hold down just the menu button. A normal shutdown takes a few seconds of holding the menu button before it shuts off.
If you pause playback and leave the express running, it should shut off normally as well.
If it doesn’t restart normally after a normal power-down, I’d suspect corrupt files messing up the database. You could back up all the files on the thing, then reformat and copy them back. A format will get rid of any ghost or corrupted files.
Even with doing all this, I’ve found that the express can sometimes reset rather than shut off, both during normal attempts to shut down and button presses trying to navigate the menus, especially the “return to music list”. I’ve given up on trying to use the Go list for anything more than short term song picks, because I know it will get reset some time or another when connecting to the PC or whatever.