If your headphone has a long straight plug, then I suggest that it not be plugged directly into the player, but instead in a Y cable or cable with a volume control that has a right angle plug, with the right angle plug being plugged into the player. Having a long straight plug sticking out of the player may put plenty of force on the player’s headphone jack if it is pressed against.
You don’t need new earphones or headphones if yours has a long straight plug, you just need something like this. If you are using the earphones that came with the Fuze though, it would be a good idea to get something better.