Ah, the wee Clip+ is driving an interesting load there. You could get creative, and use a dual-3.5mm adaptor, with a headphone plugged in. The open jack runs to your stereo. If all is well, you could make a shunt adaptor with a dummy load of, say, 100 ohms on each channel, and try dropping down to something in the 50-ohm range (using another parallel pair of 100 ohm resistors).
If the device is happy, you can modify a basic patch cord with the required load resistance.
Amplifiers have a widely variable input impedance, depending upon the application. The Clip+ may see this load, and be trying to protect the output.
For a home stereo, I like the “bigger” Sansas, resting in a dock. The Griffin Sansa dock is one such monster, and they’re available really cheap. You will have the advantage of a stable device on the desk / countertop, DC power for it from the dock, and a handy larger display.