This idea came via another thread, but I thought it might be useful to note here…
Rather than putting a volume control between your phones and your Clip, you could try using the EQ to lower all bands – you could then use the custom EQ for radio, and switch back to normal otherwise.
