Yes, I'd defiinitely use some speakers. Transducers of some sort are probably essential for listening to music.
Apple Mini -> Pioneer U-05 -> Adam A7X
ATV -> NuForce Icon HDP -> Genelec 8020C
Sennheiser HD650, HD595; B&W P7; Audio Technica ATH-AD900X, ATH-A900X
I agree, definently something to induce a little noise is preferable with most hifi. although i have heard some where the sound of silence would have been better.