Clip+ Sound Quality: Original Firmware vs. Rockbox

SanDisk MP3 players Sansa® Clip | Sansa® Clip+
1

Anybody done any measurements of Clip+ output fidelity/quality, comparing original firmware vs. Rockbox?

I compared Rockbox 3.7.1 vs. original firmware 01.02.15 on the Clip+.  It sounds to my ears like the original firmware is slightly better fidelity/quality output than Rockbox.  I did not do a true blind ABX comparison or anything like that, just switched back and forth between OF and RB repeatedly, on the same passages of certain music FLACs.

The difference is very slight in my estimation, but I can hear it.  Again this is subjective and not truly blind, but I don’t think there is “user bias” influencing the results because I really want to use RB instead of OF, and RB should yield output fidelity at least as good as OF by all indicators.  Unfortunately, my ears say otherwise.

Curious to hear other opinions and especially to see any data that may be out there.

2

I know what you are talking about. I used to run rockbox on my sansa fuze and I could hear a slight but definite difference in the sound quality. While the fuze’s SQ wasn’t really great in comparision with the clip’s, it seemed clearer on the OF than on rockbox. It was still enjoyable but it definitly wasn’t as great as what I could get on the OF or with the sansa clip.

Sansa fuze 4GB + 2GB sandisk uSD card &qu
Sansa clip 2GB MSC mode
Windows 7 Home edition 64x
Ubuntu 10.10 32x
Mandriva 2010 32x

3

@paul_willy wrote:

I know what you are talking about. I used to run rockbox on my sansa fuze and I could hear a slight but definite difference in the sound quality. While the fuze’s SQ wasn’t really great in comparision with the clip’s , it seemed clearer on the OF than on rockbox. It was still enjoyable but it definitly wasn’t as great as what I could get on the OF or with the sansa clip.

The Fuze and Clip sound the same…they run off the same AMS chip, you know.

4

@sandclip wrote:

Anybody done any measurements of Clip+ output fidelity/quality, comparing original firmware vs. Rockbox?

 

I compared Rockbox 3.7.1 vs. original firmware 01.02.15 on the Clip+.  It sounds to my ears like the original firmware is slightly better fidelity/quality output than Rockbox.  I did not do a true blind ABX comparison or anything like that, just switched back and forth between OF and RB repeatedly, on the same passages of certain music FLACs.

 

The difference is very slight in my estimation, but I can hear it.  Again this is subjective and not truly blind, but I don’t think there is “user bias” influencing the results because I really want to use RB instead of OF, and RB should yield output fidelity at least as good as OF by all indicators.  Unfortunately, my ears say otherwise.

 

Curious to hear other opinions and especially to see any data that may be out there.

I assume you’re doing the switch with the EQ diabled? With the EQ disabled, it shouldn’t sound any different…the hardware didn’t change.

5

@marvin_martian wrote:

 

@sandclip wrote:

Anybody done any measurements of Clip+ output fidelity/quality, comparing original firmware vs. Rockbox?

 

I compared Rockbox 3.7.1 vs. original firmware 01.02.15 on the Clip+.  It sounds to my ears like the original firmware is slightly better fidelity/quality output than Rockbox.  I did not do a true blind ABX comparison or anything like that, just switched back and forth between OF and RB repeatedly, on the same passages of certain music FLACs.

 

The difference is very slight in my estimation, but I can hear it.  Again this is subjective and not truly blind, but I don’t think there is “user bias” influencing the results because I really want to use RB instead of OF, and RB should yield output fidelity at least as good as OF by all indicators.  Unfortunately, my ears say otherwise.

 

Curious to hear other opinions and especially to see any data that may be out there.

I assume you’re doing the switch with the EQ diabled? With the EQ disabled, it shouldn’t sound any different…the hardware didn’t change.

 

Yes, compared them with EQ disabled.

I wonder if the change is somehow related to the different clocking done by OF vs. RB.  I read that the RB frequency accuracy is better, related to this, but maybe it also affects other aspects of sound quality.  (Which might be one reason SanDisk is apparently not going to improve frequency accuracy with their firmware.)

The OF sounds clearer; I can hear both lower and higher frequencies more clearly, less distorted sounding, and more detail within the music.

As for the frequency error of the OF (+0.2%):  I hear no discernible difference in pitch, switching back and forth between OF and RB on a given music passage.  I’m sure that if I heard a test sine wave tone played simultaneously by OF against an accurate tone of same frequency I would hear the phase shifting due the two being “out of tune”, but just listening to a music track they sound the same, played back-to-back.  At least, to my ears.

6

@sandclip wrote:

 

@marvin_martian wrote:

 

@sandclip wrote:

Anybody done any measurements of Clip+ output fidelity/quality, comparing original firmware vs. Rockbox?

 

I compared Rockbox 3.7.1 vs. original firmware 01.02.15 on the Clip+.  It sounds to my ears like the original firmware is slightly better fidelity/quality output than Rockbox.  I did not do a true blind ABX comparison or anything like that, just switched back and forth between OF and RB repeatedly, on the same passages of certain music FLACs.

 

The difference is very slight in my estimation, but I can hear it.  Again this is subjective and not truly blind, but I don’t think there is “user bias” influencing the results because I really want to use RB instead of OF, and RB should yield output fidelity at least as good as OF by all indicators.  Unfortunately, my ears say otherwise.

 

Curious to hear other opinions and especially to see any data that may be out there.

I assume you’re doing the switch with the EQ diabled? With the EQ disabled, it shouldn’t sound any different…the hardware didn’t change.

 

 

Yes, compared them with EQ disabled.

 

I wonder if the change is somehow related to the different clocking done by OF vs. RB.  I read that the RB frequency accuracy is better, related to this, but maybe it also affects other aspects of sound quality.  (Which might be one reason SanDisk is apparently not going to improve frequency accuracy with their firmware.)

 

The OF sounds clearer; I can hear both lower and higher frequencies more clearly, less distorted sounding, and more detail within the music.

 

As for the frequency error of the OF (+0.2%):  I hear no discernible difference in pitch, switching back and forth between OF and RB on a given music passage.  I’m sure that if I heard a test sine wave tone played simultaneously by OF against an accurate tone of same frequency I would hear the phase shifting due the two being “out of tune”, but just listening to a music track they sound the same, played back-to-back.  At least, to my ears.

Well, since your library is mostly FLAC, according to your sig, then you really have no choice but to use Rockbox, unless you want to charge your player every six or seven hours…lol. At least the excellent EQ should let you compensate for the slight difference you hear…although it may involve changing up some frequency bands from their stock settings.

7

@marvin_martian wrote:

 

<snip>

Well, since your library is mostly FLAC, according to your sig, then you really have no choice but to use Rockbox, unless you want to charge your player every six or seven hours…lol. At least the excellent EQ should let you compensate for the slight difference you hear…although it may involve changing up some frequency bands from their stock settings.

 

I only need about 2 hours max play time between charges most of the time, 4 hours tops.  The unit is charged after each use regardless of how much it is discharged; Li-Ion does best if you keep it charged up, no memory effect to worry about.  So the 7 hours runtime give or take is fine - excellent in fact, compared to the old days before Li-Ion.  :slight_smile:

An EQ will not mitigate out what sounds like distortion or muddiness, or maybe intermodulation distortion.

8

@sandclip wrote:

 

@marvin_martian wrote:

 

<snip>

Well, since your library is mostly FLAC, according to your sig, then you really have no choice but to use Rockbox, unless you want to charge your player every six or seven hours…lol. At least the excellent EQ should let you compensate for the slight difference you hear…although it may involve changing up some frequency bands from their stock settings.

 

I only need about 2 hours max play time between charges most of the time, 4 hours tops.  The unit is charged after each use regardless of how much it is discharged; Li-Ion does best if you keep it charged up, no memory effect to worry about.  So the 7 hours runtime give or take is fine - excellent in fact, compared to the old days before Li-Ion.  :slight_smile:

 

 

An EQ will not mitigate out what sounds like distortion or muddiness, or maybe intermodulation distortion.

It was a thought, anyways…for me, there is no going back to the original firmware…I don’t hear whatever it is you’re hearing that you don’t like, and some of my headphones need the RB EQ in order to get the proper balance of sound, and I like the battery life bonus too.:smiley:

Per your original question about measurements with Rockbox and the OF, there may be something of the sort over on AnythingbutiPod, if you’re good with a search engine. I know there are RMAA measurements there somewhere, but I don’t remember how extensive they were.

9

I know that both of them are supposed to have the same sound quality but, for some unknown reason, both my first and my second fuze don’t seem to sound the same as the clip. High notes sound better at equal volume on the clip than on the fuze. I admit that it might be because of the fuze’s audio jack which might loosen faster than the clips, thus affecting the sound.

10

 Now that you have mentioned it there does seem to be a distinct  difference in playback between  Rockbox  and the Clip+ OF.I tested it by creating a mix of a complex sound pink noise rich in harmonics and a simple sine wave using cool edit 2.0 I adjusted the relative levels of the pink noise and the sine wave so the sine wave is at a low level about 10% and the pink noise is at 60%

  When I saved the mix as a wav file and played it back on my sansa clip+ with the original firmware, the relative  levels of the two components sounded the same as on the pc sound card. With rockbox though the pink noise was sharply attenuated and the tone level was very high.

  I have had this problem with other portable media players , but I attributed it to the fact that these were low cost chinese made no name brand products. I think the reason  for this problem may lie in the free codecs that rockbox and the cheaper brand of chinese players may be using. The clip OF  uses licensed commercial quality codecs,  as does the typical  windows pc. The free codecs may have design deficiencies that may lead to degradation of playback quality.                                                                            

 That is just my opinion and I may be wrong about this, but  there is to my perception at least a huge gap in performance and reliability between free software products and the commercial stuff written by full time programmers who get paid for their work and who can devote all their time to the product development. In other words you get what you pay for.

11

@sandclip wrote:

Again this is subjective and not truly blind, but I don’t think there is “user bias” influencing the results because I really want to use RB instead of OF, and RB should yield output fidelity at least as good as OF by all indicators. 

No ABX = bias.

12

@contrapuntal wrote:

 

<snip>

 

No ABX = bias.

IMO it is more accurate to say:  “No ABX = potential for bias”.  Bias, in non-blind subjective comparison, is dependent upon evaluator.  Some evaluators are able to guard against bias better than others; my experience is that I fall into the former category of evaluator, for several reasons, but others have no way of verifying whether I am more, or less, biased.

In any case, that is the reason that additional user opinion and especially augmenting data are useful.  And we are gathering both, in this thread, supporting the observation that RB firmware produces lower-fidelity audio output compared to original SanDisk firmware in the Clip+.

I really wanted to use RB, but I will choose the best-sounding solution, if they differ in this respect.  If RB, great.  If OF, so be it.

Other observations, opinion, and data are most welcome, whether they support or contradict the premise (RB < OF fidelity).

13

@sandclip wrote:

IMO it is more accurate to say:  “No ABX = potential for bias.”

Even ABX cannot eliminate all biases.  If, for example, the subject believes very strongly that he is not going to hear any differences, he may not be trying his best to hear differences.

Bias, in non-blind subjective comparison, is dependent upon evaluator.

All subjective comparisons are biased.  That is why they are called subjective.  All evaluators doing subjective comparisons are biased.  That is why, when evaluating the sound quality of audio devices and technologies such as amplifiers, CD players, wires, lossy encoders, etc., ABX has proven itself to be vastly superior to biased evaluations.

Some evaluators are able to guard against bias better than others; my experience is that I fall into the former category of evaluator, for several reasons, but others have no way of verifying whether I am more, or less, biased.

Some evaluators might be able to guard against bias better than others.  No evaluator, however, is able to guard against bias completely.  You claim “for several reasons.”  Which reasons?

There is a way of verifying whether you are more or less biased.  And that is to observe how you do in an ABX test.  Many individuals have claimed to be able to hear a difference.  Many of these same individuals have, after an ABX test, shown that they were not able to hear a difference.

In any case, that is the reason that additional user opinion and especially augmenting data are useful.

User opinion can sometimes be helpful.  Very often, however, user opinion is worthless.  Let us look at one extreme example.  Let us say some users of extremely expensive loudspeaker wires–$30 000 US or more—all swear that using such wires improves sound quality.

 

For Rockbox or SanDisk, the augmenting data I would look at would be things like frequency response, signal-to-noise ratio, channel separation, total harmonic distortion, dynamic range, intermodulation distortion, etc; do these things show that the sound quality is worse?  Or do these things show that, while they could be better, there is no decrease in sound quality.

And we are gathering both, in this thread, supporting the observation that RB firmware produces lower-fidelity audio output compared to original SanDisk firmware in the Clip+.

So far I have yet to see any objective data presented in this thread. 

14

@contrapuntal wrote:

 

@sandclip wrote:

 

IMO it is more accurate to say:  “No ABX = potential for bias.”

 

Even ABX cannot eliminate all biases.  If, for example, the subject believes very strongly that he is not going to hear any differences, he may not be trying his best to hear differences.

 

/Here we go:  The relevant issue in this thread is whether RB has better audio fidelity/quality than OF, or not.  Now it is getting highjacked into a side contention about bias, etc.  In any case:  ABX itself is frequently questioned by evaluators as a source of inaccurate results.  For example, people who would like to believe that $10,000 speaker cables improve sound quality will claim that the ABX connections themselves “color” or otherwise affect the result, therefore the ABX result showing no difference with $10,000 cables is “invalid”./

 

Bias, in non-blind subjective comparison, is dependent upon evaluator.

 

All subjective comparisons are biased.  That is why they are called subjective.  All evaluators doing subjective comparisons are biased.  That is why, when evaluating the sound quality of audio devices and technologies such as amplifiers, CD players, wires, lossy encoders, etc., ABX has proven itself to be vastly superior to biased evaluations.

 

Some evaluators are able to guard against bias better than others; my experience is that I fall into the former category of evaluator, for several reasons, but others have no way of verifying whether I am more, or less, biased.

 

Some evaluators might be able to guard against bias better than others.  No evaluator, however, is able to guard against bias completely.  You claim “for several reasons.”  Which reasons?

 

/Getting off topic with the “which reasons”?  It is irrelevant to the subject./

 

There is a way of verifying whether you are more or less biased.  And that is to observe how you do in an ABX test.  Many individuals have claimed to be able to hear a difference.  Many of these same individuals have, after an ABX test, shown that they were not able to hear a difference.

 

/That would be a way to verify to OTHERS whether I am more or less biased.

 

If an individual had no way of discerning better-sounding audio output without doing an ABX test, the entire field of hi-fi would never have started.  There have been many many people over many years who select better-sounding results as part of their work in the field, without any ABX test whatsover.

 

Really, it mainly boils down to “honesty” and seeking the truth in reality experienced, as an inherent discipline (or not).  Many people have difficulty separating their wishes (or ulterior motives) from the truth, so much bias comes from this.  Sometimes it comes from wanting to feel better about their selection (e.g. $10,000 speaker cables, RB, etc.), sometimes it comes from dull senses, and sometimes it comes from wanting to fool other people (e.g. wanting to sell those $10,000 speaker cables).

 

In any case, that is the reason that additional user opinion and especially augmenting data are useful.

 

User opinion can sometimes be helpful.  Very often, however, user opinion is worthless.  Let us look at one extreme example.  Let us say some users of extremely expensive loudspeaker wires–$30 000 US or more—all swear that using such wires improves sound quality.

 

For Rockbox or SanDisk, the augmenting data I would look at would be things like frequency response, signal-to-noise ratio, channel separation, total harmonic distortion, dynamic range, intermodulation distortion, etc; do these things show that the sound quality is worse?  Or do these things show that, while they could be better, there is no decrease in sound quality.

 

And we are gathering both, in this thread, supporting the observation that RB firmware produces lower-fidelity audio output compared to original SanDisk firmware in the Clip+.

 

So far I have yet to see any objective data presented in this thread. 

 

/Depends on what you mean by “data”.  Do we need a test report from a certified accrediated third-party lab using NIST-traceable equipment, costing thousands of dollars?  Or can we consider information such as posted by mroberts200?  We have no way of verifying the veracity of information posted here; pretty much anything you see posted here could be fabricated.  But I consider it in combination with my own hearing, and other posted subjective opinion, and so far they mostly agree.

 

Whatever.  Hey, if someone prefers RB, that’s great.  As for me, I hear a difference - RB is inferior sound quality - so I will stick with OF./

15

@sandclip wrote:

 

Whatever.  Hey, if someone prefers RB, that’s great.  As for me, I hear a difference - RB is inferior sound quality - so I will stick with OF./

 

Well, you’re certainly in the minority with this opinion…but hey, to each his/her own. :smiley:

16

@sandclip wrote:

Here we go:  The relevant issue in this thread is whether RB has better audio fidelity/quality than OF, or not.

Yes.

Now it is getting highjacked into a side contention about bias, etc.

Do not worry about hijacks.  If one of the moderators believes this thread has been hijacked, he will lock this thread.

In any case:  ABX itself is frequently questioned by evaluators as a source of inaccurate results.

Examples?  Do you have any citations from credible sources that believe biased evaluations are superior to ABX evaluations?

For example, people who would like to believe that $10,000 speaker cables improve sound quality will claim that the ABX connections themselves “color” or otherwise affect the result, therefore the ABX result showing no difference with $10,000 cables is “invalid”./

A more plausible explanation is that no differences were heard, because there were no differences to be heard.  Anyway, this theory has already been tested by physically swapping wires.

Earlier I wrote: “There is a way of verifying whether you are more or less biased.  And that is to observe how you do in an ABX test.  Many individuals have claimed to be able to hear a difference.  Many of these same individuals have, after an ABX test, shown that they were not able to hear a difference.”

 /That would be a way to verify to OTHERS whether I am more or less biased.

Yes.  It is also a way to verify to yourself whether you are more or less biased.

If an individual had no way of discerning better-sounding audio output without doing an ABX test, the entire field of hi-fi would never have started.

Yes.  But we now do have access to ABX.  And using ABX, audio researchers have developed lossy formats such as MP3, ACC, WMA, and Vorbis.

Really, it mainly boils down to “honesty” and seeking the truth in reality experienced, as an inherent discipline (or not).

High-end audio often vilifies ABX.  And honesty is definitely not a word that is associated with high-end audio.  It does not boil down to honesty.  It boils down to properly evaluating sound quality.  From The Audio Critic: “We believe in measuring and we believe in listening but we don’t believe in measuring with our ears.”  If you are sincere in seeking the truth in reality experienced, you will be better off looking for objective data and trying to understand why ABX is superior to biased evaluations.

Many people have difficulty separating their wishes (or ulterior motives) from the truth, so much bias comes from this.

This is why the vast majority of high-end-audio “reviewers” hate ABX.

Sometimes it comes from wanting to feel better about their selection (e.g. $10,000 speaker cables, RB, etc.), sometimes it comes from dull senses, and sometimes it comes from wanting to fool other people (e.g. wanting to sell those $10,000 speaker cables).

Yes!

Earlier I wrote: “So far I have yet to see any objective data presented in this thread.”

 /Depends on what you mean by “data”.  Do we need a test report from a certified accrediated third-party lab using NIST-traceable equipment, costing thousands of dollars?

Please see message 8 in this thread for a hint at the possibilities.

Or can we consider information such as posted by mroberts200?

I prefer technical tests such as the ones I stated in my previous post.

We have no way of verifying the veracity of information posted here; pretty much anything you see posted here could be fabricated.

This is the point I was trying to make in my previous post.

But I consider it in combination with my own hearing, and other posted subjective opinion, and so far they mostly agree.

“We believe in measuring and we believe in listening but we don’t believe in measuring with our ears.”

Whatever.  Hey, if someone prefers RB, that’s great.  As for me, I hear a difference - RB is inferior sound quality - so I will stick with OF./

Earlier you wrote: “The OF sounds clearer; I can hear both lower and higher frequencies more clearly, less distorted sounding, and more detail within the music.”  It will be interesting to look at the total harmonic distortion using the different firmwares.  To me, SanDisk’s firmware sounds as good as Rockbox’s firmware—no signal processing activated.

17

 Here is a link to another discussion on this topic: http://www.hydrogenaudio.org/forums/index.php?showtopic=86306

18

@mroberts200 wrote:

 Here is a link to another discussion on this topic: http://www.hydrogenaudio.org/forums/index.php?showtopic=86306

I heard the subject extraneous noise when stopping/starting tracks etc. also, but I could overlook that if that was the extent of the audio output flaws.  Unfortunately, it is not the only significant flaw with the RB audio output, according to my subjective comparison with OF.

Another thought:  It is possible that people who cannot hear the difference between RB and OF audio quality are using headphones with lesser fidelity/clarity than the ones I’m using.  Don’t want to get the thread sidetracked into another side issue debate, but headphones are a key part of the audio reproduction system.  I tried several in-ear headphones, including well-regarded (advertised) names such as Shure, Ultimate Ears, and finally settled on the ATH-CK7 set as the closest-sounding to two separate decent home audio systems used for comparison.  They are relatively clear, with both high and low frequency response and overall accuracy significantly superior to many other headphones (including the Shure and UE I tried), and they reveal details in music that other headphones do not.

19

@sandclip wrote:

 

Unfortunately, it is not the only significant flaw with the RB audio output, according to my subjective comparison with OF.

 

Why not submit a bug report?

20

Hi, just Rockboxed my Sansa Clip+ yesterday, so I could play 24/96 flac files without having to downsample them. The difference in audio quality is quite huge. The OF has a smoother and rounder sound which I think sounds a lot better.