File Limit increase in next FW please!

Hi!
I’ve been lurking around these forums for a few months now and finally got myself an 8 GB Fuze + 16 GB uSD card.
I really like the Fuze but now I have a problem after loading my music onto the device.
Of the 11428 songs I put on it (3038 internal + 8390 SD), it only shows 5294 of them, so about 50% of my music is missing! :frowning:

This is what the system settings info shows:
FW V02.02.26F
Memory 7548 MB
Free 143 MB
SD Card 15288 MB
Free 50 MB
Songs 5294
(Others 0)

I’ve read about a limit of 8000 files but the device does not even reach that number. (Why ?)

A request for the devolopers: Please increase the file limit to 16000 or higher (32768 maybe) in the next FW so that people with lots of music on their Fuze can actually play their music.
I don’t even want to think about the users who’ll be getting 32 GB SD cards for their Fuze.

Please help.
Blueyyyy

Did you upgrade the firmware after purchase?  If you did, and you use MTP mode, you will need to reformat the device for the newer 8000 limit.

You must use very low bitrate to get that many songs on there.

If you turn on the player without the card in it, do all the files show up? Are your files MP3, WMA, or something else? With so many files, I guess you are using 96 kbps or else have very short songs? Are your songs properly tagged, or do many songs show up as unknown? If you go to folder mode(music, folders) are all your songs visible? If everything seems to work okay without the card, then turn off the player, insert the card, let the player rebuild the database, and see how the player works now. you didn’t mention when you updated the firmware. Did you format the player(using the player’s menu) after that?

I use 256 kbps for music, while others use 192 kbps. Imo using under 192 kbps results in sound quality that is inadequate. Since space is an issue for you, you should use variable bitrate to save space(decoding it might decrease battery life though compared to using a constant bitrate).

Yes I did upgrade the firmware to the latest version right after purchase (before loading music onto the Fuze).
The device has always been in MSC mode and I want to keep it that way.
I’m using 96 kbps VBR OGG files which gives me good enough quality (I usually can’t hear any difference to the source files which are 192 - 320 kbps mp3).

Looking through the folders all files on the Internal memory seem to be visible (3038 files). On the SD card, only about 2200 files are visible, which corresponds to the number of songs the system settings info shows.

Since all the files on the Internal memory are already visible, taking out the SD card has no effect on the visible files there (only 3038 files).
After reinserting the SD card and the DB rebuild the same files are missing on the SD card.

The songs are properly tagged, none shows as unknown.

I have now reformatted the device (which only seems to format the Internal memory).
After the DB refresh there were now more songs from the SD card visible.
The system info shows 4809 songs (only about half of what’s on the card).

i have never used OGG, so I don’t know about OGG tags. Are the files on the card deeply nested? The player might not be able to see files that are too deeply nested(ie. folders in folders in folders, etc.) Can you see any pattern to the files on the card that the player sees, and those on the card that it doesn’t see? How do they differ? So if you put the player in folder mode it still only sees some of the files on the card? Are the files that the player doesn’t see individual files from here and there, or full folders full of files? Are there any unusual foreign charachters in the filenames or the tags? Have you tried renaming or retagging some of the files that didn’t show up to see if it makes a difference?

If all the files are similar though, including the ones that show up and the ones that don’t, then that points to the possibility of a nesting problem. There may also be a folder limit for the the player. I have not seen any information about what it might be though. Are you using a very large number of folders?

Starting from the MUSIC folder, the nesting goes down max 4 levels (mostly 3) e.g.
MUSIC\Category1\Category2\Album\CD1\
There are 177 folders on the Internal memory and 398 on the SD card.

On the SD card only folders starting with ‘T’ to ‘Z’ are visible (including all files).
If I format the Internal memory, I can see folders ‘K’ to ‘Z’ on the SD card.
I believe the Fuze sees those folders first, that got copied to the SD card first (Windows copied the folders in inverse alpha-numeric order), and when it builds the DB it stops when hitting the file/folder limit.
There is then a single folder where some files are visible and others are missing e.g.:
…\folder A\ (folder/files not visible)
…\folder H\ (folder/files not visible)
…\folder R\ (folder/files not visible)
…\folder S\ (folder visible, some files missing)
…\folder T\ (folder visible, all files visible)
…\folder Z\ (folder visible, all files visible)

There are some files with foreign characters in the tags but removing those files did not have any effect.
Removing a few folders from the SD card makes some of the invisble folders visible again. Copying the removed folders back leaves them invisible. So it really seems to depend on the copy order which files from the card get loaded into the Fuze’s DB.

The DB seems to be stored in the file MTABLE.SYS. There are 5294 song entries reflecting what I can see in the Fuze’s folder mode. The rest of the DB file is filled with 0x00 which makes up about 1/5 of the file so I would assume there should still be some space left to fit up to 1000-2000 more entries in there. The file is 1.77 MB in size. To fit all of my songs into it, a size of 5 MB or more would be required I think. Manually increasing the file size has no effect and brings up the “free 90 MB…” message (with >1 GB free internal space).

 I was having the same problem with my 16g card…folders earlier in the alphabet not showing up on the Fuze, but clearly visible when viewed through Windows Explorer on the computer. My total number of songs on the internal memory and card were below the 8000 limit, so I suspect there is some kind of folder limitation also. My solution was to use a smaller card for now and let my 16g card sit idle until this gets fixed in the next firmware update.

So you basically answered your own questions. Judging from what you wrote, it looks like there may be a 250 or 300 folder limit. Try simplifying your folder structure, reducing the number of folders, and the nesting depth.

P.S. now that I think about it, the player may allow many more than 250 or 300 folders if they aren’t deeply nested. The problem might be the limit for directory space for the folders, with each one taking up more space the more deeply nested it is.

Message Edited by JK98 on 09-13-2009 01:02 PM

Message Edited by JK98 on 09-14-2009 11:32 AM

@jk98 wrote:
P.S. now that I think about it, the player may allow many more than 250 or 300 folders if they aren’t nested.

 

I use nested folders, have 150 folders in the internal music folder, have 188 folders in the external music folder, and have no problems accessing all of my music.

“I use nested folders, have 150 folders in the internal music folder, have 188 folders in the external music folder, and have no problems accessing all of my music.”

 Perhaps your files aren’t deeply nested. Perhaps your filenames and folder names might be on the short side, so the path for each file might not take up so much space. Withouht knowing exactly how  this works, we can only guess about what is going on, although some experimentation might yield some useful results. I wonder if just shortening a number filenames or folder names might allow more files to appear?

Message Edited by JK98 on 09-14-2009 11:43 AM

“Perhaps your files aren’t deeply nested.”

 

Some of my files are nested like:

 

Internal Memory -> Music -> Classical -> Mozart -> Chamber Music -> [Three Folders]

 

and

 

External uSD Card -> Music -> Classical -> Various Composers -> Orchestral -> Concertos -> [Five Folders]

 

“Perhaps your filenames and folder names might be on the short side, so the path for each file might not take up so much space.”

My folder names are all short—no more than about twenty-six characters.  I am using the MTP mode, and in the MTP mode, long folder names are problematic.  Some of my file names, however, are long—e.g., 2 Songs for Medium Voice & 8 Instruments, 1 and String Quartet, op. 127, 2, Adagio, ma non troppo e molto cantabile.

This does seem odd. Perhaps his problem might after al be related to specific issues using OGG and not to having too many folders, or having file names or folder names taking up too much space?

A question for the OP. Have you tried deleting MTABLE.SYS? Doing this will force the player to rebuild its directory from scratch. You did say you formatted the player after you did the last firmware update?

After deleting the MTABLE.SYS I get the same DB from the rebuild.

By moving all files into the MUSIC\ folder and deleting all nested folders the DB takes about 800 files more (~6000 total). Finding a song in that mess is nearly impossible but it was a nice experiment to confirm what’s been said.

This means that the number of files the Fuze can put into the DB depends on the length of the file path names. You can fit only so many filenames into a limited number of bytes (MTABLE.SYS is always of size 1862468 bytes). Why does the size have to be fixed?? Why can’t you (the FW) simply keep writing to the file until all songs are inside the DB (or you run out of disk space)?

Now there’s hoping that the devs are aware of this problem and will increase the limit to accomodate all those files in the DB, or make the DB grow as necessary.

""Now there’s hoping that the devs are aware of this problem and will increase the limit to accomodate all those files in the DB, or make the DB grow as necessary. "

It isn’t that simple though, as this is a low cost, low power consumption device, so physical ram is very limited. It just shows that excessive nesting should not be used, and filenames should be  short. I guess one could use a bit of nesting, and use very short folder names if the plan to cram 8,000 files on the player.

Message Edited by JK98 on 09-15-2009 01:46 AM

Hi all, interesting discussion with a lot of good input! Haven’t had a chance to step in earlier because I just returned from vacation.

Nevertheless I’d like to add some of my findings:

  • folder nesting works at least up to 10 levels (see here)

  • my fuze handles up to about 9200 audio files (see here)

My setup in these tests was to use mp3 and wav files with really short names and little or no tags (<20 characters) on internal storage only. I’m running MSC USB connection mode all the time.

Moreover I’d like to mention that there are obviously more hard-coded limits of all sorts in the database file besides the allocation space used for folder/file names. There seems to be little or no error checking if you run into these limits as it has been noticed by different users (see for example here).

I understand that most users wont have problems at all because they never have that many audio files on the player. If there would be a next firmware update increasing the file limit without lowering the database refresh time I believe many more customers would complain like “Why does it take so long to add/delete a single file to my player?” or similar things which are not so easy to understand and accept by the average user. On the other hand I would greatly appreciate to see SanDisk developers to tell us where the current database limitations are. This might allow us to find workarounds for special user requirements.

Back to the problem of the OP. I’d recommend to

  • Always stay at less than 8000 (maybe 9200) files! Otherwise all sorts of problems might occure.

  • If unsure about tagging issues copy files in several chunks, not all files at once.

  • Avoid long tag field strings!

  • Use short folder/file names whenever possible.

A little update:

After a long and painful process of moving files on and off the fuze and retagging all files, I managed to get a maximum of 9999 ogg files onto my Fuze. 1 song more and the firmware will badly screw up the list of displayed files (folder browsing). The entries in the MMTABLE.SYS look correct though, so it seems to be a firmware bug.

Even with a 32gb card I wouldn’t reach that many files. I’m using OGG on quality 5, 16gb of space filled up with 3114 songs. I’d still be shy of 8000 at that rate. Everyone’s different, though, I guess they prioritised price over giving a little more leeway.