Audio files on SD card are segregated from the main device. Why?

“Perhaps they did have to rewrite the code to suit the new processor, but it’s not as though they didn’t already know how to write code that would allow the databases from the SD card and the device to be combined.”

It is not a matter of that.  To be reasonably responsive, the full song database needs to be in ram. The Clip sport and Clip Jam probably have much less ram than the Clip+ and Clip Zip. i don’t know if these players have the maximum ram the processor can support, or if Sandisk decided to skimp on the ram. Perhaps the ram was kept small not just for cost savings, but for power savings as well? 

Have you ever done any programming of computers?  If so, you would have an understanding of the constraints.  Add to those the need to keep power cosumption low, as they want long battery life from a small battery. There are a number of tradeoffs that need to be considered in the design. You and I might not agree with all the decisions they made in dealing with the tradeoffs. If it were up to me, I would have used a very low power monochrome display like a digital watch uses. These displays can operate for a few years on all the time using the power of a tiny button battery. I would want the display on all the time the player is on, which would be a small fraction of the time that a digital watch has its display on. I would not have skimped on the database capability though. I find it amazing how much capability Rockbox on the Clip+ and Clip Zip offers. It seems obvious though that Sandisk is much better at designing mp3 hardware than software. I would love it if Sandisk outsourced its player firmware design to Rockbox, and Rockbox came as the default firmware with the player. Of course this would likely mean giving up all capability to play any protected files(including Audible).

“our statement that the processor on the Sport is much less powerful than on the predecessor models is interesting, considering that the Sport’s startup and refresh times are significantly faster than on the previous models.”

The size of the database is its length times its width. With a much smaller database, the database refresh requires so much less processing. Getting rid of support for protected WMA files, and getting rid of MTP mode also greatly simplified the firmware.