Don’t use the Enhanced mode when you download your book from Audible. Use Mode 4. No more delay. I also found that if I use Ehanced mode, it chops the last few minutes of my book off every time.
Why a new player like Sport can’t use Enhanced mode is beyond me. A big step backward from the Clips.