Your player plays digital audio files. It doesn’t care whether they be music or spoken word (audio books). So any playlist creation method you use will work equally well for either. There are many methods to create playlists; you’ll just have to experiment to find the one that you like best and/or find easiest. The most important thing about audio book files is to have them tagged correctly. You can’t trust them to have their ID3 tags set up in the way your player needs them to be for proper operation. MP3Tag is a great, free program makes makes it very easy to edit them.
Here’s a similartutorial for creating .m3u format playlists in Winamp. It is dead simple and I have created dozens over the years.