Some time is required to load tracks from a USB memory device or USB hard disk.
It takes time when the folder hierarchy is deep or when folders contain many files. It also takes time when there are files other than music files, so it is a good idea to keep non-music files in different folders.
