Can I immediately load and play back the performance I've recorded on a USB device?

Yes. You can load recorded sound from the [PIONEER REC] folder. The recorded file won't be analyzed by rekordbox though, so the enlarged waveform and sync features won't be available.
*Files can't be loaded while they're still being recorded.