I cannot record the REC OUT sound from the DJ mixer. The SEND/RETURN ON/OFF button on the DJ mixer is lighting up.

Start DJM-REC, disconnect the USB cable and connect it again.
Or, Open the PLAY/EDIT, FILE or SETTING screen and then go back to the REC screen.
When the SEND/RETURN ON/OFF button on the DJ mixer is turned off, you can record the REC OUT sound.