By default, the nudge value is set to follow the main time scale of your session, but you can disable this from the nudge value popup.
It is important to bear in mind that nudging audio will always move the selected audio without moving adjacent clips, regardless of edit mode. Next select the clip(s) with the grabber tool and use the plus and minus keys on the Numeric keypad to move the clip(s) later or earlier in the track by the selected nudge increment.