Keeping them in - your time indication will be correct, your light will be ready to use and you do not risk breaking the knife whilst replacing the battery.