Because if you reset the day ... it never happened. Look, you put a clause into this scenario that meant whatever you did, you could completely take it back. Undo it. Wipe it from existence.
Lets put this in legal terms. Imagine you murder someone and reset the day. Would you be convicted? There's no body, no victim, no crime is deemed to have taken place. you wouldn't be imprisoned. You'd be found NOT GUILTY. The same is true of these sexual encounters. You aren't guilty of them, because they haven't happened. They only now exist in your head. I'm sure there are plenty of married men who have sex with all kinds of women in their heads ... there's nothing wrong with that. Its called a fantasy.