Older Man's Regret

I’m engaged and things with my fiancée are great overall. But last week, we went on a trip for our anniversary to this charming town by the coast. We were staying at a quaint inn and one night, after she had gone to sleep early due to jet lag, I ended up chatting at length with the owner over drinks about life and his travels. It wasn’t until he mentioned how much women seemed to appreciate an older man’s perspective that I realized we might be flirting, or something like it. We kissed briefly before parting ways, but it was clear neither of us intended for anything more than a momentary escape from our usual lives. Yet, now back home, the guilt eats at me as I question if this is normal behavior even in an affair-like situation that led nowhere physically? What are your thoughts on these fleeting encounters when they don’t affect anyone truly?