Been Keeping A Secret From My Partner

Hi everyone, I've been struggling with this for months now and I really need some advice on what to do. My partner, let's call them Alex, is wonderful in every way - kind, supportive, caring... you name it. We’ve been together for about four years now and have a good relationship overall. However, there’s something that I've been keeping from Alex that’s eating away at me inside. A few months back, while on vacation without Alex, I found myself drawn to someone else in a way I never expected. It started off as just innocent flirtation over drinks, but it quickly escalated beyond that. I ended up meeting this person (let's call them Jamie) again and we spent some time together during the night, away from the prying eyes of others. It was an incredibly intimate experience with Jamie, filled with a level of passion and connection I hadn’t felt in quite a while. We both agreed to keep it secret though, understanding how much Alex means to me. Since then, we’ve kept in touch sporadically but nothing has happened again. It’s just hanging over my head like an anchor, making me feel terrible. Alex doesn’t deserve this and I’m worried about what it would do to them if they found out. Yet the guilt of lying by omission is crushing. So here’s my question: am I being unfair or unreasonable by hiding this from Alex? Should I come clean even though it risks hurting our relationship more than leaving things as they are? What would you all do in a situation like this?