Caught Red-Handed in a Betrayal Scandal

So I got dumped by my fiancé this week. Not because he found out about me and another dude in the office, though. That's actually fine. No one had to know that we were sneaking around for two months except us. We kept it light-hearted and playful, more like a harmless dalliance than anything serious or meaningful. But here’s where I screwed up: My fiancé was also my best friend since college, and when I told him about the affair, he didn’t react how I expected. He was cool with me being happy and exploring things… until we got caught in an awkward situation at a work event. The other guy's wife noticed us holding hands behind a curtain, giggling like schoolgirls. I tried to cover it up by saying we were just friends talking, but the damage was already done. My fiancé felt betrayed and humiliated; he’s still processing everything now. And I… well, I have no idea how to fix this mess. So yeah, am I really that big of an asshole for playing around behind my fiancé's back, or is it more about getting caught? What do you think?