Not Sharing My Kink Is Fine?

So I'm nonbinary, mid-twenties, and have this thing for spanking as part of our sex. Nothing crazy, just the idea that a partner can be strict or demanding with me during play is hot. But my current boo doesn't like it at all. We've been together four years now, and they're great in bed, but no go on that one kink. I got into this stuff maybe five years ago, right around when I came out about my gender. Figured out some control issues from childhood were still there, but the spanking part of it helped me deal with them somehow. It's not about pain or violence for me, just a way to let go and feel like someone else is in charge, even if only briefly. We've had talks about how I'm okay doing it solo, and that I'm careful about boundaries, it's always consensual. They say they trust me but can't get past the idea themselves. Tried a few times to be understanding or find something similar, but I kind of miss having someone take charge even if it’s just pretend. Recently thought about looking for casual encounters where this is an option, and honestly... not sure if that's fair to my partner. But when they told me last night we should focus more on compatibility and maybe be friends someday instead, I got to thinking. Would leaving over one kink make me the asshole? Or are there other ways to have both my boo and my kink?