Workplace Confession

I work at a busy retail store and I've been seeing someone for about six months now. He's really supportive of me even though my job is stressful and unpredictable. But there’s something that makes me feel ashamed and isolated, like it doesn't fit with how we talk about our relationship in public. It started small, just little touches when no one else was around, like when we'd meet for lunch on his day off. We ended up doing more behind closed doors at the store where I work because it's quiet back there. He says he feels guilty too but doesn't know how to stop. Does anyone out there ever have something like this in their relationship? Should I just wait it out and see if it fades or talk to him about it?