Don't worry about it. At J.'s and my age, we're still going out every weekend...sometimes with other swingers, but a lot of times it's with a group of vanilla friends. He is flirty-naught with his friends, and I am flirty-naughty with my friends, and both groups of them think that's just "how we are". We are also very sure to "be together" when with them. What I mean by that is, it's very obvious to anyone who cares to look, that there is no jealousy, no anger, and lots and lots of love. I don't think either of us cheating has crossed anyone's mind.
But it wouldn't hurt to be sure you have LOTS of fun at the club