Ahhhhhh,
Taking one for the team!
Most if not all of us have done this at least once in the past. For most, like us, when first starting out.
The fix...Don't do it. There has got to be many more men out there your wife finds attractive.
The biggest problem we have found with swinging with couples is trying to find one where all 4 click and are attracted to each other. It can be hard to do and tends to suck a lot of the fun out of it. At least it did for us.
Sorry, thats probably not what you want to hear but it is how I see it.