Ok, I am probably one of those women that will give their swing partners an inferiority complex. lmao
Seriously, what you are doing probably feels good...but I have only had an orgasm with 1 of my swing partners (and that is including the times we have gone to the club and only played with each other). We've been doing this for about 10 months and only had a handful of playmates.
However, the few that we have played with....we have come across what I would term 'very sensitive' women. Some would call them multiorgasmic, but I call them sensitive...they are the ones that you barely have to breathe on and they are hitting an orgasm. To me, that would give any partner they have a 'stud complex'...as is "WOW!! 3 strokes in and she's hitting the first one....and there's another! And another!"
Jeff told me of one girl he knew when he was back in high school like that (she was just kind of passed around as a toy for lack of better defining the situation). And all the guys knew it. On one hand, it's probably pretty neat to experience...OTOH, I would think that it would be an ego blow in there somewhere to know that pretty much any dick will do that to her....no matter the skill level (or lack there of).
Anyway, if your wife is one of those 'sensitive' ones and can rocket off like no one's business: 1. Lucky lady. lol

and
2. you need to realize that the vast majority of other women are not like that.
If your partner can sense your frustration (and we usually can), and you are basically distracted by your wife and the other man....maybe you should consider separate room play? Then you might feel less pressure to perform....you can take your time and relax a bit instead of possibly worrying about keeping up with your wife and her playpartner (even if it's an unconcious kind of thing).
Good luck,
Maria