I think some lines WERE crossed.. In a way I can understand your husband thinking he had the okay to have sex but, THERE IS NO EXCUSE AT ALL FOR THE NO CONDOM!!What was his reason in that? I'm sorry your swinging experience went so badly. I do think it would be best for you to CLEARLY tell your hubby/b/f exactly how you feel. Swinging is not for everyone and personally I would NEVER trust him to swing with again. He showed a disregard for you and your health and he cheated. If you don't have a rock solid relationship swinging will hurt the both of you and the couples you play with. The other women cheated also how horrible of her to ask him not to tell you
I'm glad he did tell you.... maybe you will be able to work thru this and learn from it. I would say to CUT ALL ties with them before anything else could happen. "Best of luck."