Maybe we're not getting the whole story here. Do you two have all of your guidelines set up? If you're fairly new to the lifestyle, I'm guessing you're probably still hashing through the details. This is one of them. I always say, it's a contact sport; expect it to be messy and occasionally painful.
My feelings on this: If you made it perfectly clear that you were NOT interested in playing with this couple, and you have an only-play-together policy, well MEOW!!!
I would be beyond pissed at him!! In my relationship with my husband, this would be a MAJOR faux pas. A simple misunderstanding is one thing, but if this is a matter of trying to play stupid (as in "OHHH, you meant BOTH of us weren't supposed to play!!" when he clearly should have understood this), then somebody needs to be set straight. Fast. Maybe I come off sounding a bit harsh, but this kind of blatant disregard for one's spouse has absolutely no place in the lifestyle. I'm wondering how this other woman's husband felt about the whole thing, or how she felt for that matter.