First of all,
to the Swingersboard! You've found the right place for discussion, information and support.
Sounds like poly to me. I don't think your post mentioned the word "love", but your relationship definitely goes beyond swinging. Sounds like it works for everyone.
The dangers: you sound smart enough to know what the dangers are. They are there, but personally I wouldn't let that stop something that seemed like a good thing to me. (Of course, I've been known to run off a cliff or two and end up unhappy after impact.) I'd say don't make big monetary investments together. And make sure the other wife is really okay with the arrangement, or at least isn't gathering evidence in a campaign to take away everything P has.
I suggest you take a look around the forums, especially the Poly forum. There's a lot of accumulated experience there.