Situations like this are probably one of the bigger down sides to swinging. You have what would normally be a totally harmless flirtation with your wife's friend. If you weren't a swinger, it would most likely never go any further than that, regardless of whether the husband is a jerk or not.
But because ya'll are swingers and have--with others in the lifestyle been able to go beyond flirting, you're now entertaining these thoughts. Not good. Most folks on here will advise that swinging with vanilla friends is a major no-no.
The reason for that is that it's not worth the potential drama and loss of friendship.
The BIGGER problem you have here is that the husband doesn't know about the flirting, and would not be on board with your intentions. Anytime you do something without the
knowledge and
consent of everyone involved, it is
CHEATING. I do not, nor do many of the other folks on this board, condone this behavior.
My advice is to let this go NO further. Best of luck to ya'll.