I understand that you brought the topic up and you suggested your friend as the guy and you started inviting him over. On the night it happened she took control and made it happen. It leads to my other questions and analysis. Personally, if my wife was ignoring me during something like this and told me that *I* made her uncomfortable with my presence I would be worried about what was going on. My wife and I are focused on making sure that we are both comfortable and we go out of our way to ensure that. We also trust in each other that if anything is happening that we're uncomfortable with we will speak up in the moment. It will either get resolved right there (if its not a big deal) or we'll stop what is going on. No questions asked. If she wasn't totally bought into that I don't think I'd agree to any kind of swinging.
It is great that you are talking openly with your friend about this, though you obviously didn't talk to him before the first encounter. When I say that you run the risk of permanently damaging the friendship I don't just mean that someone might get regrets the next day. I am talking about an enormous amount of issues that may come up; emotional attachment, his wife finding out, someone changing their mind, situations changing (you start traveling, they start wanting to see each other alone all the time), he gets divorced.....this could go on forever here. My point here is that with someone who is not connected to you right now, if something was to come up due to the sexual relationship then cutting ties with them is not as important to your life. If things turn sour with a guy who is like a brother to you, well that is a major impact on your life. Are you willing to risk that? Being willing to risk it is the only way that doing this is worthwhile. If you think there is no chance of that happening you are fooling yourself.
My wife and I swung with a couple that were good friends with us. We talked openly about everything, we were all up front about what was going on, how we felt etc. All before we even did anything at all. After playing a few times things got very tense when the other girl decided she didn't want to swing at all anymore and her fiance kept trying to push her to keep playing. things crumbled very fast and for a long while we thought we had lost two good friends. Even now, long afterward, things are strained to a degree when we're together with them. There is no longer a comfortable fun vibe like we had prior to even talking about this.
It is a real risk, especially when it seems like that won't happen.
I know I didn't really tackle the question you are asking, but I think there are other important things here for you to consider and think about. The two primary ones are: should you be swinging with your best friend and what are your wife's true intentions here?
Whether you think you need to think about those or not, I hope it all works out for you!
I look forward to hearing updates on your situation and hope to see some great positive stories about how it all turns out.