Ok. Here it goes:
YOU'RE NOT GAY
Now, I gess you may go ahead.
Seriously, if you're so concerned about the "being gay" stuff, you may not even have something hard for the other guy to blow.
What's interesting here is that you're not facing any dilemma with your wyfe "being gay", nor she does, and I am pointing this out because this is not a matter of "being fair", like "if wyfe want to do an elephant, then I shald do an elephant myself", got what I mean?
Everyone have his/her own limits, and these limits have to be accepted. Here we often say "the velocity should be the one from slowest one", and in this case, "the limit should be drawn over the most restrictives ones both of you have". If you're not confortable with a guy, then you should leave that fantasy in fantasyland.
But I'd consider your "B U T" here. I wouldn't be with a guy even when I have some "gay" fantasies, and I know my motivations (among others, I tried to and couldn't take it), so I gess i am "more gay" than what you are even when there is no "B U T" for me, at all, so your "B U T" may be hidden your wish to try, without feeling gay because of this.
It's like a common fantasy shared by a lot of women about being raped, to be forced to engage in activities BUT because someone's else is the one who want her to do it, providing a context where to fantasize "for free" about these activities without having to concern about feeling herself a whore.
If you're up to "reluctanctly accept" to be with a guy because "youre forced to do it" (to make it fair or whatever reason) by your wyfe, you'll have an excuse to avoid your concerns about "being gay". Perhaps you need such an excuse to be able to enjoy being with another guy.
The cuestion, then, would be... what does mean for you "being gay"? What's wrong with that? Which knots in your identity tapistry "being gay" would be untiing? and more important... how easily these knots could be untied?