I think coming from two such diametrically opposed sides will make this a bit harder for the two of you. But if this is something that you both really want, you'll find a compromise. It might take you longer to get there, but it will be worth it in the end. In my opinion, it's going to take a lot more talking before anything happens. Or should happen. Your relationship is primary here. The swinging is secondary. The two of you need to find middle ground where both are comfortable.
I'm going to ask the experienced couples here to chime in with advice on how to find that middle ground. I'm not sure how to do it other than talking about it. Maybe someone out there has been thru this problem.
Good luck. Keep us posted.
DragonsLair
He is T. I am A.