I thank everyone for their input, it's helped me see where I'm going wrong (and right) with this. We talked about it this morning and Nav thinks maybe we're trying to go about this too quickly for comfort. Maybe this lifestyle is for us and maybe it's not, that's why I'm here talking to you nice people, to get more information to make the decision to actually start swinging.
To be completely honest, I do have issues with trusting the outcome (I forget who mentioned it, sorry, not up with the names yet...). There's a small part of me (with a very loud voice, actually) that thinks maybe he could fall for someone else. I read some posts about not falling in love, but I think we're thinking of a much too complex situation to completely avoid that (the opened marriage thing). We're both fairly young (me 21, nav 23). We've been married since I was 17 and we have two children. I think between the kids and both of us going to med school, I'm not sure how it would work out in the ideal way. Both of us have 3 years of school and four year residencies ahead of us (he has an extra three for surgery fellowship, I have an extra 7 for Neonatology fellowship). I'm still terrified about the way our friends view us and what if we got 'found out'? Plus, with the minimal time we'll have with all of the school we're looking at, how do we divide that between 'us' time and 'swinging' time?
As you can tell, we're still working out the kinks and talking, which I think, is a reason I was so thrown off about the trance he was in while talking to this girl. I didn't really think it'd go so fast from talking about it to doing it. And here I thought I was the one 100% comfortable with it and he was iffy. Turns out the tables have turned.

We have more talking to do about this and I have more reading and learning here.
Thank you all!