I'm a newbie, too, and it sounds as if my wife and I have basically the same level of experience as you--one multi-male outing, but no swap with a couple. So, while I'm hardly an expert, I can definitely relate to what you and your husband are going through as you consider swapping partners.
This is one of the major things my wife and I have discussed as we consider the idea, and we haven't really come to an agreement--more discussion is needed.
I can't speak for my wife, but my sense is she feels very similar to you. She's absolutely adorable, in great shape for her age and place in life (mom with kids, hard-working with little time for working out, etc.), and she gets plenty of attention from men, my friends included. Yet, one of her biggest concerns going into this is that she "isn't attractive enough"--she wants to lose weight, she's worried about any sagging caused by motherhood and age, etc. She's slowly coming to grips with it, and I do all I can to support her and remind her of how beautiful she is and that if I had even a 1% doubt of my love or loyalty to her or our relationship, I wouldn't risk swinging out of fear that we'd end up apart.
If that's true, then why should an extremely attractive female be a threat, right? But I suspect it is, even though she hasn't come out and said it in so many words. As I said, we need to discuss this more, and doing so openly involves her coming to grips with the insecurity over her own body image. It's a major reason why we're not rushing.
At the same time, I can honestly say that in any potential couple, I hope the male half is very good-looking. Call us shallow, but we'll admit that physical attraction is a major component for us in considering swinging partners, and my attitude is that if you're going to do it, DO IT! Knowing that a knockout of a guy found her desirable would only add to my excitement. That's not to say that we wouldn't consider less-attractive couples--I'd MUCH rather an average-looking couple who's down to earth than a beautiful one with their noses up in the air. But if I'm going to watch her with another guy, I'd just as soon know that his looks are a major turn-on for her, and that she's acting out a fantasy on that level, as well as a physical one. And being with an equally attractive woman would be just as exciting to me, but it would be spoiled if I knew my wife felt overly threatened by her.
As far as your husband's change of heart--having been jealous in the past, but now assuring you he won't be if you're with someone else with him present--perhaps he's like me. Before my wife ever slept with anyone else, I got uptight if another guy even looked at her. But once I'd observed her letting loose in all her sexual glory and watched another guy please her while knowing she was going home with me, etc., the jealousy diminished greatly, becoming just another part of the overall excitement I get from such experiences--letting go a little bit, feeling the tug, and knowing I have nothing to worry about, being free to enjoy it for what it's worth.
Or, perhaps he just grew up a bit--as I also did--and became more confident in your love for one another. Or both.
Just some thoughts based on my and my wife's experiences. Good luck in your journey!