It is great that you are concerned about your marriage. Swinging can reveal any weakness you have in your trust & communication. If you have a healthy & strong relationship, you have a good chance of actually growing closer in your relationship. I know that sounds weird but it is very common.
You should proceed slowly to be safe. You can always go crazier next weekend or the next party. But you can't undo actions which is why slower is usually safer.
You shouldn't worry about playing with couples. Other swinging couples are also happy with their relationship and they don't want you getting in the way of their marriage. Things can get stickier with singles or couples that give hall pass for solo play. That can open the door to an outsider becoming too clingy or intruding on your relationship. This isn't a big issue with experienced swingers. Swinging is more like a lending library and we all want to go home at the end of the party with our own favorite book. We may temporarily lend it out but we want our own book back at the end of the party.