My guess is that you need help putting your irrational thoughts about your father behind you, Anon... from your husband and the other couple.
In my opinion, and as you wrote, no one else did anything wrong. It was all in your mind. It's up to you to get it out of there but you don't have to do it alone.
Start with, "Y'know, a funny thing happened in my mind last time we were together." Stress how good the evening was but that the thought of your father put a real chill on you. If your husband and the playcouple are as great as they seem to be, the four of you can see the humor in it and they (particularly the other husband) can do their best to make him decidedly "undadlike."
Good luck and please keep us informed.
Alura