We don't think others at clubs will mind if you don't want to be touched. We have been at times when we were not in a position to play, but we like the atmosphere. We feel its important to know how to express to others diligently that your not playing.
If its the groping your are worried about, just bear in mind that it even happens with a vanilla bar as well. Rarely for us has it happened at a swingers club.
Personally, I (Mr.fun4ds) Like a good gropeing, what can I say

, cop a feel anytime ladies
Mrs.fun on the other hand, one should be very cautious. It would be wise.