First off this happens al the time. Nervousness is the main cause, plus men are brought up to think it is wrong to get hard with another man in the room and then, on top of all that the condom thing makes it worse (they are needed but mr happy does not like them) so, practice helos, and encouragement, and getting to know others you swing with so there are not the first time jitters is also a huge help. Just know it is not uncommon at all. Best to you!!!
