Not alone with him, who knows who this guy is...maybe it has been a single male all along or a married male looking to cheat on his wife. Maybe the whole lets get together just us two girls has just been a primer for him.
Either way if you play with your partner present, you meet with your partner present. If he has an issue with that then you are better off away from the situation.
Play safe