You are doing something that is not condoned by society at large. You have probably been taught\acultured that sex is for the person that you love, or have strong emotional feelings for. What you have essentially done is enjoyed sex as a raw pleasure- which many of us are taught is wrong. Hell, many religions permit sex ONLY for procreation. You just updated your value system, and you have not been completely reconciled to the "new you."
I never felt guilty- but I did feel a little sleezy at first
Congratulations, you are on the path to much physical pleasure.