I have to tell you that the poly forum I belong to is having somewhat of a similar discussion in one of their threads right now....the difference between poly and swinging and if/where they overlap.
Chicup, I can understand your definition of poly. Most people do not have such a restricted view as this. In fact, the poly world is much like the swinging world, in that you'll find just as many definitions for poly there as you do swinging here, but you will find that these for poly are broader than the one you have.
Webster's Online Dictionary also states a more lenient view of what the term polyamory means. The truth is, sex doesn't have to enter the equation in a poly relationship. (Not the kind for me.)
Some people, myself included, happen to belong to both the swinging and the poly camps. And I don't confuse myself at all.
Vol