I absolutely agree. Right now the people putting on the gigantic swing con Swingfest next month are using the following language as an intro to one of their press releases:
Open marriages, couples dating,
polyamory, recreational sex; it has many names, but it is most widely known as Swinging and it has become so popular it now has its own 4-day festival. (The complete press release is
here.)
I'm a member of the
Loving More board of directors, and we've respectfully and politely explained why most polyamorists would take exception to this statement and asked that this language not be used in the future. No one is saying that swinging isn't a valid choice. Instead it's that many polyfolk just don't identify as swingers, nor is their behavior like that of most swingers.
We all still have the same enemies and need to standby each other, but polyfolk are proud of their identity and do not wish to be assimilated and have it dilluted, especially without their consent.
TNT's explanation is on the right track. I also add that swinging tends to be a very couple-centered activity whereas polyamorists come from every kind of relationship form you can imagine, from single to married to in committed relationships with any number from two to X. Also, the polyamory community tends to be very inclusive and welcoming of gay and bisexual males, which is not the case in the swing community, though that is starting to improve, or so I am given to understand.
Lastly, and most importantly, a considerable majority of polyamorists establish their non-monogamous relationships with the intention of making a long-term commitment analogous to what two people do when they legally marry. As I understand it, many swingers make long-term bonds with their swing partners, but this tends to be emotionally limited to friendship and familial with no romantic component permitted.