There are plenty of mature swingers, and it's even pretty common for them to prefer folks their own age. Don't think you're alone.
If you hang out at boat bars, ballroom dances, and other gathering spots for the older but active crowd, you might be surprised at what all goes on, and how much "hooking up" there is.
P.S. Don't think we're in the "prefer" group above, either - B likes the young guys.