That doesn’t seem right. I have some that I’ve removed 100+ times and they are still good as new. They are made of plastic, so you don’t want to stretch the mounts open further than needed when removing them from the arms. If any of your mounts are no longer able to stay securely attached, let me know and I’ll send you a replacement.
You can keep mounts attached to the front legs when folded. The mounts have to be removed from the rear legs since they face the body after the rear legs are rotated and closed.