Implants should be removed irrespective of whether they are causing any problems. Reasons- keeping implant in body has no extra advantage. Rather if one sustains another injury the magnitude of damage is much more with implant inside than without it. And implants are stress risers. Once your ankle implants are out by 2-3 weeks of rest is fine enough.

Usualy implants are tried to retain as long they donot pose any risk or issues... But you can certainly disscuss your concerns with surgeon who can address you well.