I’m not sure if you’re referring to the oyster app, or the oyster website… but on the website, when I went through this, I had to attempt to add another top up and on the last page of that the option presented itself to remove any unwanted cards (as a link under each card).
NB you can’t delete a card that’s associated with auto top up, so if you do have such a card you’ll need to go to auto top up settings and change the associated card first.