I did finally get to speak to someone at TfL who were remarkably unhelpful and told me that as it’s a pending transaction that I have to contact the bank.
Which I did and I had to raise a dispute over the £5.25 pending transaction.
The bank have said it will take a week to review the dispute and up to a further 12 weeks to resolve so if they agree with me then I’ll get the pending transaction reversed by 2nd February 2024!
It’s obviously far too much hassle so as I said in my initial post I’ll be using Oyster from now on as clearly contactless has a few issues still 🙁