Transport for London: Annual Gold Card
An Annual Gold Card discount does not give any discount to peak fares on TfL services when added to an Oyster card:
Get 1/3 discount on off-peak pay as you go fares and off-peak daily caps. Travel any time on weekends and public holidays and after 09:30, Monday to Friday
I think what you are finding is that if you travel from a station outside Zone 1 to a station in Zone 1 between 16:00 and 19:00, Monday to Friday it is classed as off-peak.
Transport for London: Tube and rail fares – Peak and off-peak times
