I had two DE cycles- the first I did not have any caffeine at all and got a whopping BFN.
The second, I allowed myself to 'live a little' and had one single shot latte and an occasional cup of tea in the afternoon everyday, except during the 2ww. Got a BFP on that cycle and continued with my single shot latte throughout my pg. My OB's guidelines allowed for 2 1/2 shots of espresso a day (can't remember what it equates to in regular coffee).
I found when it came to caffeine, I was better with a little.