Your numbers aren't terrible. Even on TRT, I try to keep my E2 below 70 because that seems to be my optimal. Last week I forgot to take my AI and E2 went to 110, I felt like shit. Your mileage may vary.
You have to go by symptoms and not just numbers because each man has different androgen receptor sensitivity. Some men need less T to activate the receptors than others. My normal and your normal can look different, as long as we are both within the T reference range.
I would say, keep an eye on your T. Because you are over 35, get it re-tested once a year. Make sure you get it tested in the morning before 9am for the most accuracy. Try to get it tested around the same time every time to establish the baseline. If, between now and, say, age 40, the numbers keep dropping, then that could justify TRT. But you have to have the symptoms as well.
Based on symptoms it doesn't sound like you need TRT. However, to be fair, most GPs won't Rx TRT even if you do need it.
