Tramadol=opiate+SSRI
SSRI's cause problems for nearly everyone. They are used as medication for premature ejaculation, lol.
Opiates can also cause problems like that, but not as severe and in less people especially with an opiate that is as weak as tramadol, but it is still possible.
So year, it is very likely that tramadol is causing your problems. It's totally normal.
If you want an optimal sex drive then SSRI's and opiates are not for you. You got to be willing to sacrifice your sex drive for their effects.