I have social and generalized anxiety and/or autism spectrum disorder, and ADD which progressed into ADHD. For me most drugs tend to be beneficial, everything stimulating (stims obviously, but also empathogens, dissociatives, even opioids as long as no tolerance involved) tend to ease my symptoms. At last, they're euphoriants which raise dopamine levels and dopamine is what is lacking in ADHD. Downers on the other hand, specially benzos, were never recreational for me, they leave me depressed. Alcohol is but only during the short lived stimulatory window which opens up upon fast ingestion of ethanol.
Caffeine is the possibly worst example out of all more widely used drugs, it only works when I'm seriously tired and even then comes with an uneasy edge. Other stims are good unless overdosed, which is a small margin with most of them, even 18mg XR methylphenidate can be too much until tolerance begins to raise.
I think that psychosis, or a history thereof, is what really interferes with getting high. I caught a cup full of psychosis from chronic sleep deprivation and too many dissociatives done together and it wasn't pleasant at all. Imagine your brain misinterpreting the noise of driving-by cars or flowing water to voices shouting and whispering at you.