So, I have used basically all the drugs that can be used and have been a part of nearly every "subculture" at one point or another. I'll rank them from least annoying to most:
1. MDMA users---not the rave scene, but people who use MDMA in their homes or in a park with close friends. This was probably the least overall annoying group to be a part of, obviously partly because MDMA makes you empathize with others but mostly because so little bad came out of it. It never turned into fiending "we have to roll everyday" mindset, it never took on MDMA is the only drug for me, and it mostly just brought some already good friends closer. Having said that, I have little to no interest in doing that drug ever again as it is just too mentally/physically draining and the allure of it wore off.
2. Stoners--- yes, they can be annoying. But really, they are naive and fairly innocent, for the most part. They don't know any better, and hey, if they feel a little cool for being "rebellious" in smoking the ganj let them feel it. And if they start to get preachy or whatever, eh who cares, having been an avid pot smoker for years I know how that shit messes with your mind so I give them some space. The only thing that really is annoying about them is the overanalyzation that results in social awkwardness often (and I've been there but it's really annoying to be around stoned or not stoned).
3. Psychedelic Users---ah, these are a fun bunch. I definitely had a period where I thought hallucinogens had most of the answers to the universe, and I wasn't afraid to talk about it! And that's the problem! Tripping certainly alters your perspective and makes you more aware/appreciative of many aspects of life that before seemed mundane or stale, but damn, it really doesn't change you all that much for the better. The snobbery of psych users is easily what makes them so annoying.
4. Opiate/Benzo users--the downers. This is the group I always felt "the best" in because I just personally prefer downer drugs. Of course, these fuckers (opiate users in particular) can't be fully trusted when they are on a full habit. But when everyone has their junk in them, the mood is chill and conversation can flow easily. It's pretty nice actually. However, these days, I'd rather just use alone because that's what happens when you mess with smack.
5. Alcoholics- ugh, alcohol. What a disgusting drug. Sure, I don't mind a beer or two here and there, but man this is a nasty drug that somehow is widely accepted. It brings out some of the worst in people at all times and generally always leaves the people a complete mess physically, emotionally, and/or mentally. I just really don't like anything about the alcohol culture.
6. Last is the speedy folk. I have never seen meth/known any users, so I will leave them out of here. But I have indulged in coke, crack, and amphetamines on various occasions. Coke can sometimes be fun, but overall, this category produces the most frustrating subculture to be around. People become paranoid, ego-filled, emotionally wrecked, angry, scared, word-vomiting, fiending creeps. I say this acknowledging the fact that I STOLE what seemed like an 8 ball of coke and proceeded to use it for the next 48 hours non-stop, even went in to work that day with coke and heroin on me and kept snorting in the bathroom. Fuck speed.