Yes...I have had that problem. I used to smoke a good amount every day for a few years in a row, doing around 10 grams a week or so. After a few too many one-off binges with amphetamines last winter, though, I got really anxious with my drug usage, of any kind, and I got very paranoid when smoking weed for a few months. Before this break, even, I had just been smoking to smoke. The main issue was never giving myself a break from smoking - the anxiety I got from having done too much amphetamines added to this and I just decided to retreat from the drug world for a while (alcohol's included in this, so I didn't drink either).
So during my extended break, which lasted 5 and a half months altogether, I did try to smoke by myself in my apartment two times, separated by a few months. Both times I did get high, but, hey, I was just by myself, and I just got anxious about using drugs again, and both times I just smoked a gram or so (over the course of a few days) and flushed the rest of what I had down the toilet. Back to sobriety again...
Flash forward to last May, and I was able to hang out with my old stoner buddies from high school - about 5 of us got together to hang and go see a movie and after we had all gotten together to carpool over, one of my friends, who was driving, drove over to a nearby suburb to smoke and cruise for a few minutes before heading over to the theater. On a whim, I decided that I'd give smoking weed one last shot with my old friends - I took three or four hits and I was literally stoned as hell for about 5 hours, well after the movie ended. After getting high this time, I had the giggles like a motherfucker...everything was hilarious and I just had a lot of revelations about how weed was absolutely fine, you just had to do it with the right people at the right time.
Now I can smoke by myself at anytime and just go back to that moment I smoked with my friends again before going to see a movie - it's pretty great.
However, this whole experience is definitely idiosyncratic to my life.
That being said, my advice would be to have planned (even though mine wasn't - I actually had basically planned to not smoke weed again, but I knew it wasn't bad for me in a health sense, so I gave it one last shot when the opportunity came up) a set and setting that is very nostalgic to you, and that can easily make you feel much more at peace and content with the high that weed provides - these things allowed for an extremely smooth come up in my case. My old friends seemed to be the difference-maker in my case, though, if I had to pinpoint a difference between this time smoking weed and the two other times over the previous months.
I have met plenty of people who, for some reason or other, just cannot enjoy weed, definitively. Taking a long break and reconnecting with this psychoactive plant in a familiar and comfortable set/setting sure helped me out, but that's just my case. To be honest, sometimes I wish I hadn't started smoking again, because I've wasted some money on it which I could've put to more pragmatic uses. Not much, but enough to not be proud of. There's always a pro and a con to everything...
And also, since the OP talks about drinking, I should mention that one of the primary reasons for me to give smoking weed a few chances again after taking a few months off from it, and doing perfectly fine being sober, is that I really don't like alcohol for the most part, so drinking really wasn't a viable option for me. Other than weed, I really don't have a vice that I can go to...