Well, I'd say weed has had the most negative effects on me. It's also had the most positive effects on me. It's also the only illegal drug I've done more than 10 times. It's also the only drug I've done in the last 8 months.
Use for use, it's a softer drug on the mind than just about anything else, save alcohol. In general, my head feels like it's jammed full of cotton for about 2 days after I smoke, I feel less like socializing, and all of my issues seem like they're on display to people. I think it cost me a potential girlfriend, but at the same time looking back almost a year out I don't think the relationship would have worked because I had a lot of issues. Pot helped me solve those issues; under its haze a ton of painful stuff was allowed to come to the surface and release. I don't think I could have done it sober, I really don't. It hasn't cost me at all professionally, I've always been an aggressive type-A person and the stoned me is about as laid back as a normal sober person. I guess I've noticed a little decreased lung capacity, but not much as I run 5 times a week. I also have some skin problems with heavy use, but I haven't been doing that recently. Oh, and I get some pixellation in the dark when I'm tired but I don't know if that's pot's fault. Smoking certainly aggravates it though.
I guess the biggest problem I have right now is that I had a lot of issues and didn't like socializing, I moved to a new city and spent time smoking pot and working on my problems rather than meeting people, now I'm relatively sane and much happier but my social calendar is slim. And once you're out of college it's harder to fill. An easy solution is just to smoke more pot.
At this point I'm scaling back and plan on quitting, but more because it's time to move on than because I feel permanently damaged.
FWIW, the drug that did the most damage in a short time frame was clearly AMT. Only 3 (well-spaced, moderate dosage) uses left me looking over the edge of the schizophrenic abyss. I was randomly spacing out and losing control, I'd have ideas and I couldn't really control them, paranoid thoughts about everything being bugged that half of me knew were bullshit, but only half. Not good at all. It took a good 6 months before I felt totally better.