I agree that it's not the ideal thing for you to do, but if you wanna do it, you're gonna do it. I can honestly say that I smoke weed, and have every day for the past 6+ years. Has it inhibited my growth? Not that I can tell, but maybe. Am I happy with my body regardless? At the moment, kind of (only cuz I haven't been to the gym in months - but for DIFFERENT reasons! lol) but in general, yes. I'm not exactly huge either, but I'm well above average and pretty fucking ripped, and when I'm training I'm more fit than 90% of the people in my gym. I used to smoke before I'd workout, but only one little hit. I wouldn't get super-stoned cuz it would make me really lethargic. I found that one small puff of some goodness would help me really get into my workout (kind of get in that zone, if you know what I mean) and push myself beyond my usual limits. I haven't done that lately only cuz I work out right after work, and don't like carrying shit to my job with me.
The stuff that was said earlier about not eating ideally when high was a poor generalization of "stoners". I find that if you're really serious about it, getting high will only make you eat more food that's good for you. I have several pothead friends who are HUGE. I know a lot of you will laugh, but Schwarzenegger smoked, and he was a 7-time Mr. Olympia champion. The bottom line is it all comes down to willpower. If you have the willpower to continue to train correctly and eat correctly while smoking every day, you won't have any problems. If smoking gets you sloppy and lethargic, or makes you lose your motivation or get out of your routine, you need to choose one or the other.