I think if you looked at me you'd probably be able to guess that I smoke weed at least occasionally, but that's only because I look a bit "alternative".
Other than that, I think it's pretty difficult to tell I take a wide and colourfull range of drugs, and that's the same with most people.
If you know what to look for you can spot a junkie, but other than that it's difficult to tell if people take drugs. Drug users come from all walks of life, it's difficult to stereotype "drug users" because different drugs are used by different kinds of people, and often "alternative" looking people don't use any drugs at all.