Similar to just about any emotion, the anxiety doesn't actually need a REASON behind it...if that makes sense. Personally a few different kinds of stimulants will trigger mild anxiety for me, most notably Clenbuterol HCl. Even though I've got nothing to worry about, I feel like there's something I SHOULD be worrying about. It's easy enough to ignore when you know for certain it's just the substance, but the emotion itself is still there. Mushrooms, while not exactly a stimulant, can have a very similar effect too. At one stage during my last trip, I remember my emotions changing almost every other second haha....one moment pure bliss, then fear, then confusion, then bliss again, suddenly I'm crying (lolwut), then anxiety, so on and so forth.
Make sense?