"Taking psychedelics" can mean so many things. It could mean taking 2.0g mushrooms on a calm, sunny day in the woods with friends, or it could mean taking DOI at a nightclub with folks you barely know, or it could mean doing DMT or Salvia in your room alone...
So many factors. And I would dare to say that the factors surrounding the ingestion of the psychedelic weigh far MORE on the outcome than the fact that you took a psychedelic. You see, many people get psychologically hurt or confused by the trip, and also many emerge with their depression or anxiety healed, at least temporarily.
This is a complex subject and it isn't at all a black-and-white answer.