It certainly can be. If you're prone to psychotic illness or have someone in the family who has one then it's particularly recommended to stay away from it. I think checking out erowid wouldn't be a bad idea; it can't hurt
https://www.erowid.org/chemicals/dxm/dxm_health1.shtml.
I seem to remember reading in there a while ago that a full weekend of binge-drinking did a similar amount of brain damage as a line of ketamine. I'm not going to try to find it, but I clearly remember reading this..
For some people (those who have treatment-resistant depression), the benefits might outweigh the negatives. But we're talking about a particular sect of society, not everyone. And then there's the whole problem of purity...
I would go as far to say that taking ketamine once a week (not in super-high doses and without other drug use) for a while might not be horrible for you, but what do I know? More so, what do doctors know? We haven't begun to broach the enigma that is the human brain, much less the brain and its interaction with foreign substances.
But yes, NMDA antagonists are known to simulate symptoms of schizophrenia in those without the disease, as the intoxicated display both "negative" and "positive" symptoms, whereas stimulant intoxication more displays "positive" symptoms.
Some people do indeed develop hallucinations that don't go away or permanent brain damage as a result of NMDA antagonist abuse. The problem is that it's often too late when you do the damage (to fix it), and there's no going back.
Happy sailing!