Firstly, it is a question whether you actually had serotonin toxicity or not. I assume you know serotonin is a chemical in your brain. The following are symptoms / toxic effects from having too much of it:
Cognitive effects: headache, agitation (easily irritated and getting worked up), hypomania (feeling insanely excited, feeling invincible, etc, extremely positive about things), mental confusion, hallucinations, coma
Autonomic effects: shivering, sweating, hyperthermia (overheating), hypertension (high blood pressure) , tachycardia (fast heartrate / pulse), nausea, diarrhea.
Somatic effects: myoclonus (muscle twitching), hyperreflexia (manifested by clonus), tremor (shaking, trembling).
Unfortunately a number of those are also possible common DXM effects. In any case if the above have gotten severe at some point you might have gotten serotonin syndrome (a medical condition of serious toxic effects from too much serotonin) or at least some toxicity of it.
On the long term, it is very hard to say and dependent on how severe the toxicity was but it can do damage to nerve cells that have to do with serotonin. Excitotoxicity for example which basically means you damage or kill nerve cells by getting it overexcited, for example by too much serotonin.
All of this can have any number of consequences, serotonin is involved with a hell of a lot of functions such as sleep and mood. These and many more could get disturbed.
I don't want to alarm you - my previous response was meant to say that what you did put you in danger of this but if you showed no signs of bad reactions to too much serotonin it seems you got off seemingly without damage.
You should just avoid such a combination, but worrying about brain damage that was already caused might really be overreacting.
Using DXM without SSRI's would be as safe for you as DXM is for anyone else. I don't understand why you ask. It is not like the SSRI's make you permanently unable to take DXM. SSRI's cause the serotonin system of your brain to behave differently. Because of that it can also cause you to react differently to certain drugs which is why a number of them should be avoided. When you discontinue SSRI's, this serotonin system slowly returns back to normal. When that has happened it should be like before.
I strongly recommend against getting off your medication, especially if it is only to trip on DXM. Discuss this with your doctor or psychiatrist, you got on them for a reason. If that reason is not valid anymore and the SSRI is no longer necessary that should be the reason to get off them and it should have nothing to do with DXM.
Hopefully it wasn't to scientific for you but some things just aren't simple. Look up a few words, you might learn something.