If you take MDMA while on an SSRI, it won't "fuck your brain", but you likely won't roll.
SSRIs work by blocking the serotonin reuptake transporter (SERT); this basically causes serotonin to accumulate in your brain since once it's released it can't be reuptaken back into the neuron that released it. It ends up sitting in the synaptic cleft, repeatedly stimulating your serotonin receptors until it's metabolized. MDMA works in a similar way by binding to SERT, but it reverses the transporter rather than blocking it, causing a flood of serotonin (as well as dopamine and norepinephrine, since it reverses DAT and NET too) to fill your brain.
The problem is that SSRIs and MDMA have to compete for the same binding site. SSRIs generally have higher binding affinity for SERT, so they win out and you end up not rolling (at least not in the full sense, since you'll still get the mild DA/NE kick). The exact results of taking MDMA and an SSRI can depend on a number of factors: the binding affinity of your particular SSRI, your dose of both the SSRI and the MDMA, the SSRI's half-life, your sensitivity to an increase in serotonin, etc. The result of this is that *some* people can take MDMA while on an SSRI and still end up "rolling" (keep in mind this is a subjective term, especially if you've never done it before), but the majority end up either having a muted roll or not rolling at all.
As for the "fucking your brain" thing: if you're taking an SSRI it's pretty likely that you're suffering from anxiety or depression. While an SSRI + MDMA won't necessarily worsen this, taking MDMA at all when you have a mental health condition may cause it to get worse.
Finally, I urge you not to abruptly discontinue your medication. Abruptly discontinuing an SSRI can lead to a host of side effects collectively called
SSRI discontinuation syndrome. You mentioned that your anxiety isn't bad, so I doubt you're going to have an episode if you discontinue, but there are plenty of people abruptly who go off of their medication and end up having a severe depressive bout or a panic attack. If you do want to discontinue your SSRI, talk to your doctor gradually decreasing your dose, and what kind of treatment you want to explore (if any) in its place.