It comes from impurities that result from various things during production. Some say it's other things, like I've heard before that some chemists pour just a splash of sassafras oil into the final product to indicate that it was made from safrole during a time when there was a lot of imposter MDMA on the blackmarkets, all of them bright white and odorless (which, ironically, is how absolutely pure MDMA is described in the Merck Manual). Idk how much I believe this "oil splash" theory though. Another unlikely myth is that some MDMA is smuggled out of The Netherlands for international shipment dissolved in wine inside of wooden caskets. When it arrives at other countries, the MDMA is extracted out, but some of the wine tannins come along during the process and they're difficult and time-consuming to remove, so they just labelled it "purple MDMA". While inventive, it also seems kinda far-fetched, but who knows? Maybe.
More than likely it's the result of somewhat sloppy, large-scale chemistry; a lack of analytical tools, lack of oversight and zero quality control standards. For example, let's say that the chemist uses a little too much heat while isomerizing safrole into isosafrole. This causes a portion of the 3,4-methylenedioxy ring to break open, introducing a new 3-,4-dihydroxy species into the process. The chemist then forms the ketone from the allylbenzenes present in the solution, and because it's a time-consuming pain in the ass, the chemist doesn't bother to vacuum distill this ketone. It gets washed, and maybe even formed into a crystal with sodium meta-bisulfite as a purifying step, but the 3-,4-dihydroxy chemical just keeps rolling along with the intended 3,4-methylenedioxy portion. Everything is reductively aminated and we end up with some portion of MDMA and some portion of 3-,4-dihydroxymethamphetamine (DHMA), which is tan in color as a crystal, even when pure. This discolors the normally bright white crystalline salt of MDMA and we wind up with "champaign molly" that has a speedy kick to it courtesy of the DHMA. The experience may or may not be "meh" depending on the amount of DHMA present vs intentional MDMA. Or it may be so minuscule it only colors the crystal but doesn't affect quality much. And this is only one possibility.
Another setup may involve the clandestine chemist manufacturing their own methylamine, which also produces a portion of dimethylamine and trimethylamine, which if the chemist doesn't know how to separate it could potentially have a negative net effect on the end product. So they perhaps just use it as is, converting a portion of their PMK (aka: MDP-2-P) into MDDMA and another portion into MDTMA which both have a countering effect on MDMA. If cleaned properly, the other compounds will still be present, but also quite clean, and the end product may still appear bright white but be an overall disappointing experience. So discoloration doesn't necessarily prove anything about the experience or qualitative effects.