Ok, if i understand correctly, Isomerism and Polymorphism aren't the same thing.
here's an image borrowed from a thread in 2010 ( i never read until now ) that I'm sure you remember:
We know ketamine has stereoisomers which are the same structure in a mirror image. Here we see two batches of crystalline ketamine. User reported different effects from each batch. At first it seems that the left is racemic and the right is a single isomer. However, what we are looking at are polymorphs NOT isomers! You can't see an isomer and you can't see purity, but you
can see a polymorph. The structural orientation of stereoisomers on a molecular level are mirror images, while the structure of molecules packed together into a crystal is the polymorph.
However, since appearance will not be indicative of stereoisomerism or purity, appearance means nothing. It seems you can have two different stereoisomers with the same crystal polymorph.
It seems to me that isomerism may alter effects in one way, while polymorphism will alter effects in another. A stereoisomer will fit into the receptor slightly differently, the same way a right hand in a left handed glove wouldn't fit perfectly. Then the shape of the crystal (polymorph) will, at the very least, affect the rate of absorption, thus creating differentiation in psychological response as the rate of metabolism changes.
I could be totally off but that's what i've gathered. LSD's isomers are all inactive, but pure LSD may exist in several polymorphs, potentially explaining the age old debate.