Absolutely - dancing or stretching or just walking around, or sitting with your back straight, shoulders relaxed and breathing deep should all help.
But in my experience, yoga is by far the best way to respond to this: I wouldn't say 'cure' it, because once you've developed some experience in yoga or chi kung, or just meditative tripping, then you can start to feel these sensations in the back and elsewhere as positively pleasurable.
If you believe the theory that psychedelics work by opening up chakras (energy centres) in the body (which is arguably a crude model, compared to a proper understanding of neurochemistry and the nervous and hormonal systems, but is still useful for understanding what is going on), then you would absolutely EXPECT to feel weird sensations up and down your spine, and you would expect that if you don't make an effort to concentrate on your posture in a meditative way, or stretch / loosen up with yoga or some similar exercise then those sensations would feel a little uncomfortable.
Even if you don't believe that, the points made above in the thread about the general stimulation of the nervous system producing these sensations are still absolutely relevant.