YOu shouldn't take acid if you are an anxious person. You might not be psychotic, but you're on the meds for a reason.
A mind is a terrible thing to waste.
I dunno, I am a fairly anxious person, but I feel like I have massively benefitted from taking psychedelics. I do try to ensure an optimum and setting though, and always have a benzo on hand to take down the trip's intensity a notch (or abort it outright) when things feel like they're getting out of control.
That said, I kind of agree with your concerns here. Going cold turkey off an antipsychotic to be able to trip may not be the best idea.
Mirtazapine is an antagonist at 5HT2A (so it will block the effects of LSD), and zyprexa even is an inverse agonist (so it is literally the diametric opposite of LSD), and they both have very long half-lives (~30 hours on average), so OP would probably need more than 3 days to get this stuff out of their system.
On the other hand, abruptly quitting 15 mg's of zyprexa could lead to rebound effects, so that once the meds are out of your system, you're experiencing more emotional lability than normal, during which period it might be a *really* bad idea to trip.
OP, if you're taking olanzapine to sleep, you might consider asking your doctor about carefully reducing the dose, and/or at least partially replacing it with a "gentler" drug. Don't get me wrong, olanzapine is an excellent drug for actual psychosis/bipolar/schizophrenic illness, but as a sleep medication, it is pretty much overkill; especially at a dose as high as 15mg/day.