My psychiatrist said there has been no evidence connecting it to LSD and he thinks it's anxiety. I disagree because my anxiety is normal now.
So now I'm just really worried
^ Worrying and anxiety are one and the same. Your doctor is probably speaking a lot of sense, if you're worried about these visual disturbances you'll find it hard/impossible for them to go away as your mind will only amplify them further.
One time after one of my trips the enhanced vision the trip provided led to me noticing the floaters in my eye more, so I got this weird shining flowing effect in the bottom left hand corner of one of my eyes. I thought it was HPPD and I was really scared, it got more and more intense with each day. Then I went to see my opthalmologist, and they told me all that had happened was I became more aware of it and by focusing on it I made the problem seem worse - they told me to just ignore it as if it wasn't there.
So I did, a day or two later it was completely gone and hasn't come back
HPPD is a very real thing, but in my experience most people who report having HPPD don't. They simply either: A) Tripped recently so still have some lingering visuals (this is not HPPD, HPPD is when the visuals do not go away), or B) Their trip made them notice one of their visual defects more, and through focusing on it they're not allowing it to go away.
Try ignoring it for a while and see if that helps. Also, since it's possible that it
is HPPD, you should avoid any drugs - including cigarettes, alcohol, cannabis, and even caffeine, for at least a few weeks. Eat healthy, exercise, stay hydrated - and you should see improvement
Unfortunately if you're on any medication for your anxiety this could be the cause of the issue or could make it worse, but stopping medication abruptly just for that reason would be a bad idea - if you are on any medication you should talk to your doctor and see if changing it might help.