The reason people think that food poisoning is the cause of the hallucinations is because 9 times out of 10 if you get labs done after tripping on shrooms you would show that you have food poisoning. Psilocybin and psylocin are the cause for the hallucination and potential food poisoning is the cause for the upset stomach/vomiting.
How or why would a lab show that? It would be a false positive. You can get an upset stomach from mushrooms that were produced under semi-sterile conditions like the shrooms that were legally grown 'industrially' here in the netherlands for a long time. There aren't other bacteria etc on there upsetting your stomach. Not exactly positive whether it is the fibrous tissue or chemicals in the mushrooms but yeah there is something unrelated to the alkaloids there which can sometimes cause an upset stomach. Calling it food poisoning is way overstated in that respect, just like indigestion isn't a serious sickness. Much more likely a lab will call it poisoning / intoxication because the drugs themselves are considered toxic by their standards. Institutions like that usually wouldn't want you to function out of the ordinary including tripping.
If you accidentally eat shrooms (, never had any intention of tripping ever) and end up in a hospital or lab tested because of concerns, then they would have a point with that attitude.
Psilocin like compounds in pure form cause considerably less GI issues and/or much less frequently than mushrooms. They are not really the point here.
Someone around here also claimed mushrooms cause more tachycardia than psilocin, or rather: elevated HR that shouldn't ever be serious to healthy persons - "tachycardia" does suggest serious or unsafe.
Anxiety attacks and palpitations are a different matter altogether of course. Not sure how substantiated that claim was by the way.
Bottomline: there are other materials in mushrooms than the alkaloids which can be an issue, but I don't think it's anything to worry about. And similar things can happen with edible (as in not gourmet/choice but 'safe for consumption just not interesting') mushrooms like plenty of them. Sometimes your stomach doesn't like them. It takes a little more to be called food poisoning.
Yada yada yada, don't get crazy about the term 'poison' or 'intoxication', it's complicated as it is always purely a matter of dosage and how you measure seriousness of symptoms. With so-called poisons, the dosage to cause serious (by any standards) health problems is just very low. Other things can intoxicate you, even water, but it takes so much that it is rarely an issue at all.